add S3DownloadStrategy

downloads tarballs from public and private S3 buckets

Closes Homebrew/homebrew#22779.

Signed-off-by: Adam Vandenberg <flangy@gmail.com>

Library/Contributions/example-formula.rb

@@ -26,6 +26,7 @@ class ExampleFormula < Formula
   #     `:curl` (normal file download. Will also extract.)
   #     `:nounzip` (without extracting)
   #     `:post` (download via an HTTP POST)
+  #     `S3DownloadStrategy` (download from S3 using signed request)
   #     `UnsafeSubversionDownloadStrategy` (svn with invalid certs)
   url 'https://some.dont.provide.archives.example.com', :using => :git, :tag => '1.2.3'
 

Library/Contributions/manpages/brew.1.md

@@ -436,6 +436,14 @@ can take several different forms:
 
 ## ENVIRONMENT
 
+  * AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY:
+    When using the S3 download strategy, Homebrew will look in
+    these variables for access credentials (see
+    <http://docs.aws.amazon.com/fws/1.1/GettingStartedGuide/index.html?AWSCredentials.html>
+    to retrieve these access credentials from AWS).  If they are not set,
+    the S3 download strategy will download with a public
+    (unsigned) URL.
+
   * BROWSER:
     If set, and `HOMEBREW_BROWSER` is not, use `BROWSER` as the web browser
     when opening project homepages.

Library/Homebrew/download_strategy.rb

@@ -254,6 +254,42 @@ class LocalBottleDownloadStrategy < CurlDownloadStrategy
   end
 end
 
+# S3DownloadStrategy downloads tarballs from AWS S3.
+# To use it, add ":using => S3DownloadStrategy" to the URL section of your
+# formula.  This download strategy uses AWS access tokens (in the
+# environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY)
+# to sign the request.  This strategy is good in a corporate setting,
+# because it lets you use a private S3 bucket as a repo for internal
+# distribution.  (It will work for public buckets as well.)
+class S3DownloadStrategy < CurlDownloadStrategy
+  def _fetch
+    # Put the aws gem requirement here (vs top of file) so it's only
+    # a dependency of S3 users, not all Homebrew users
+    require 'rubygems'
+    begin
+      require 'aws-sdk'
+    rescue LoadError
+      onoe "Install the aws-sdk gem into the gem repo used by brew."
+      raise
+    end
+
+    if @url !~ %r[^https?://+([^.]+).s3.amazonaws.com/+(.+)$] then
+      raise "Bad S3 URL: " + @url
+    end
+    (bucket,key) = $1,$2
+
+    obj = AWS::S3.new().buckets[bucket].objects[key]
+    begin
+      s3url = obj.url_for(:get)
+    rescue AWS::Errors::MissingCredentialsError
+      ohai "AWS credentials missing, trying public URL instead."
+      s3url = obj.public_url
+    end
+
+    curl s3url, '-C', downloaded_size, '-o', @temporary_path
+  end
+end
+
 class SubversionDownloadStrategy < AbstractDownloadStrategy
   def initialize name, resource
     super

share/man/man1/brew.1

@@ -471,6 +471,10 @@ Homebrew can install formulae via URL, e\.g\. \fBhttps://raw\.github\.com/mxcl/h
 .SH "ENVIRONMENT"
 .
 .TP
+AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
+When using the S3 download strategy, Homebrew will look in these variables for access credentials (see \fIhttp://docs\.aws\.amazon\.com/fws/1\.1/GettingStartedGuide/index\.html?AWSCredentials\.html\fR to retrieve these access credentials from AWS)\. If they are not set, the S3 download strategy will download with a public (unsigned) URL\.
+.
+.TP
 BROWSER
 If set, and \fBHOMEBREW_BROWSER\fR is not, use \fBBROWSER\fR as the web browser when opening project homepages\.
 .