sstephenson/ruby-build — pull request #240 by sstephenson (2012-11-15 16:44:04+00:00)
Title: Faster package downloads via S3

I'm tired of waiting on slow downloads from ftp.ruby-lang.org! This branch makes downloading packages much faster by pulling the files from S3 instead. :zap:

It's two features rolled into one. Here's the breakdown:

**Checksum support**. You can now optionally specify the MD5 checksum of a package file as an anchor on its URL. When you do, ruby-build will compare this checksum to the checksum of the downloaded file and refuse to continue if the checksums do not match.

The package URLs in every definition have been updated to include checksums. A before-and-after example for the libyaml package:
```diff
-install_package "yaml-0.1.4" "http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz"
+install_package "yaml-0.1.4" "http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz#36c852831d02cf90508c29852361d01b"
```

Checksumming gives us a way to ensure with reasonable certainty that the file we downloaded is the one we asked for. It also gives us a short, unique key by which we can identify the file, leading us to...

**Mirror support**. If a package URL has an associated checksum, ruby-build will first try to download the file from the URL specified in the `RUBY_BUILD_MIRROR_URL` environment variable.

By default, this variable points to the Downloads root of my [ruby-build-download-mirror](https://github.com/sstephenson/ruby-build-download-mirror/downloads) repo. GitHub Downloads are hosted on S3, but you can use a different mirror URL if you'd like to host packages on your internal network, for example.

Each mirrored file is named by its MD5 checksum. If the mirror request fails, or if the checksum of the downloaded file does not match, ruby-build will fall back to the original URL specified in the definition.

---

Remaining work to be done:
* ruby-build should degrade gracefully if you do not have `md5` or `md5sum` installed. It should behave as before: no checksum verification and no mirror support.
* We should let you pass a command-line option (or equivalent) to bypass the download mirror. I don't know what it should be called.
* The "Downloading..." message could be improved to better explain that the file is coming from a mirror or not.