colinmarc/sequins — pull request #1 by colinmarc (2014-10-05 01:03:46+00:00)
Title: Create a 'Backend' interface, and add an S3Backend (wip)

A 'Backend' represents a filesystem that we can sync locally. This adds general
S3 support, meaning you can pass in s3:// urls as file paths and it'll
download them before indexing.

This also adds a new flag, --check-for-success, which will make sequins only
accept versions with a _SUCCESS file in them. This is useful to avoid loading
partial results from hadoop.

Finally, there are some other minor fixes.