Add ability to read from s3.
Add iteration of s3 files
Issue #137: Adding command line parameters to global config so that logging is configurable
Merge pull request #187 from mortardata/cmiller_s3_open_r

Updated S3Target to allow open('r') for an S3 path.
Merge pull request #157 from gf-atebbe/logging

Issue #137
Disabling gzip compressions since it breaks the browser

Change-Id: Ia2c1a7964ab6df674ece5a67994c79508aa396b4
Reviewed-on: https://gerrit.spotify.net/gerrit/87762
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Lars Albertsson <lalle@spotify.com>
Merge pull request #219 from jakobalander/master

Disabling gzip compressions since it breaks the browser
Add snakebite support to improve HDFS operation performance
Replacing inheritance with decorators
Replacing inheritance with decorators in luigi.util

- Added docstring to @inherits (thanks, alexbw) and deprecated the Derived class
- Added a @requires decorator
- Added some helper method for tasks using @inherits
- Added a @copies decorator and deprecated the Copy class
- Added a @delegates decorator and deprecated the CompositionTask class
Merge pull request #218 from erikbern/decorator-fixup

Refactored luigi.util and deprecated Copy, Derived etc
Add a command line argument to pass in the logging configuration file
Merge pull request #220 from alanbbr/master

Add snakebite support to improve HDFS operation performance
Refactor worker#run to prepare fix for #222

Break up worker#run into smaller methods and give each a meaningful
name so it's easier to figure out what it's doing and allow code reuse.

Collapse duplicate if constructs.
luigi-history configuration option for writing stuff to Hadoop job output directory

Specifically for tracking the job ID in a configuration where no history is stored in the output directory by Hadoop.
Let worker wait for running tasks to complete.

If

- running tasks is non-empty, then another worker is running my tasks
- n_pending_tasks is non-zero, then I have something to do after she is done

And I shall wait for `wait_interval` seconds (with random jitter).

This adds the following configuration options:

- worker-keep-alive [boolean]
- worker-wait-interval [int]
Merge pull request #224 from spotify/uldis/history

luigi-history configuration option for writing stuff to Hadoop job outpu...
Merge pull request #225 from jimjh/worker-wait

Let worker wait for tasks running on other workers to complete.
Some very small cleanups: Remove unused imports, break long lines, add newline at end of file. Smallish things..
Merge pull request #229 from rantav/cleanups

Some very small cleanups: Remove unused imports, break long lines...
Clarify README.md wording for Tasks with multiple output Targets.

Based on luigi-user post https://groups.google.com/d/msg/luigi-user/YOCf9fJcqUE/Po2q0z0TV9AJ