separate functions for hive script/cmd.

Expose a new function for running commands stored in a file
rather than as a string.
Give better message in case of missing param.

We've had a number of user trip up because you have to specify
e.g. --foo-bar rather than --foo_bar. So I think this'll help
avoid the common confusion.

Example error message looks like:

luigi.parameter.MissingParameterException: No value for
'date_interval' (--date-interval) submitted and no default value
has been assigned.
Merge pull request #27 from jcrobak/better-error-msg

Give better message in case of missing param.
Fix incompatibility with latest graphviz svg format

Change-Id: I8a0f0d31166e273384de6809ba2a784bcf079903
Reviewed-on: https://gerrit.spotify.net/gerrit/45144
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
convenience method for delegating jobs to subtasks

Change-Id: Icfb1a6aa136954c962c0621bba70fcf813811c0f
Reviewed-on: https://gerrit.spotify.net/gerrit/45426
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
fixed bug with LocalTarget and is_tmp where files were created at instantiation

https://github.com/spotify/luigi/issues/29

Change-Id: Ibc6ad08a1084605c549ed00be90d84d8d3c700de
Reviewed-on: https://gerrit.spotify.net/gerrit/46388
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Add final execution hooks to luigi, as well as batched counter increments

Change-Id: I6aba342a446a8b039ac6fcd7379cfce9f859f659
Reviewed-on: https://gerrit.spotify.net/gerrit/43537
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>

Conflicts:
	.gitignore
	luigi/hadoop.py
Merge pull request #20 from jcrobak/hive-script

separate functions for hive script/cmd.
Fix failure tracking for non-streaming jobs.

* move the @staticmethods to luigi.hadoop namespace.
* refactor all uses of those functions.
* clean up some random lint errors and imports.
Merge pull request #31 from jcrobak/fix-url-tracking

Fix failure tracking for non-streaming jobs.
introduce ABCMeta for Target and Task.

Task's metaclass inherits ABCMeta and Target uses ABCMeta as its
metaclass. This lets you define abstract targets and tasks that
can't be instantiated directly.

I have a few simple tests to demonstrate the expected usage and
to make sure that things still work ok.

Open question: Does it make sense for Target.open() to exist? It
only seems to be applicable to HDFS and File Targets (maybe an
OpenableTarget interface?). Or should we leave it without the
@abstractmethod annotation.
Merge pull request #32 from jcrobak/abcmeta

introduce ABCMeta for Target and Task.
adding copy command

Change-Id: I0898a0e273d5e567a2fd947731200788f451484f
Reviewed-on: https://gerrit.spotify.net/gerrit/46993
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>

Conflicts:
	luigi/hdfs.py
Use LocalJobRunner for MapReduce if there are non-HdfsTarget outputs

Change-Id: Ie406af4a5e479dc4bb7ff8c6ecb8469713b3fe99
Reviewed-on: https://gerrit.spotify.net/gerrit/44957
Reviewed-by: Elias Freider <freider@spotify.com>
Renamed _repr_reader/writer and created an alternative reader/writer using pickle

Change-Id: If3c1873dad874bf3c7f3f50ea1dff9dd47dbaf8d
Reviewed-on: https://gerrit.spotify.net/gerrit/47106
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Reviewed-by: David Whiting <davw@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Makes scheduler configurable, no retry failed task

luigi.server.RPCHandler passes arguments from luigi.interface.get_config
to the scheduler it creates.

Passing a negative retry_delay to CentralPlannerScheduler will cause
tasks to never be retried, until a new luigi invocation. This is needed
for our very long running pipelines. A task that continually fails uses
up cluster resources and luigi worker processes.
Merge pull request #33 from JoeEnnever/serverconfig

Makes scheduler configurable, no retry failed task
Include subprocess out,err in run_track_hadoop_job
Just write to file, print it out after

Refactored to be less indented
Added test
Fix a log typo.

Double 'is' removed.