separate functions for hive script/cmd.

Expose a new function for running commands stored in a file
rather than as a string.
add scheduler_port param
Merge pull request #21 from bitly/port_param

add scheduler_port param
Better error message if requires() returns non-Tasks

Change-Id: Ieaf0f450dd3708772ffcd2f39700200058a3c301
Reviewed-on: https://gerrit.spotify.net/gerrit/43770
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Moved hdfs operations into HdfsClient class, added aliases so they can be used as module methods as before.

Change-Id: I972e45e595ab3af035d7c66034d40791b15b2b75
Reviewed-on: https://gerrit.spotify.net/gerrit/43945
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Reviewed-by: Fabian Alenius <fabian@spotify.com>

Conflicts:

	luigi/hdfs.py
add skip_trash option to remove.

Add the ability to pass -skipTrash to hadoop fs -rm command line.
Merge pull request #23 from jcrobak/add-skip-trash

add skip_trash option to remove.
Update README.md

Rewrote introduction

TODO: right now the image link refers to /erikbern/luigi, fix this if/when the PR is merged
Merge pull request #25 from erikbern/master

Updated introduction
Refactor of how CopyToTable copies data into pg

- New method copy(self, cursor, tmp_file) that calls
  cursor.copy_from, to allow subclasses to overwrite
- Overridable column separator for the tmp_file, instead
  of always using \t
Merge pull request #22 from JoeEnnever/master

postgres.CopyToTable refactor
Using hadoop fs -ls -d for testing file existence, instead of hadoop fs -test -e which is semi-broken

Added unit test to test connection errors which would previously have been seen as file non-existence
Improved exception messages from other hadoop cli calls

Change-Id: I5a0e37023777663cfdca74d95eeeb4f19b865b61
Reviewed-on: https://gerrit.spotify.net/gerrit/42362
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>

Conflicts:

	luigi/hdfs.py
New test that sets up a server in the background etc.

Trying to find a bug, but it's impossible. At least we have some new tests :)

Change-Id: I7a3fa8c1fb6716acb943a5dd533fcef81d8a7d57
Reviewed-on: https://gerrit.spotify.net/gerrit/42836
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>

Conflicts:

	debian/control
Fix file transactionality issue when using the with keyword on LocalTarget

Change-Id: Ib60adbf71bdc507104b9757b117f9f2a8f867d4f
Reviewed-on: https://gerrit.spotify.net/gerrit/44956
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Improved log messages for interface/worker execution

Change-Id: I0aa2670b66aeb6d795da9cb85466b4e838dae4ab
Reviewed-on: https://gerrit.spotify.net/gerrit/44958
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Give better message in case of missing param.

We've had a number of user trip up because you have to specify
e.g. --foo-bar rather than --foo_bar. So I think this'll help
avoid the common confusion.

Example error message looks like:

luigi.parameter.MissingParameterException: No value for
'date_interval' (--date-interval) submitted and no default value
has been assigned.
Merge pull request #27 from jcrobak/better-error-msg

Give better message in case of missing param.
Fix incompatibility with latest graphviz svg format

Change-Id: I8a0f0d31166e273384de6809ba2a784bcf079903
Reviewed-on: https://gerrit.spotify.net/gerrit/45144
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
convenience method for delegating jobs to subtasks

Change-Id: Icfb1a6aa136954c962c0621bba70fcf813811c0f
Reviewed-on: https://gerrit.spotify.net/gerrit/45426
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
fixed bug with LocalTarget and is_tmp where files were created at instantiation

https://github.com/spotify/luigi/issues/29

Change-Id: Ibc6ad08a1084605c549ed00be90d84d8d3c700de
Reviewed-on: https://gerrit.spotify.net/gerrit/46388
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>