be less scrict about hdfs paths.

urlsplit the HDFS path so that fully-specified urls,
e.g. "s3n://" and "hdfs://namenode:port/foo/bar" work.
Merge pull request #2 from jcrobak/more-liberal-hdfs-paths

be less scrict about hdfs paths.
Added bin/luigid and fixed some minor issues with README.md

Change-Id: I8f04cbf91e7408f9a3aacd6e3397dd64d4d9d8c3
Reviewed-on: https://gerrit.spotify.net/gerrit/32969
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Prevent infinite recursion in InputPipeProcessWrapper/OutputPipeProcessWrapper

http://users.cs.cf.ac.uk/J.P.Giddy/python/gotcha/getattr.html

Change-Id: Ia690ba3b52f5f33320e654d85bb4d519c59d5c29
Reviewed-on: https://gerrit.spotify.net/gerrit/33000
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Fixed problem with tests caused by the global register. Only occurred if tests executed in a certain order

Change-Id: Ib691f4fd6bcf46f9d0f63c6a9a4917b8fb10e152
Reviewed-on: https://gerrit.spotify.net/gerrit/33003
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Add equality operator for DateInterval to allow defaults. Added unit test that broke previous build

Change-Id: I919b8aec66e1f4878458379ee4f8b23e06a8ba0a
Reviewed-on: https://gerrit.spotify.net/gerrit/32793
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Cleanup tmp directory + slightly improved exception fetching

Change-Id: Iba7a562750cd105c6cbaf41c696250347134ca90
Reviewed-on: https://gerrit.spotify.net/gerrit/32252
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
re-enable the visualizer

Change-Id: I7cac8408fbe3ff9c7688e04cbf7d20893bb4d54c
Reviewed-on: https://gerrit.spotify.net/gerrit/33156
Reviewed-by: Elias Freider <freider@spotify.com>
added some more comments and exception handling to hdfs.exists, because i've got pwned by the same behavior twice now

Change-Id: Ica0c309b63361a256fb16e3fc24a2d564e89eb9e
Reviewed-on: https://gerrit.spotify.net/gerrit/33046
Reviewed-by: Elias Freider <freider@spotify.com>
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
removed @luigi.expose from all tests

Using luigi.expose (and therefore the global state) lead to problems with unit tests when they were executed in certain order. This didn't manifest itself on TeamCity, but on my laptop etc.

Replaced with luigi.build in some places where we weren't testing cmdline integration

Change-Id: I6b7a3cc3d81eec94c56057d6df9cbc4ab57f7a14
Reviewed-on: https://gerrit.spotify.net/gerrit/33015
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
More randomness in workers

Change-Id: I82154491576fa724d87ef30668f015f1c58fa93a
Reviewed-on: https://gerrit.spotify.net/gerrit/34115
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Refactored notification emails. Crashes in complete() doesn't abort the entire scheduling process anymore.

Change-Id: I40f7a30bdf34fecbc6457f6911942b36e48997be
Reviewed-on: https://gerrit.spotify.net/gerrit/34538
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Allowing non-boolean returns from the complete method. This should fix endsongsource and a few other sources

Change-Id: I31d8ef1161534c8f03c9fffb92afa391d0d0ea7c
Reviewed-on: https://gerrit.spotify.net/gerrit/34698
Reviewed-by: Anders Nyman <nyman@spotify.com>
Commented out broken test

Change-Id: Ib3f6242b5ecce61f4754090667776c423a22228a
Reviewed-on: https://gerrit.spotify.net/gerrit/34699
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Adding dummy __enter__ method should hopefully fix a Python 2.7 problem

Change-Id: If28b1e41ceb2e95f770c5c7c226da93c9ea3d156
Reviewed-on: https://gerrit.spotify.net/gerrit/34887
TC-Compile-And-Test: teamcity teamcity <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Fixed problem in previous commit (sorry, forgot that the HDFS tests are not run automatically)

Change-Id: Id4363d7ac0d7de45ac16e80b7ef9d7f133f6dbef
make luigid executable
consolidate tmpfile logic

Also use tempfile so that "/tmp" isn't hard-coded.
make hdfs cdh3 compatible
Merge pull request #6 from jcrobak/luigid-exec

make luigid executable