Support setuptools

Fall back to distutil if setuptools isn't around. Useful for
bundling as an egg with bdist_egg, for example.
Merge pull request #10 from jcrobak/setuptools

Support setuptools
Cleanup. Adding PlainDir format for HdfsTarget
* Avoid double logging when running several interfaces
* Don't send error emails during in unit tests

Change-Id: Id12dd4b77bed111f8c0dd3f4f1519ff00f731924
Reviewed-on: https://gerrit.spotify.net/gerrit/36456
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
improved write access functionality to work as intended with newer versions of Hadoop

Change-Id: Iaa94963d3eef1f6ee37ab6f77757c18843ba12c0
Reviewed-on: https://gerrit.spotify.net/gerrit/36944
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Added method for adding config overrides

Change-Id: I98ebad8f962e6e0eb59fe2fb17c2cc88c15994f8
Reviewed-on: https://gerrit.spotify.net/gerrit/36770
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Fix hadoop_jar arg handling.

* HadoopJar was only doing Target-> path coercion if atomic output
  was enabled. This fixes it to always do it.
Merge pull request #15 from jcrobak/fix-hadoop-jar-output

Fix hadoop_jar arg handling.
Hive task definition

HiveQueryTasks run a query in Hive. Their output is a HiveTableTarget
or a HiveTablePartitionTarget, depending on the query. HiveTableTask
is an ExternalTask that checks if a particular table/partition exists.
Some more fixes to hive commands.

Summary:
* Adds ability to retrieve paths for tables.
* extract out hive_cmd() function.
* fix a syntax error.

Notes:
* I think having the new hive_cmd() should make the targets fairly
  easy to unit test. We can discuss if you'd like.
Add variants of config.get for int/bool/float.

I was having issues with get and converting to a bool, so I thought
it might be a good idea to bubble up this functionality from from
RawCOnfigParser.
Merge pull request #17 from jcrobak/getvariants

Add variants of config.get for int/bool/float.
Simple postgres integration using psycopg2.
Not suitable for too large data sets. We should implement a similar task with sqoop functionality for exporting large data sets from hadoop.
psycopg2 shouldn't be a dependency for luigi unless you use this functionality, so imports are try-except guarded

Change-Id: I038495614259688ab3554964d298bf2c572b3fee
Reviewed-on: https://gerrit.spotify.net/gerrit/37669
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Accept strings instead of tuples for column names when not using column types

Change-Id: If117b3d7b09cd75d4c2138cac949ce13167d2fcc
Reviewed-on: https://gerrit.spotify.net/gerrit/39440
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Replace deprecated jobconf by -D. Make repr_reader/repr_writer overridable. Fix lint warnings

Change-Id: I02f2be494e450758b1b2863b8cb1bd7535532a2d
Reviewed-on: https://gerrit.spotify.net/gerrit/39783
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Adding support for showing the modification time when listing files

Change-Id: I2cb4a79fda63d58bfe2cdcf3c6cb13a0d491292e
Reviewed-on: https://gerrit.spotify.net/gerrit/40283
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
include task failures in traceback, increased timeout, fixed bug with <10 tasks

Change-Id: I522e5e2abcee83be18073d826fb305c94fb79fd6
Reviewed-on: https://gerrit.spotify.net/gerrit/38213
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
respect KeyboardInterrupt

Change-Id: Ib4295320d6444cff9fb471ee54c126781b6ff387
Reviewed-on: https://gerrit.spotify.net/gerrit/40747
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Added pseudo-random sort of lines when running in local mode

Change-Id: Ie6e6191b6812901201ae424168bc77642ab80063
Reviewed-on: https://gerrit.spotify.net/gerrit/39928
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Making it possible to configure the local tmp directory for hadoop jobs

Change-Id: I9a47b733d2a8d3f874c1c96800362d6a38e27a94
Reviewed-on: https://gerrit.spotify.net/gerrit/41637
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Applied freider's change, fix whitespace, refactor

Added partition_spec and table_location as static methods,
changes 2 spaces to 4 spaces for indenting.