Hive task definition

HiveQueryTasks run a query in Hive. Their output is a HiveTableTarget
or a HiveTablePartitionTarget, depending on the query. HiveTableTask
is an ExternalTask that checks if a particular table/partition exists.
Some more fixes to hive commands.

Summary:
* Adds ability to retrieve paths for tables.
* extract out hive_cmd() function.
* fix a syntax error.

Notes:
* I think having the new hive_cmd() should make the targets fairly
  easy to unit test. We can discuss if you'd like.
Simple postgres integration using psycopg2.
Not suitable for too large data sets. We should implement a similar task with sqoop functionality for exporting large data sets from hadoop.
psycopg2 shouldn't be a dependency for luigi unless you use this functionality, so imports are try-except guarded

Change-Id: I038495614259688ab3554964d298bf2c572b3fee
Reviewed-on: https://gerrit.spotify.net/gerrit/37669
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Accept strings instead of tuples for column names when not using column types

Change-Id: If117b3d7b09cd75d4c2138cac949ce13167d2fcc
Reviewed-on: https://gerrit.spotify.net/gerrit/39440
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Replace deprecated jobconf by -D. Make repr_reader/repr_writer overridable. Fix lint warnings

Change-Id: I02f2be494e450758b1b2863b8cb1bd7535532a2d
Reviewed-on: https://gerrit.spotify.net/gerrit/39783
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Adding support for showing the modification time when listing files

Change-Id: I2cb4a79fda63d58bfe2cdcf3c6cb13a0d491292e
Reviewed-on: https://gerrit.spotify.net/gerrit/40283
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
include task failures in traceback, increased timeout, fixed bug with <10 tasks

Change-Id: I522e5e2abcee83be18073d826fb305c94fb79fd6
Reviewed-on: https://gerrit.spotify.net/gerrit/38213
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
respect KeyboardInterrupt

Change-Id: Ib4295320d6444cff9fb471ee54c126781b6ff387
Reviewed-on: https://gerrit.spotify.net/gerrit/40747
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Elias Freider <freider@spotify.com>
Added pseudo-random sort of lines when running in local mode

Change-Id: Ie6e6191b6812901201ae424168bc77642ab80063
Reviewed-on: https://gerrit.spotify.net/gerrit/39928
Reviewed-by: Erik Bernhardsson <erikbern@spotify.com>
Making it possible to configure the local tmp directory for hadoop jobs

Change-Id: I9a47b733d2a8d3f874c1c96800362d6a38e27a94
Reviewed-on: https://gerrit.spotify.net/gerrit/41637
TC-Compile-And-Test: Team City <teamcity@spotify.com>
Reviewed-by: Anders Nyman <nyman@spotify.com>
Applied freider's change, fix whitespace, refactor

Added partition_spec and table_location as static methods,
changes 2 spaces to 4 spaces for indenting.
Fixes indentation error
Merge pull request #16 from JoeEnnever/master

Hive task definition
Fixed a couple of hive-related bugs

* Fixed incorrect order of arguments in ExternalHiveTask
* Using dictionary parameters broke with instance cache since dict is
not hashable