Fixes visualizing a task graph with missing tasks

They show up as UNKNOWN (like luigi-grep). Prints out a warning message
when this occurs, though it can be quite noisy.

Fixes https://github.com/spotify/luigi/issues/122
Refactored the task history support added by jennever@foursquare.com
Changing HadoopJobRunner to use move_dir when moving results to final output since move_dir is atomic
Revert "Changing HadoopJobRunner to use move_dir when moving results to final output since move_dir is atomic"

This creates issues as well since we will have an empty output dir that will trigger other jobs to run.

This reverts commit daa6e6368706759622e60db98c0ec73c7198fc0c.
Retry if ping of scheduler fails

* Makes ping interval configurable
* Also adds accessible stop() method to stop workers' keep-alive-threads
Additional test for new Worker.stop method
Refactor Worker.add

Refactored the Worker.add method to make it more readable. Breaks out
some of the email/error logic for across the module.
Flatten recursive add into an explicit list-based stack
Remove __del__ on worker as it's undeterministic
Improve parameter parsing so it doesn't break the scheduler on weird parameter values

Adds failing unit test exposing broken parameter split
Adding a manifest file for distutils so that bin/luigid is included in the source tarball that gets built
Hadoop binary path can now be specified in config.  Sets are converted to tuples in Task class.
Additional unit tests in the hdfs module and a new hive client command class supporting the Apache release of Hive, which returns a non-zero exit status if tables and partitions dont exist