Refactor webhdfs.py

I refactored the whole file using the hdfs library instead of whoops because apparently whoops has some problems when uploading files.

I also added a basic WebHdfsTarget that provides basic read and write operations. The WebHdfsTarget is based on the S3Target. A simple example would be:

from luigi import webhdfs

target = webhdfs.WebHdfsTarget('TEST/file.py')

output = target.open('w')
output.write('pew pew pew\npow pow pow\n')
output.write('piw piw piw\npow pow pow')
output.close()

target = webhdfs.WebHdfsTarget('TEST/file.py')
input = target.open('r')
print input.read()
print [i for i in input]
input.close()

Output:

pew pew pew
pow pow pow
piw piw piw
pow pow pow
['pew pew pew', 'pow pow pow', 'piw piw piw', 'pow pow pow']
Adding support for basic auth for task esindex.CopyToIndex & target
esindex.ElasticsearchTarget & test coverage

Allow Elasticsearch tests with HTTP basic auth.

Added test support for HTTP basic auth and factored out ES settings.

To use a non-standard host and port, use `ESINDEX_TEST_HOST`,
`ESINDEX_TEST_PORT` environment variables to override defaults.

To test HTTP basic authentication use `ESINDEX_TEST_HTTP_AUTH`.

Example running tests against port 9201 with basic auth:

    $ ESINDEX_TEST_PORT=9201 ESINDEX_TEST_HTTP_AUTH='admin:admin' nosetests test/_esindex_test.py
Add a lock for when the process forks

Deadlocks were occuring with DNS lookups because of the following case:

"The non-obvious problem in this approach is that at the moment of the
fork(2) call some threads may be in critical sections of code, doing
non-atomic operations protected by mutexes. In the child process the
threads just disappears and left data half-modified without any
possibility to "fix" them, there is no way to say what other threads
were doing and what should be done to make the data consistent.
Moreover: state of mutexes is undefined, they might be unusable and the
only way to use them in the child is to call pthread_mutex_init() to
reset them to a usable state. It's implementation dependent how mutexes
behave after fork(2) was called. On my Linux machine locked mutexes are
locked in the child."
[1] http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them

This thread-fork issue can be replicated quickly by having a background
thread call getaddrinfo() repeatedly, and then have the main thread
start a child process which itself calls getaddrinfo().
Timing-dependent, the child process may hang in a locked state. Example
script can be found here:
https://gist.github.com/andpol/ca7432afc09b1bc914f5
Merge pull request #504 from danielfrg/patch-1

Refactor webhdfs.py
Annotating tests needing the minicluster
Merge pull request #535 from anyman/master

Annotating tests needing the minicluster
Add support for writing a string directly to s3
Updating debian/control with dependencies
S3Client put_string testcase added
Merge pull request #538 from anyman/master

Updating debian/control with dependencies
Merge pull request #532 from dvxam/dvxam/es-basic-auth-support

Adding support for basic auth for task esindex.CopyToIndex && target
Merge pull request #534 from andpol/master

Add a lock for when the process forks
Merge pull request #537 from mojn/master

Add support for writing a string directly to s3
Use the new build env on Travis

faster vms, more ram, more cpu, faster boot times,

docs coming soon
cat an error log if it fails
Merge pull request #539 from joshk/patch-1

Use the new build env on Travis