Introduce the hdfs_scheme
PyCurl: Handle temporary redirect when uploading a file.

It is convenient to issue a single upload and have the files uploaded even if
the server responds with e temporary redirect.  However, we should note that
this is very easy to misuse.  The main sender might not want to receive the
actual payload.  If that is the case, we should not send the payload in the
first request.
JobPack: Introduce the save_info field.

This field will be used for transferring the meta-data needed for saving the
results of a job into hdfs.
JobCoordinator: Factor out save_ddfs from finish_pipeline.

This is a non-functional change.
DdfsHttp: Make the http request standard.

The HTTP RFC dictates that PUT should contain the Path and The Query.  It was
not important for the internal ddfs usage but if we'd like to use this http
client for other purposes, in this case hdfs.
Hdfs: Add support for saving the results of the jobs to HDFS.

This support is not complete yet and should not be used in production.  An
example has been added for a job that reads the input from hdfs and writes the
data in hdfs.
HttpClient: Extract http_client module out of ddfs_http.

The operations in this module are general and can be used for filesystem
operations other than ddfs.
Merge pull request #454 from pooya/shayan/hdfs

Shayan/hdfs