Yelp/mrjob — pull request #443 by irskep (2012-04-15 22:43:31+00:00)
Title: Filesystem functionality refactor

This branch moves all filesystem-related code into `mrjob.fs.local`, `mrjob.fs.s3`, `mrjob.fs.ssh`, and `mrjob.fs.multi`. There are many new tests under `tests.fs`.

I'll test this live some time this week or next, whenever I get a chance.

Downsides:

* `MRJobRunner` needs a custom `__getattr__` implementation for backward compatibility
* `mockssh` and `mockhadoop` are uglier than they were before to support the new tests' monkey-patching of `Popen` (however this method is MUCH faster than actually copying files and calling `Popen`)

Possible future work:

* `HadoopJobRunner` still contains a manual call to `invoke_hadoop(['-fs', 'mkdir', path])` which should be pulled out at some point
* More unit tests for `MultiFilesystem`, though it's currently covered pretty well by the existing integration tests
* Add `mrjob.fs` to the public documentation