Yelp/mrjob — pull request #474 by irskep (2012-06-08 22:00:35+00:00)
Title: Refactor: precursor to multiple languages

This branch splits `mrjob.job` into `mrjob.job` and `mrjob.run`. I left a few things undone pending verification that we want to commit to this path.

`mrjob.run` contains the `Launcher` class. The purpose of `Launcher` is to start a **runner** from the command line, but not to run the task itself. The purpose of `MRJob` is to execute tasks in a Hadoop Streaming context, though it subclasses from `Launcher` for backward compatibility.

The only functional change here is that you can now do this:

python -m mrjob.run mrjob/examples/mr_word_freq_count.py [normal args]

I made `mr_word_freq_count.py` a standalone executable during testing, but it probably isn't necessary.

Future work:

* Rename or remove `python_bin` so we can support any executable as the runner
* Document interface for streaming executables
* Write examples in other languages

I was able to get a self-contained Python 3 job running *without converting mrjob*, after some ad hoc modifications to remove `python_bin` that I did not include. My intention here is to refactor now and improve separation over time, then actually support other languages in 0.5.