Yelp/mrjob — pull request #432 by irskep (2012-04-03 23:01:12+00:00)
Title: Encapsulate basic config logic in OptionStore objects

Targeted for 0.3.4.

I pulled all of this logic out of the runner classes:

* Defaults
* Allowed option lists and validation
* Combiner mappings
* Combining a list of option dicts into one and calculating option priority
* EC2 option logic
* "Was this option changed from the default on the command line or in `mrjob.conf`?"

`OptionStore` looks like a `dict` (in fact it subclasses `dict`). `RunnerOptionStore` adds `__init__()` parameters for the runner alias, runner kwargs, and `mrjob.conf` path, and does these things:

* Invokes the `mrjob.conf` logic to load the list of dicts
* Invokes `OptionStore` logic to populate the underlying `dict`
* Validates final output

Ideas for later, perhaps in this pull request but not necessarily:

* Move more `mrjob.conf` loading into this class to cut down on imports in `mrjob.runner`
* Introduce derived options that are only computed at runtime if they are not defined by the user (some of the EMR job flow size/type options)
* Break runners into packages to separate this logic more

There are no user-facing changes here, and PyFlakes and the test cases all pass.