Yelp/Tron — pull request #253 by forexviii (2013-06-19 22:15:41+00:00)
Title: First pass at refactoring the Job class structure

This is just a way to review the Job class structure refactoring in order to make sure that the core idea and structure of the refactor is good. Afterwards, a ton of unit tests will have to be rewritten in order to work with the new structure (as well as creating new unit tests for the new parts of the Job structure).

The Job class is gone, and the JobContainer class is mostly untouched. A chunk of the Job stuff was moved into JobScheduler, including things like the action_graph and action_runner, which are required by the jobrun module to actually generate JobRuns. There is now a JobContainer object, which holds a new JobState object, a JobScheduler object, and the MCP's state_watcher object.

The state_watcher object is needed in order to induce a new behavior where JobRuns are directly observed by the state_watcher, rather than observing a JobScheduler observing a Job observing JobRuns. Unfortunately, because JobRuns are often created by delayed callbacks by the JobScheduler, this object must also be referenced in the JobScheduler as well.

The main goal of this refactor is to push the JobRun states upwards in the observer/observable pattern in order to directly serialize the individually changing JobRuns, rather than all of them in one chunk. The JobRun state_data has been unchanged in this refactor, but will probably have to be changed in the near future.

There's still some uncertainty as to what should be done with restoring state for Jobs- because the way the state_data for Jobs is saved is different now, restoring states for Jobs is pretty much broken. I'm not certain as to how Job state data was retrieved (and by what) for purposes of restoring state previously, so I can't in confidence refactor the restore_state functionality just yet. Once I get a handle on what's actually getting the job data and throwing it to a restore_state function, I'll change that code to be something that works with the new model.