netz98/n98-magerun — pull request #274 by astorm (2013-12-16 00:47:00+00:00)
Title: Running Setup Resource Scripts Incrementally

A recent question about [setup resource migrations and transactions](http://magento.stackexchange.com/questions/11510/does-the-magento-database-upgrade-happen-in-a-transaction/11512) on the Magento Stack Exchange reminded me how opaque the setup resource system is, and how difficult it can be to debug problems related to it.

This command aims to fix that. The new `sys:setup:incremental` command will analyze an un-cached configuration tree and determine which setup resource scripts need to run. Then, one by one, each setup resource object will run its scripts. In addition to providing more system transparency, it also reduces the risk of a script bailing half through through due to resource consumption and leaving the database in an invalid-to-Magento state. This [screencast](https://vimeo.com/81775194) of a development version should give you an idea of what the command does.

Implementation Details
--------------------------------------------------
Magento's architecture isn't built to do something like this, and I used lots of reflection and runtime configuration to minimize duplicating core functionality.

The two bits of jiggery-pokery that are worth calling out are

1. To avoid the error prone problem of recreating the logic of which files to run when, the command runs the scripts by removing every setup resource from the configuration-in-memory, adding the setup resource to run back to the configuration-in-memory, and then calling Magento's `applyAllUpdates` or `applyAllDataUpdates` methods. This allows to command to blindly use Magento's script-running-versioning logic as a black box on a single resource

2. The setup resource system is a two pass operation — the first pass being structure (`applyAllUpdates`), the second pass being data (`applyAllDataUpdates`). These passes happen in **different** event contexts — when `applyAllUpdates` runs there's no event area loaded, and when `applyAllDataUpdates` runs the `global` event area is loaded. Because magerun runs with the `global` area loaded, the `_stashEventContext` and `_restoreEventContext` are used to remove the `global` area for the `applyAllUpdates` run.

Experimental Status and Future Work
--------------------------------------------------
The command is as sound as it can be without real world testing, so I'm not sure if this should be labeled experimental or not.

In addition to real world testing, there's a few issues that could use some feedback and/or help from the community

- **Automated Tests**: During development I had a simple command line script to build out a 1.6.x environment so I could test the scripts, and then have a fresh environment almost immediately so I could quickly iterate. This was vital for me, but I wasn't sure where to start incorporating this work into the magerun test suite.

- Running interactively. It would be nice if there were options for running each part of the script non-interactively.

- Suggestions on different output/reporting options are welcome — I don't actually do that many upgrades these days so I'm not sure what's right for a working developer.

- There's over 30 protected methods in the command class, refactoring some those those into helpers is probably a good idea.