mozilla/socorro — pull request #1621 by peterbe (2013-10-24 19:50:39+00:00)
Title: fixes bug 928048 - crontabber to store all state in postgres instead of json

@selenamarie (cc @rhelmer @AdrianGaudebert @twobraids) r?

I know this is a massive patch but actually significant changes are quite small. It's just a bunch of tests had to be re-hacked.

* The state used to be stored in a dict like object that worked like a dict but added `.load()` and `.save()`. Now it's using the `StateDatabase` class which also, sort of, works like a dict [1]
* From now on, all tests in socorro/unittest/cron/jobs/* are all integration tests. Ie. they all require talking to PG
* A new feature beyond just storing the state in PG is that it executes `_remember_success()` and `_remember_failure()` which records when the app was run, if it succeeded, and another possible python exception and traceback. Nothing is yet done formally to read this table but a future middleware could expose this on the Admin UI.
* Migration from `crontabber.json` is handled automagically. Every time the crontabber is instanciated, if checks if there's *any* data in the `crontabber` table. And if there isn't it opens the `cronabber.json` file and copies everything from that over to PG.
* The most complex test to convert was `test_matviews.py` it used to mock all psycopg2 traffic so it didn't matter what stored procedures it fired because they were all voids. Now, I override, in run-time all stored procedures of all matviews with a temporary harmless equivalent. The code got messy after this but I hope it all makes sense.

[1] Example:
```python
# select 1 from crontabber where app_name='app-name'
if 'app-name' in self.database:
# select * from crontabber where app_name = 'app-name'
stuff = self.database['app-name']
stuff['something'] = 'different'
# either update crontabber set ... where app_name = 'app-name'
# or a insert into crontabber (app_name, ...) values ('app-name', ...)
self.database['app-name'] = stuff
# delete from crontabber where app_name = 'other-app'
del self.database['other-app']
```