gtracy/SMSMyBus — pull request #13 by epaulson (2012-03-04 23:00:13+00:00)
Title: ScraperWiki errors: don't retry forever

To be a bit more defensive against scraper wiki errors (the city website seems to format something different every 1st of the month and break the scraper), set a key in memcache and keep track of how many times we've had a failure. If we get more than 3, stop bothering to check for any API requests that come in for the next 6 hours.

There's a slight race condition that I'm willing to live with - we may try a few extra times if someone overwrites our value until eventually someone wins and gets to 3, but I don't see how to set an expiration time with the memcache API if you use the atomic increment features.

The settings of 3 failures and 6 hours are completely arbitrary and feel free to change them, or as always, anything else.

I will fix the scraper next and hopefully when April rolls around it won't need a fix again.