etsy/skyline — pull request #49 by kryptx (2013-09-17 15:49:17+00:00)
Title: Abstracting alerters and adding HipChat and PagerDuty

Hey Abe & folks,

We had been trying for a while to figure out how be notified about anomalies, since we don't have someone monitoring a wall board 24/7. When we noticed you added alerting we were really excited but felt e-mail was a little bit impractical for our purposes.

To address that I've abstracted the alerting code, trying to follow a similar pattern to your algorithms, and added alerts that will send messages to HipChat and/or trigger incidents in Pagerduty. Additional alerts can be added with minimal effort.

The alert settings schema has changed a bit, but the changes are backwards compatible with settings.py from your previous commit. If you feel that's not necessary then this can be cleaned up a bit.

I'm interested to hear your feedback on these changes. I'm happy to refactor if you'd prefer it be done differently.

Thanks!