EducationalTestingService/gridmap — pull request #12 by dan-blanchard (2013-11-14 21:43:28+00:00)
Title: Add back features from Python Grid (and improve some of them)

After running into issues with SGE never accurately telling us when jobs had finished via DRMAA, I've decided to switch back to using 0MQ for things like the original pythongrid. In the course of looking through their code, I've also added back a couple job monitoring tools that we had previously removed:

- An optional CherryPy web monitor that can tell you which jobs you're waiting on and what happened to jobs that finished. The interface is very clunky at the moment, and it could have a lot more information, but I'm sure someone (@desilinguist) could fix that. :smile:
- An optional error reporting email service. I should probably test that out more before we merge this, come to think of it. It can even do crazy things like send you a matplotlib graph of your memory and CPU usage.
- There are also a boat load of new environment variables you can specify to tweak a lot of aspects of the configuration, including how frequently heartbeats happen and stuff like that. In the original `pythongrid` these could only be modified by changing a Python file inside the `site-packages` directory, which I wasn't willing to make people do.

There are a couple little refactoring things that I'd like to take care of in the future (e.g., `_send_zmq_msg` shouldn't create a new socket for every message), but having a system that doesn't constantly tell you jobs died when they didn't seemed like a bigger priority.