WorldBank-Transport/open-transit-indicators — pull request #388 by kshepard (2014-10-30 01:42:35+00:00)
Title: Speed up Django/Scala communication to reduce timeouts

This implements a variety of changes in an attempt to fix the timeout issue:
* Switch to gevent
* Increase workers and max requests for Gunicorn
* Increase all Spray client timeout settings
* Set niceness for the Scala process in order to give precedence to Gunicorn
* Switch to Well Known Binary to cut down on request size and parsing time
* Add sleeps to throttle requests from Scala to Django

With all of these changes implemented, I was able to have indicators calculated for Zhangzhou without seeing timeouts. However it's cutting it a little close -- if I reduce the sleeps, the timeouts come back. Ideally no sleeps would be needed. If anyone has any good ideas for things I missed, please pull down this branch, comment out the sleeps and give it a try.