@jamezilla this can be removed. 

It was only meant for testing locally. Previously I was running the crawler with `$ rake cron`. This would take a few minutes and would be killed by rails which limits process times. This is why the I use rack timeout. I hadn't considered it would effect heroku though.