queryall/queryall — pull request #30 by ansell (2012-05-18 00:24:46+00:00)
Title: Fix logging in production

Log levels for some errors have been changed to be effectively debug, even though they come out in the test logs with ERROR or WARN messages. This is because many of these errors can be triggered by users, or they can make bad situations worse in times of high load, so although they are legitimate errors, they do not need to be logged in production.

Also added Crawl-delay to robots.txt with a fixed value of 10 seconds currently. TODO: Make crawl-delay number configurable in future