fix bug 844528 - no robots on allizom hosts

https://bugzilla.mozilla.org/show_bug.cgi?id=844528

supersedes #919 - no cron nor django view in favor of apache-level rewrite to static files.