mozilla/kuma — pull request #919 by darkwing (2013-03-08 20:24:12+00:00)
Title: fix bug 844528 - Disable robots on dev servers

A few things here:

0. I've added a cron routine to create the robots.txt file
1. I'm hard-coding the robots.txt content in the view; history shows we never touch it.
2. I'm uploading a new, disallow-all robots.txt to replace anything on a dev server so that if they don't have CRONs already, we don't need to do a bunch of config

STOP! TWO IMPORTANT THINGS:
1. WE MUST MAKE SURE THAT "ENGAGE_ROBOTS" IS "True" IN PRODUCTION
2. WE MUST RUN THIS CRON IN PRODUCTION AS SOON AS IT'S PUSHED SO WE DON'T DISALLOWING INDEXING IN PRODUCTION. WE WOULD BE HOSED.

That is all.