Another pass at serving the right robots.txt.