Fixed robots.txt method.