Update robots.txt rules.