modify several files for robots.txt