maintain robots.txt from removed files