clean up and add robots.txt