upload appropriate robots.txt