logging for robots.txt