Added robot.txt documentation