Adding a test for robots.txt to verify that it exists.