Fixed 'robots.txt' file handling.