Add additional pages to robots.txt