Fix several issues with robots.txt