Fix robots.txt and other.