robots.txt change allow all / version

removed crawl delay, robot version, and request rate, added an allow all
for all agents before disallow.