updated robots.txt to remove more links