added robots.txt (for some reason they were not there before, but it worked on production server)