added an robots.txt
now retrieve robots.txt too