Update README.md

Added note about robots.txt