hey, robots.txt isnt part of this