Update to robots.txt

Prevent a few non-public paths from being indexed.