allow robots.txt

this commit allows robots.txt to be placed into the htdocs root directory.