start sending robots.txt (and humans.txt) again, and set to dissallow if...

... its not the real docs site

Docker-DCO-1.1-Signed-off-by: Sven Dowideit <SvenDowideit@docker.com> (github: SvenDowideit)