Add /robots.txt and /deny endpoints