nodeapps/http-server — pull request #61 by andrefs (2013-12-23 23:23:23+00:00)
Title: Add option to dynamically serve a /robots.txt

I added a --robots or -r option to dynamically respond to /robots.txt. This allows you to expose you http-server instance without the risk of having your folders indexed by a search engine crawler.