Introduce default robots.txt

See #2062 

This is based on generally accepted advice on base robots.txt 

* http://moz.com/learn/seo/robotstxt
* http://yoast.com/example-robots-txt-wordpress/