BK-1120 Booktype core application should host robots.txt file

Correct URL is /<book id>/_export/ and "Disallow: /_export/" is not going to match it. It will match "/_export/<book id>/" but not the one we really use.

Check the Robots.txt specification for more info - https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt