Assertion: engine crawler | /r/CapableOf | file the crawler
Facets:
Source sentences:
text: A robots.txt file tells search engine crawlers which pages or files the crawler can or can't request from your site.