Assertion: engine crawler | /r/CapableOf | begin crawling website
Facets:
value: by downloading its robots.txt file
type: MANNER
Source sentences:
text: All commercial search engine crawlers begin crawling a website by downloading its robots.txt file, which contains rules about what pages search engines should or should not crawl on the website.