Assertion: website | /r/CapableOf | limit the crawl rate
Facets:
Source sentences:
text: Websites can declare if crawling is allowed or not in the robots.txt file and allow partial access, limit the crawl rate, specify the optimal time to crawl and more.