Assertion: web crawler | /r/CapableOf | ignore this request
Facets:
Source sentences:
text: You can do this using robots.txt, but some web crawlers have been known to ignore this request.