Assertion: crawler | /r/CapableOf | respect the robots.txt standard
Facets:
value: for controlling access to websites
type: PURPOSE
Source sentences:
text: Our crawler only visits publically available pages, and it respects the robots.txt standard for controlling access to websites.