Assertion: crawler | /r/CapableOf | find on each page
Facets:
Source sentences:
text: A crawler starts with a list of URLs to visit, and it will follow every hyperlink it can find on each page and add them to the list of URLs to visit.
text: However, if the absence of these resources make the page harder for Google's crawler to understand the page, you should not block them, or else Google won't do a good job of analyzing pages that depend on those resources.