Assertion: web crawler | /r/CapableOf | standardize url
Facets:
Source sentences:
text: To Avoid Duplicate Downloads: Web crawlers sometimes modify and standardize URLs so that they can avoid crawling the same resource multiple times.
text: Web crawlers sometimes modify and standardize URLs so that they can avoid crawling the same resource multiple times.