Patent ID: 9679056
Date: 2017-06-13
CPC Classifications: G06F

Claim:
1. A method, comprising: at a computing system having one or more processors and memory storing one or more programs executed by the one or more processors: setting a respective reuse flag for a corresponding document in a plurality of documents, wherein the respective reuse flag is based on a document importance score associated with the corresponding document, wherein the document importance score associated with a document is based, at least in part, on a query-independent metric of an importance of the document for a search engine; and performing a document crawling operation on the plurality of documents in accordance with the reuse flag for respective documents in the plurality of documents, including reusing a previously downloaded version of a respective document in the plurality of documents instead of downloading a current version of the respective document from a host computer in accordance with a determination that the reuse flag associated with the respective document meets a predefined criterion.