Patent ID: 7502773
Filing Date: 2009-03-10
Classification: G06F,Y10S

Abstract:
1. Computer-readable instructions embodied on a computer-readable storage medium that when executed on one or more processors implement a page index system, the system comprising: a page data store that stores reference information associated with a page, the reference information is obtained from at least one other page and is accumulated incrementally from each other page as each other page is crawled, the reference information comprising descriptive information that is adjacent to anchor text associated with a referencing uniform resource locator that references the page; and a crawler component that receives the page, retrieves the reference information associated with the page from the page data store, and provides the page and the reference information to at least an index building component; wherein failure to receive a requested page after a first predetermined period of time causes the URL for the page to be removed from the page data store after a second predetermined period of time.