Web pages change at many different rates, from daily to never. New pages come online at a prodigous rate . While page authors may request an updated visit, discovery of changes and new pages is up to the search engine. Infoseek lets the search engine sysadmin set the maximum and minimum revisit intervals and Infoseek then places pages into one of six queues according to the last modified date on the Web page. Consider the UNH Intranet, with more than 32000 pages known to Infoseek.
Search engines build an index of all the material discovered.
The search engine index, lacking a pre-defined structure, uses other techniques to aid in a search. Retrieval includes the options given to the user of the search engine to specify a search. Infoseek offers three overlapping strategies for dealing with the quality vs. quantity issue. Second, it allows a search within a set of search results. If the goal is to index and search all, or a significant fraction of, the publicly accessible Internet-wide Web pages, the search engine pushes the limits of capacity and speed needed to go beyond and to be an operational success.