Getting too many results is more likely to be a problem than getting too few, though identifying the truly useful ones may require patient sifting. Many of the search engines enable advanced searching and use complex algorithms to assess the relevance of documents. Naturally, the usefulness of such techniques depends on how the user formulates the search. The programs which perform the first task are commonly referred to as robots, spiders or crawlers. They trawl through the Internet, automatically retrieving data from one site after another for inclusion in the search engine s database, possibly following a trail of hypertext links.
Robots can assemble huge databases of information, but gathered indiscriminately.
The database may also include manually submitted URLs. The second element, the search facility, is accessed via a search form on a Web page. The user types a word or some words in the appropriate box describing what he or she is looking for. The search engine searches through its index database for information which seems to match what s required and returns a list of matching resources. The list of matches is usually ranked according to how well they seem to satisfy the search, and each list entry includes a hypertext link to the actual resource it references.