The spider , also called a crawler or bot, visits a web page, reads it, and then follows links to other pages within the site. The spider returns to the site in a timely fashion to record changes. The data that the spider finds goes into the index , a searchable database or catalog of documents. The software program sends the spider, then sifts through all the pages recorded in the index to find matches to the search query and ranks them in order of relevance. The search algorithm and ranking system programs are highly secret proprietary products.
Search engines are great for searching specific information.
Google is an outstanding search engine. It is one of the largest with an index of over a billion pages. It is very effective in finding the best website for broad, topic-level queries. Google sorts search results using its patented PageRank analysis. It ranks sites by the number of links that point to the site with greater weight given to authoritative sites.