Product advertisements routinely cite URLs. The Web has grown so fast and is changing so rapidly that we've had to change the way we use it to find information. Today the best approach to working with this amorphous and chaotic mass of information is to mine it with a good Web search tool. Usually these are search engines that query large, descriptive indexes of Web pages. Each search engine typically searches its own index of page descriptions, not the Web itself.
Some engines add thousands of entries a day to their databases.
Although most search tools encourage you to register your site with them, these vast indexes are primarily built by robots. Robots, also called bots, spiders, wanderers, worms, and crawlers, are programs that live on the search services' host computers. Despite the names, spiders and such don't actually traverse the Web. The information retrieved is then used to build the indexes. Effective search engines revisit sites periodically to update their databases.