Along with the provenance method, using subject directories is the only truly effective way of finding information on a topic .
Subject guides provide a limited number of professionally-selected and evaluated webpages.
You may be missing lots of other sites the indexers didn't know about, but at least the ones you get are important and appropriate. You may argue that subject directories only index and keep track of a minute part of the web. You are correct. These programs are each different and run on the Internet at different times and frequencies. The search engine retrieves a listing from the current index of all the webpages that contain the keyword or search strategy you typed into the search box.