The study indicates that the HotBot search engine has the most comprehensive index of the Web, but even so, covers only about 34 percent of the indexable pages. Robots are physical and represent a step up from simple automation. Robots can vary widely in their level of intelligence. The simplest robots are unintelligent and depend on simple environments and precise placement of work. More intelligent robots are able to deal successfully with complex environments and can tolerate imprecision.
Artificial life studies the rules governing complex systems.
One strategy is to become more knowlegeable about Web searching. See the following sites for information on how to become a better searcher. Using an Agent in Web Searches AltaVista provides a software agent that anyone can use. With it, users can construct specialized Web queries. They will often return tens of thousands of pages in response to queries, perhaps leaving users no closer to the information they seek.