There is some debate on the automated searching of the web. Automated searchers retrieve pages from servers faster than people do, thus eating network bandwidth and server resources. XWebcomber tries to be a good network citizen and minimize its impact on net resources. It does not load any images, nor video, sound, or other binary data. For a given search XWebcomber will not load the same web page more than once.
Circular references are no problem.
XWebcomber limits the depth of the search. Finally, XWebcomber passes the User-Agent and From fields to the webservers. If any web site is burdened by XWebcomber, the webmaster can restrict its access. Webcomber is shareware. Right now, the price is some email telling us that you like it.