analysis of crawlers and spiders.