re-structured the crawler.