refactored crawlers