Added changes to crawlers