adding crawl update, crawl download, other fixes