complet crawl function