Complete crawling function for search page