Added multithreaded crawling