Multi-threaded crawling now works.