implemented crawl in separate processes