first rough implementation of concurrent crawler
implemented crawler.Run()