moved to a threadpool model to avoid goroutine memory explosion