Stub out crawler with a threadpool.
Combine crawler scripts into one threadpool guy.