Stop using multiprocessing.dummy.Pool

This implements a custom job queue with workers based on threads.
Collection discovery is now done in a separate thread.