Fix and improve threading support

Added a thread pool to actually control how many threads are created.
Threading is also now optional and off by default. Recursive search and
threading are still incompatible.