EverythingMe/redash — pull request #181 by arikfr (2014-04-22 12:03:24+00:00)
Title: Switch to multiprocessing instead of threading

Until now the `manage.py runworkers` command would start the manager along with N threads that were used as workers. Those workers, when executing a query, would fork a new process that will do the actual processing. This was done to preserve the memory footprint and to be able to cancel running queries.

The problem was that mixing forking and threading results in weird issues, including the workers freezing. We used `atfork` to mitigate this, but still once in a while the workers would freeze. This pull request removes the mixture of threads and forking.

Eventually we will remove the whole "homemade" workers solution in favor of some proper workers solution like `rq` or `Celery`.