Fix TaskWorker dependencies in setup.py. #4052