Moved reference counting from workerHandle to scheduler.