Decrement numWorkers on worker exit instead of shutdown function