Handle workers that are still running when shutdown starts