Do not block removing clients if notification manager quit.