Don't stop client temporary client removal if clean us thread removed it
already