Removing temporary load balancer member