Fast remove of local peers on stop.