Clients remember host address and add it to a peer when necessary