Only set PeerHAndler otherPeerID when we know it