[BUGFIX] Ensure that Call#peers are correctly maintained

This was broken by Rayo moving to full call URIs in joined/unjoined events

Fixes #405