KAZOO-3208: track other leg, just don't bind

@onnet See if this patch works for you. It is lighter weight than your approach, and takes advantage of the fact that we had the information, we just discarded it!