don't call connection callbacks more than once

May happen if the underlying lib does some automatic recovery.