Allow changing the Sockaddr in ComboAddress
Reuse existing connection when possible