channel: Add a precondition that we have a connection

Was hitting this with hotssh.