session: remove channel reference

To get the local and remote address, the user can simply query the
channel directly.