Separate net.Conn handling from bytestream

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)