Only toggle the stream crypto handshake once

Courtesy of @DaveRandom and @rdlowrey