Repository: netty/netty
Issue: Openssl-based SSLEngine differs slightly in unwrap behavior
Labels: defect

We are experimenting with netty's openssl based SSLEngines as an alternative to the JDK one ([for our effort](https://github.com/akka/akka-http/tree/jr/http2-on-netty-openssl) to produce an HTTP/2 implementation for akka-http). We use 
netty-tcnative-boringssl-static, version 1.1.33.Fork23. Thanks for providing that! When we talked at reactive summit you mentioned that you put much effort into making the openssl-based SSLEngines compatible with the JDK ones. So, you might be interested in the difference we found (knowing that the SSLEngine API has enough degrees of freedom to warrant a lot of behavior...).

I found that our stream-based wrapper around SSLEngine doesn't read all incoming data from the SSLEngine when there is no new incoming data from the network, so the application gets stuck waiting for incoming data. After some debugging I found out that with netty's openssl SSLEngine unwrap seems to produce plain text data in smaller chunks (probably single TLS frames) and keeps buffering the rest of the data internally. The src buffer is fully consumed but calling unwrap again with an empty src buffer will still produce more data. This differs from what the JDK SSLEngine does in two points:
 * the JDK SSLEngine consumes and produces as much data as possible in one go while the openssl one produces less output in one call
 * the JDK SSLEngine doesn't buffer encrypted data internally between calls to `unwrap` but "puts them back" into the src buffer

This also somewhat documented with this comment from https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLEngine.html:

> The SSLEngine produces/consumes complete SSL/TLS packets only, and does not store application data internally between calls to wrap()/unwrap().

Consequently, we [implemented the `unwrap` loop](https://github.com/akka/akka/blob/master/akka-stream/src/main/scala/akka/stream/impl/io/TLSActor.scala#L402) as

```scala
private def doUnwrap(ignoreOutput: Boolean = false): Unit = {
    val result = engine.unwrap(transportInBuffer, userOutBuffer)
    // ...
    result.getStatus match {
      case OK ⇒
        result.getHandshakeStatus match {
          case NEED_WRAP ⇒ // ...
          case FINISHED ⇒ // ...
          case _ ⇒
            if (transportInBuffer.hasRemaining) doUnwrap() // loop here only if there's still src data available
            else flushToUser()
```

With the openssl SSLEngine we are seeing the src buffer fully consumed after the first call to `unwrap` and we won't continue to call `unwrap` in a loop.

We can fix it for now by changing the condition to 

```scala
if (transportInBuffer.hasRemaining || (result.bytesProduced() > 0)) doUnwrap()
```

to keep looping while `unwrap` produces output.

Wouldn't it be more efficient to do this loop internally in the openssl implementation of `unwrap` to write as much data as possible into the output buffer?

Issue comments:
Related? https://github.com/netty/netty/pull/4254

Yes, seems so. Thanks for the pointer.

@jrudolph would be possible to submit a small unit test that works with JDK SSLEngine and fails with the OpenSslEngine ?

@normanmaurer yes, will try to build one, shouldn't be too hard.

Thanks a lot!

> Am 03.11.2016 um 15:19 schrieb Johannes Rudolph notifications@github.com:
> 
> @normanmaurer yes, will try to build one, shouldn't be too hard.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@jrudolph also when you say "the JDK SSLEngine doesn't buffer encrypted data internally between calls to unwrap but "puts them back" into the src buffer" do you mean that if you not completely drain   stuff into the dst buffer it will just adjust the position of the src buffer and so try to process it on the next call again ?

Here's a test I added to SslEngineTest: https://github.com/jrudolph/netty/tree/w/5975-SslEngine-test

It seems I was wrong about my first point. The JDK SSLEngine also only decrypts only one frame per unwrap call. Still it doesn't consume data that it hasn't read so far.

> @jrudolph also when you say "the JDK SSLEngine doesn't buffer encrypted data internally between calls to unwrap but "puts them back" into the src buffer" do you mean that if you not completely drain stuff into the dst buffer it will just adjust the position of the src buffer and so try to process it on the next call again ?

Hmm, no, I originally don't meant to imply that case. But, indeed, the difference can be seen here as well. JDK returns BUFFER_OVERFLOW without consuming the src buffer while openssl consumes the buffer. I added a commented out block in the test to check this behavior as well.

@jrudolph let me look into it

I think the test case is not much different than @ijuma's [here](https://github.com/ijuma/netty/commit/0f6eff24885bfbe965d0cabd61115067d14feedc#diff-08da41869dfc1d842df441440b1c605bR232), so it might indeed just be a duplicate of #4238. Hi, @ijuma btw ;)

@jrudolph are you license the shared code as ASL2 so I can include it in netty when I have a fix ?

@jrudolph so after some digging I found out that the SSLEngineImpl (the JDK one) always only consume if a full packet is present while we even consume when it is only partly present. Both implementations correctly return `BUFFER_OVERFLOW` tho. I wonder if what we do is permitted.. I could not see details in the javadocs of `SSLEngine` which state that you can not consume a partial record. I am missing anything ?

The other "problem" is that our implementation will consume multiple records in one unwrap call but not write multiple out to the dsts buffers. The JDK implementation only consume one one record at a time (per unwrap call). Again I am not sure if what we do is allowed in terms of the the API and is just an implementation detail. That said I think we should also try to fill more then one record into the dsts buffers if we also consume more then one. 

@jrudolph WDYT ?

@jrudolph actually I think this sentence may state we are not allowed to consume and still return a `BUFFER_OVERFLOW`:

```
The SSLEngine produces/consumes complete SSL/TLS packets only, and does not store application data internally between calls to wrap()/unwrap().
```

Agree ?

@jrudolph ok nevermind.. I think I know how to "fix" the problems stated here. Will need to check what performance impact this has and if it is too much I may introduce a "strict" and non "strict" mode for the `OpenSslEngine`. Will keep you posted.

> @jrudolph are you license the shared code as ASL2 so I can include it in netty when I have a fix ?

Yes, here as well ;)

> @jrudolph actually I think this sentence may state we are not allowed to consume and still return a BUFFER_OVERFLOW:
> 
> The SSLEngine produces/consumes complete SSL/TLS packets only, and does not store application data internally between calls to wrap()/unwrap().
> Agree ?

Yes, exactly. (I quoted that sentence above as well.) Hard to say if this is a requirement or just a description of the implementation. It's probably impossible to take the javadoc as a water-tight specification (I guess you know best about that...).

IMO it's nice if you can rely on the openssl engine not to keep user data in internal buffers (if possible). I haven't looked into it in detail but how does the openssl engine provide backpressure? What happens if you always call `unwrap` e.g. with input buffers of size 30000 and output buffers of size 20000? Will data start to accumulate internally in the openssl engine or will it stop consuming the input buffer at some point?

I have a fix locally here so it works as the JDK impl :) will open a PR later today 👍

@normanmaurer with the strict / non-strict as you previously mentioned? or did you manage to keep it simple?

Was able to keep it simple 

> Am 08.11.2016 um 15:51 schrieb Johno Crawford notifications@github.com:
> 
> @normanmaurer with the strict / non-strict as you previously mentioned? or did you manage to keep it simple?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@jrudolph  should be fixed by https://github.com/netty/netty/pull/5995

Fixed by https://github.com/netty/netty/pull/5995

@jrudolph  - I know it's not nice to comment on an ancient closed topic especially when it's off-topic, but I will like to say a BIG TANK YOU. Your findings and description solved my 2 days no-sleep pain with SSLEngine, where the unconsumed data was copied back in the src, leaving my implementation hanging and waiting for more data. You are a life saver, so again a big THANKS!

Pull request review comments:
@normanmaurer Have you verified that doing this each time will not impact performance? SslHandler would cache the packetLength so that it wouldn't recalculate it each time. I suspect it's not an issue.

Is it correct to use a constant here or should we get this value from the session? Or does the session always return this constant?

It always return the constants

I ran a benchmark and the perf was the same


Patch that resolved the issue:
diff --git a/handler/src/main/java/io/netty/handler/ssl/ReferenceCountedOpenSslEngine.java b/handler/src/main/java/io/netty/handler/ssl/ReferenceCountedOpenSslEngine.java
index fce7410bb4d..76cc0dda9b8 100644
--- a/handler/src/main/java/io/netty/handler/ssl/ReferenceCountedOpenSslEngine.java
+++ b/handler/src/main/java/io/netty/handler/ssl/ReferenceCountedOpenSslEngine.java
@@ -67,6 +67,7 @@
 import static javax.net.ssl.SSLEngineResult.HandshakeStatus.NEED_WRAP;
 import static javax.net.ssl.SSLEngineResult.HandshakeStatus.NOT_HANDSHAKING;
 import static javax.net.ssl.SSLEngineResult.Status.BUFFER_OVERFLOW;
+import static javax.net.ssl.SSLEngineResult.Status.BUFFER_UNDERFLOW;
 import static javax.net.ssl.SSLEngineResult.Status.CLOSED;
 import static javax.net.ssl.SSLEngineResult.Status.OK;
 
@@ -416,9 +417,8 @@ private int writePlaintextData(final ByteBuffer src) {
     /**
      * Write encrypted data to the OpenSSL network BIO.
      */
-    private int writeEncryptedData(final ByteBuffer src) {
+    private int writeEncryptedData(final ByteBuffer src, int len) {
         final int pos = src.position();
-        final int len = src.remaining();
         final int netWrote;
         if (src.isDirect()) {
             final long addr = Buffer.address(src) + pos;
@@ -430,8 +430,12 @@ private int writeEncryptedData(final ByteBuffer src) {
             final ByteBuf buf = alloc.directBuffer(len);
             try {
                 final long addr = memoryAddress(buf);
-
-                buf.setBytes(0, src);
+                int newLimit = pos + len;
+                if (newLimit != src.remaining()) {
+                    buf.setBytes(0, (ByteBuffer) src.duplicate().position(pos).limit(newLimit));
+                } else {
+                    buf.setBytes(0, src);
+                }
 
                 netWrote = SSL.writeToBIO(networkBIO, addr, len);
                 if (netWrote >= 0) {
@@ -601,6 +605,11 @@ public final SSLEngineResult wrap(
                 }
             }
 
+            if (dst.remaining() < MAX_ENCRYPTED_PACKET_LENGTH) {
+                // Can not hold the maximum packet so we need to tell the caller to use a bigger destination
+                // buffer.
+                return new SSLEngineResult(BUFFER_OVERFLOW, getHandshakeStatus(), 0, 0);
+            }
             // There was no pending data in the network BIO -- encrypt any application data
             int bytesProduced = 0;
             int bytesConsumed = 0;
@@ -775,9 +784,29 @@ public final SSLEngineResult unwrap(
                 }
             }
 
-            // Write encrypted data to network BIO
+            if (len < SslUtils.SSL_RECORD_HEADER_LENGTH) {
+                return new SSLEngineResult(BUFFER_UNDERFLOW, getHandshakeStatus(), 0, 0);
+            }
+
+            int packetLength = SslUtils.getEncryptedPacketLength(srcs, srcsOffset);
+            if (packetLength - SslUtils.SSL_RECORD_HEADER_LENGTH > capacity) {
+                // No enough space in the destination buffer so signal the caller
+                // that the buffer needs to be increased.
+                return new SSLEngineResult(BUFFER_OVERFLOW, getHandshakeStatus(), 0, 0);
+            }
+
+            if (len < packetLength) {
+                // We either have no enough data to read the packet length at all or not enough for reading
+                // the whole packet.
+                return new SSLEngineResult(BUFFER_UNDERFLOW, getHandshakeStatus(), 0, 0);
+            }
+
             int bytesConsumed = 0;
             if (srcsOffset < srcsEndOffset) {
+
+                // Write encrypted data to network BIO
+                int packetLengthRemaining = packetLength;
+
                 do {
                     ByteBuffer src = srcs[srcsOffset];
                     int remaining = src.remaining();
@@ -787,9 +816,15 @@ public final SSLEngineResult unwrap(
                         srcsOffset++;
                         continue;
                     }
-                    int written = writeEncryptedData(src);
+                    // Write more encrypted data into the BIO. Ensure we only read one packet at a time as
+                    // stated in the SSLEngine javadocs.
+                    int written = writeEncryptedData(src, Math.min(packetLengthRemaining, src.remaining()));
                     if (written > 0) {
-                        bytesConsumed += written;
+                        packetLengthRemaining -= written;
+                        if (packetLengthRemaining == 0) {
+                            // A whole packet has been consumed.
+                            break;
+                        }
 
                         if (written == remaining) {
                             srcsOffset++;
@@ -808,6 +843,7 @@ public final SSLEngineResult unwrap(
                         break;
                     }
                 } while (srcsOffset < srcsEndOffset);
+                bytesConsumed = packetLength - packetLengthRemaining;
             }
 
             // Number of produced bytes
diff --git a/handler/src/main/java/io/netty/handler/ssl/SslHandler.java b/handler/src/main/java/io/netty/handler/ssl/SslHandler.java
index 55c09aef438..b06e3a7a52f 100644
--- a/handler/src/main/java/io/netty/handler/ssl/SslHandler.java
+++ b/handler/src/main/java/io/netty/handler/ssl/SslHandler.java
@@ -197,14 +197,6 @@ public class SslHandler extends ByteToMessageDecoder implements ChannelOutboundH
      * {@code true} if and only if {@link SSLEngine} expects a direct buffer.
      */
     private final boolean wantsDirectBuffer;
-    /**
-     * {@code true} if and only if {@link SSLEngine#wrap(ByteBuffer, ByteBuffer)} requires the output buffer
-     * to be always as large as {@link #maxPacketBufferSize} even if the input buffer contains small amount of data.
-     * <p>
-     * If this flag is {@code false}, we allocate a smaller output buffer.
-     * </p>
-     */
-    private final boolean wantsLargeOutboundNetworkBuffer;
 
     // END Platform-dependent flags
 
@@ -283,7 +275,6 @@ public SslHandler(SSLEngine engine, boolean startTls, Executor delegatedTaskExec
 
         boolean opensslEngine = engine instanceof OpenSslEngine;
         wantsDirectBuffer = opensslEngine;
-        wantsLargeOutboundNetworkBuffer = !opensslEngine;
 
         /**
          * When using JDK {@link SSLEngine}, we use {@link #MERGE_CUMULATOR} because it works only with
@@ -516,7 +507,7 @@ private void wrap(ChannelHandlerContext ctx, boolean inUnwrap) throws SSLExcepti
 
                 ByteBuf buf = (ByteBuf) msg;
                 if (out == null) {
-                    out = allocateOutNetBuf(ctx, buf.readableBytes());
+                    out = allocateOutNetBuf(ctx);
                 }
 
                 SSLEngineResult result = wrap(alloc, engine, buf, out);
@@ -599,7 +590,7 @@ private void wrapNonAppData(ChannelHandlerContext ctx, boolean inUnwrap) throws
             // See https://github.com/netty/netty/issues/5860
             while (!ctx.isRemoved()) {
                 if (out == null) {
-                    out = allocateOutNetBuf(ctx, 0);
+                    out = allocateOutNetBuf(ctx);
                 }
                 SSLEngineResult result = wrap(alloc, engine, Unpooled.EMPTY_BUFFER, out);
 
@@ -1477,14 +1468,8 @@ private ByteBuf allocate(ChannelHandlerContext ctx, int capacity) {
      * Allocates an outbound network buffer for {@link SSLEngine#wrap(ByteBuffer, ByteBuffer)} which can encrypt
      * the specified amount of pending bytes.
      */
-    private ByteBuf allocateOutNetBuf(ChannelHandlerContext ctx, int pendingBytes) {
-        if (wantsLargeOutboundNetworkBuffer) {
-            return allocate(ctx, maxPacketBufferSize);
-        } else {
-            return allocate(ctx, Math.min(
-                    pendingBytes + OpenSslEngine.MAX_ENCRYPTION_OVERHEAD_LENGTH,
-                    maxPacketBufferSize));
-        }
+    private ByteBuf allocateOutNetBuf(ChannelHandlerContext ctx) {
+        return allocate(ctx, maxPacketBufferSize);
     }
 
     private final class LazyChannelPromise extends DefaultPromise<Channel> {
diff --git a/handler/src/main/java/io/netty/handler/ssl/SslUtils.java b/handler/src/main/java/io/netty/handler/ssl/SslUtils.java
index a7ad75e3e32..3c222b93dda 100644
--- a/handler/src/main/java/io/netty/handler/ssl/SslUtils.java
+++ b/handler/src/main/java/io/netty/handler/ssl/SslUtils.java
@@ -21,6 +21,8 @@
 import io.netty.handler.codec.base64.Base64;
 import io.netty.handler.codec.base64.Base64Dialect;
 
+import java.nio.ByteBuffer;
+
 /**
  * Constants for SSL packets.
  */
@@ -120,6 +122,92 @@ static int getEncryptedPacketLength(ByteBuf buffer, int offset) {
         return packetLength;
     }
 
+    private static short unsignedByte(byte b) {
+        return (short) (b & 0xFF);
+    }
+
+    private static int unsignedShort(short s) {
+        return s & 0xFFFF;
+    }
+
+    static int getEncryptedPacketLength(ByteBuffer[] buffers, int offset) {
+        ByteBuffer buffer = buffers[offset];
+
+        // Check if everything we need is in one ByteBuffer. If so we can make use of the fast-path.
+        if (buffer.remaining() >= SslUtils.SSL_RECORD_HEADER_LENGTH) {
+            return getEncryptedPacketLength(buffer);
+        }
+
+        // We need to copy 5 bytes into a temporary buffer so we can parse out the packet length easily.
+        ByteBuffer tmp = ByteBuffer.allocate(5);
+
+        do {
+            buffer = buffers[offset++].duplicate();
+            if (buffer.remaining() > tmp.remaining()) {
+                buffer.limit(buffer.position() + tmp.remaining());
+            }
+            tmp.put(buffer);
+        } while (tmp.hasRemaining());
+
+        // Done, flip the buffer so we can read from it.
+        tmp.flip();
+        return getEncryptedPacketLength(tmp);
+    }
+
+    private static int getEncryptedPacketLength(ByteBuffer buffer) {
+        int packetLength = 0;
+        int pos = buffer.position();
+        // SSLv3 or TLS - Check ContentType
+        boolean tls;
+        switch (unsignedByte(buffer.get(pos))) {
+            case SslUtils.SSL_CONTENT_TYPE_CHANGE_CIPHER_SPEC:
+            case SslUtils.SSL_CONTENT_TYPE_ALERT:
+            case SslUtils.SSL_CONTENT_TYPE_HANDSHAKE:
+            case SslUtils.SSL_CONTENT_TYPE_APPLICATION_DATA:
+                tls = true;
+                break;
+            default:
+                // SSLv2 or bad data
+                tls = false;
+        }
+
+        if (tls) {
+            // SSLv3 or TLS - Check ProtocolVersion
+            int majorVersion = unsignedByte(buffer.get(pos + 1));
+            if (majorVersion == 3) {
+                // SSLv3 or TLS
+                packetLength = unsignedShort(buffer.getShort(pos + 3)) + SslUtils.SSL_RECORD_HEADER_LENGTH;
+                if (packetLength <= SslUtils.SSL_RECORD_HEADER_LENGTH) {
+                    // Neither SSLv3 or TLSv1 (i.e. SSLv2 or bad data)
+                    tls = false;
+                }
+            } else {
+                // Neither SSLv3 or TLSv1 (i.e. SSLv2 or bad data)
+                tls = false;
+            }
+        }
+
+        if (!tls) {
+            // SSLv2 or bad data - Check the version
+            int headerLength = (unsignedByte(buffer.get(pos)) & 0x80) != 0 ? 2 : 3;
+            int majorVersion = unsignedByte(buffer.get(pos + headerLength + 1));
+            if (majorVersion == 2 || majorVersion == 3) {
+                // SSLv2
+                if (headerLength == 2) {
+                    packetLength = (buffer.getShort(pos) & 0x7FFF) + 2;
+                } else {
+                    packetLength = (buffer.getShort(pos) & 0x3FFF) + 3;
+                }
+                if (packetLength <= headerLength) {
+                    return -1;
+                }
+            } else {
+                return -1;
+            }
+        }
+        return packetLength;
+    }
+
     static void notifyHandshakeFailure(ChannelHandlerContext ctx, Throwable cause) {
         // We have may haven written some parts of data before an exception was thrown so ensure we always flush.
         // See https://github.com/netty/netty/issues/3900#issuecomment-172481830


Test patch:
diff --git a/handler/src/test/java/io/netty/handler/ssl/SSLEngineTest.java b/handler/src/test/java/io/netty/handler/ssl/SSLEngineTest.java
index b565b3da09d..8f1cbee3f00 100644
--- a/handler/src/test/java/io/netty/handler/ssl/SSLEngineTest.java
+++ b/handler/src/test/java/io/netty/handler/ssl/SSLEngineTest.java
@@ -678,9 +678,8 @@ protected void testEnablingAnAlreadyDisabledSslProtocol(String[] protocols1, Str
     }
 
     protected static void handshake(SSLEngine clientEngine, SSLEngine serverEngine) throws SSLException {
-        int netBufferSize = 17 * 1024;
-        ByteBuffer cTOs = ByteBuffer.allocateDirect(netBufferSize);
-        ByteBuffer sTOc = ByteBuffer.allocateDirect(netBufferSize);
+        ByteBuffer cTOs = ByteBuffer.allocateDirect(clientEngine.getSession().getPacketBufferSize());
+        ByteBuffer sTOc = ByteBuffer.allocateDirect(serverEngine.getSession().getPacketBufferSize());
 
         ByteBuffer serverAppReadBuffer = ByteBuffer.allocateDirect(
                 serverEngine.getSession().getApplicationBufferSize());
@@ -915,4 +914,84 @@ public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exc
 
         promise.syncUninterruptibly();
     }
+
+    @Test
+    public void testUnwrapBehavior() throws Exception {
+        SelfSignedCertificate cert = new SelfSignedCertificate();
+
+        clientSslCtx = SslContextBuilder
+                .forClient()
+                .trustManager(cert.cert())
+                .sslProvider(sslClientProvider())
+                .build();
+        SSLEngine client = clientSslCtx.newEngine(UnpooledByteBufAllocator.DEFAULT);
+
+        serverSslCtx = SslContextBuilder
+                .forServer(cert.certificate(), cert.privateKey())
+                .sslProvider(sslServerProvider())
+                .build();
+        SSLEngine server = serverSslCtx.newEngine(UnpooledByteBufAllocator.DEFAULT);
+
+        byte[] bytes = "Hello World".getBytes(CharsetUtil.US_ASCII);
+
+        try {
+            ByteBuffer plainClientOut = ByteBuffer.allocate(client.getSession().getApplicationBufferSize());
+            ByteBuffer encryptedClientToServer = ByteBuffer.allocate(server.getSession().getPacketBufferSize() * 2);
+            ByteBuffer plainServerIn = ByteBuffer.allocate(server.getSession().getApplicationBufferSize());
+
+            handshake(client, server);
+
+            // create two TLS frames
+
+            // first frame
+            plainClientOut.put(bytes, 0, 5);
+            plainClientOut.flip();
+
+            SSLEngineResult result = client.wrap(plainClientOut, encryptedClientToServer);
+            assertEquals(SSLEngineResult.Status.OK, result.getStatus());
+            assertEquals(5, result.bytesConsumed());
+            assertTrue(result.bytesProduced() > 0);
+
+            assertFalse(plainClientOut.hasRemaining());
+
+            // second frame
+            plainClientOut.clear();
+            plainClientOut.put(bytes, 5, 6);
+            plainClientOut.flip();
+
+            result = client.wrap(plainClientOut, encryptedClientToServer);
+            assertEquals(SSLEngineResult.Status.OK, result.getStatus());
+            assertEquals(6, result.bytesConsumed());
+            assertTrue(result.bytesProduced() > 0);
+
+            // send over to server
+            encryptedClientToServer.flip();
+
+            // try with too small output buffer first (to check BUFFER_OVERFLOW case)
+            int remaining = encryptedClientToServer.remaining();
+            ByteBuffer small = ByteBuffer.allocate(3);
+            result = server.unwrap(encryptedClientToServer, small);
+            assertEquals(SSLEngineResult.Status.BUFFER_OVERFLOW, result.getStatus());
+            assertEquals(remaining, encryptedClientToServer.remaining());
+
+            // now with big enough buffer
+            result = server.unwrap(encryptedClientToServer, plainServerIn);
+            assertEquals(SSLEngineResult.Status.OK, result.getStatus());
+
+            assertEquals(5, result.bytesProduced());
+            assertTrue(encryptedClientToServer.hasRemaining());
+
+            result = server.unwrap(encryptedClientToServer, plainServerIn);
+            assertEquals(SSLEngineResult.Status.OK, result.getStatus());
+            assertEquals(6, result.bytesProduced());
+            assertFalse(encryptedClientToServer.hasRemaining());
+
+            plainServerIn.flip();
+
+            assertEquals(ByteBuffer.wrap(bytes), plainServerIn);
+        } finally {
+            cleanupClientSslEngine(client);
+            cleanupServerSslEngine(server);
+        }
+    }
 }