netty/netty — Issue #137: SSLHandler does not detect or act on incoming close_notify messages
Author: wolfmanjm
Created: 2011-12-20 21:09:05+00:00
State: open

It appears that Netty does not detect the SSLEngine processing a close_notify, so you end up with a situation where the SSLEngine is in state CLOSED, but no upstream notifications are generated and there is no way for the application to know it has happened.

This can happen if the client closes the SSLSocket but not the underlying Socket. The Netty server receives the close_notify, unwraps it, and sends back the reply of close_notify, however the SSLHandler code itself does not detect the CLOSED state of the SSLEngine. This can be tested with a client closing an SSLSocket which has been created by wrapping a Socket, or by a Netty client calling SSLHandler.close().

From the SSLEngine JDK docs...

> "A peer will signal its intent to close by sending its own closure handshake message. After this message has been received > and processed by the local SSLEngine's unwrap() call, the application can detect the close by calling unwrap() and looking > for a SSLEngineResult with status "CLOSED", or if isInboundDone() returns true"

The SSLHandler does process the handshake, but does not detect the status "CLOSED".

I suggest that an optional Future be allowed on the SSLHandler that can be notified of the SSLEngine closing, as this is not a common situation, as usually the underlying socket will also be closed which will be detected normally.

I have a situation where I initially communicate via SSL, then close the SSLEngines and continue using TCP over the same socket, this works fine using SSLSocket directly as it returns -1 on a get when the SSLSocket closes but the underlying Socket remains open.