netty/netty — Issue #160: No response to write if server is using SslHandler and client is not
Author: mhgrove
Created: 2012-01-23 16:23:22+00:00
State: open

[Originally posted on Stackoverflow](http://stackoverflow.com/questions/8930630/no-response-to-write-if-server-is-using-sslhandler-and-client-is-not).

Happens in the inverse as well, the client is using SslHander but the server isn't. I wanted to have a graceful failure for both cases to let the client to that they need to enable, or disable, SSL to connect to the server. But when the two are out of sync, the communications go awry.

Part of the issue I observed was that in the SslHander decode, it was possible that the response from the other end of the connection could look like SSLv2 when it really was not. The payload happened to look like it was SSLv2, major version 8, but contained no data so the handshake never completed. I found if I forced the major version to be 2 or 3, I could get an error correctly claiming it was not a valid SSL error, but that only worked when the SslHandler was on the server only. I could not find a workaround for when it was on the client only.

I did experiment with dynamically inserting a ReadTimeoutHandler for the write of the handshake to see if I could at least timeout the handshake and get the connection to fail that way. This almost worked, the timeout did fire appropriately, but when the server got the event, it could not transmit the information about closing the channel back to the client. I did not understand why that was, that seemed to go into the SslHandler and just never come out.

Ultimately, I'd like to have a clean way to know of there is a mismatch on the SSL; both sides should be using it, or neither, and if that's not the case, I want to cleanly exit with a reasonable error message. Right now someone just ends up waiting forever for the handhake response, which is not desirable.