Repository: netty/netty
Issue: WebSocketServerHandshaker.handshake not call ByteBuf.release()

### Expected behavior
2023-02-08 01:32:25.622 [nioEventLoopGroup-3-11-] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:349)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
	io.netty.handler.codec.http.websocketx.WebSocketServerHandshaker13.newHandshakeResponse(WebSocketServerHandshaker13.java:143)
	io.netty.handler.codec.http.websocketx.WebSocketServerHandshaker.handshake(WebSocketServerHandshaker.java:194)
	io.netty.handler.codec.http.websocketx.WebSocketServerHandshaker.handshake(WebSocketServerHandshaker.java:169)
	com.game.netty.handler.WsServerProtocolHandler.lambda$channelRead$0(WsServerProtocolHandler.java:65)

### Actual behavior

### Steps to reproduce

### Minimal yet complete reproducer code (or URL to code)
public final ChannelFuture handshake(Channel channel, FullHttpRequest req, HttpHeaders responseHeaders, final ChannelPromise promise) {
        if (logger.isDebugEnabled()) {
            logger.debug("{} WebSocket version {} server handshake", channel, this.version());
        }

        ChannelPipeline p = channel.pipeline();
        if (p.get(HttpObjectAggregator.class) != null) {
            p.remove(HttpObjectAggregator.class);
        }

        if (p.get(HttpContentCompressor.class) != null) {
            p.remove(HttpContentCompressor.class);
        }

        ChannelHandlerContext ctx = p.context(HttpRequestDecoder.class);
        final String encoderName;
        if (ctx == null) {
            ctx = p.context(HttpServerCodec.class);
            if (ctx == null) {
                promise.setFailure(new IllegalStateException("No HttpDecoder and no HttpServerCodec in the pipeline"));
                return promise;
            }

            p.addBefore(ctx.name(), "wsencoder", this.newWebSocketEncoder());
            p.addBefore(ctx.name(), "wsdecoder", this.newWebsocketDecoder());
            encoderName = ctx.name();
        } else {
            p.replace(ctx.name(), "wsdecoder", this.newWebsocketDecoder());
            encoderName = p.context(HttpResponseEncoder.class).name();
            p.addBefore(encoderName, "wsencoder", this.newWebSocketEncoder());
        }

        FullHttpResponse response = this.newHandshakeResponse(req, responseHeaders);
        channel.writeAndFlush(response).addListener(new ChannelFutureListener() {
            public void operationComplete(ChannelFuture future) throws Exception {
                if (future.isSuccess()) {
                    ChannelPipeline p = future.channel().pipeline();
                    p.remove(encoderName);
                    promise.setSuccess();
                } else {
                    promise.setFailure(future.cause());
                }

            }
        });
       return promise;
 }
### Netty version
4.1.91.Final
### JVM version (e.g. `java -version`)

### OS version (e.g. `uname -a`)


Issue comments:
This is either because you don't have an `HttpRequestDecoder` or a `HttpServerCodec` in your pipeline, or something else in your pipeline dropped the handshake response in the write path.
If the pipeline checks fail, how can i release the handshake response.
> If the pipeline checks fail, how can i release the handshake response.

You can't. It's a bug in Netty that we'll fix with #13338

Pull request review comments:

Patch that resolved the issue:
diff --git a/codec-http/src/main/java/io/netty/handler/codec/http/websocketx/WebSocketServerHandshaker.java b/codec-http/src/main/java/io/netty/handler/codec/http/websocketx/WebSocketServerHandshaker.java
index 3c3fac5a49c..5271102b7eb 100644
--- a/codec-http/src/main/java/io/netty/handler/codec/http/websocketx/WebSocketServerHandshaker.java
+++ b/codec-http/src/main/java/io/netty/handler/codec/http/websocketx/WebSocketServerHandshaker.java
@@ -215,6 +215,7 @@ public final ChannelFuture handshake(Channel channel, FullHttpRequest req,
             if (ctx == null) {
                 promise.setFailure(
                         new IllegalStateException("No HttpDecoder and no HttpServerCodec in the pipeline"));
+                response.release();
                 return promise;
             }
             p.addBefore(ctx.name(), "wsencoder", newWebSocketEncoder());


Test patch: