Repository: netty/netty
Issue: JDKZlibEncoder or JDKZlibDecoder can't work correct with large data
Labels: defect

### Expected behavior
work 
### Actual behavior
decode error(protobuf decode error)
### Steps to reproduce
i have 2 server. and them communicate with tcp.
and the pipeline init code double with them.
when the send data > 16000 byte (maybe less). the decode side can't correct decode (protobuf decode error). 
### Minimal yet complete reproducer code (or URL to code)
decode error code:
Code:

        ChannelPipeline pipeline = ch.pipeline();
        pipeline.addLast(new ProtobufVarint32FrameDecoder());
        pipeline.addLast(ZlibCodecFactory.newZlibDecoder(ZlibWrapper.NONE));
        pipeline.addLast(new ProtobufDecoder(GameServer.ServerData.getDefaultInstance()));
        pipeline.addLast(new ProtobufVarint32LengthFieldPrepender());
        pipeline.addLast(ZlibCodecFactory.newZlibEncoder(ZlibWrapper.NONE));
        pipeline.addLast(new ProtobufEncoder());

i had 2 test:
     remove zlib, can be work.like this:
     
      ChannelPipeline pipeline = ch.pipeline();
        //receive
        pipeline.addLast(new ProtobufVarint32FrameDecoder());
        pipeline.addLast(new ProtobufDecoder(GameServer.ServerData.getDefaultInstance()));
        //send
        pipeline.addLast(new ProtobufVarint32LengthFieldPrepender());
        pipeline.addLast(new ProtobufEncoder());

change to the JZlibDecoder and JZlibEncoder can be work. like this:
        
        ChannelPipeline pipeline = ch.pipeline();
        //receive
        pipeline.addLast(new ProtobufVarint32FrameDecoder());
        pipeline.addLast(new JZlibDecoder(ZlibWrapper.NONE));
        pipeline.addLast(new ProtobufDecoder(GameServer.ServerData.getDefaultInstance()));
        //send
        pipeline.addLast(new ProtobufVarint32LengthFieldPrepender());
        pipeline.addLast(new JZlibEncoder(ZlibWrapper.NONE));
        pipeline.addLast(new ProtobufEncoder());
### Netty version
4.1.7
### JVM version (e.g. `java -version`)
java version "1.8.0_101"
### OS version (e.g. `uname -a`)
windows

Issue comments:
@lsqlebai can you please provide a full reproducer ? Also please update to latest netty version just to be sure.
sorry the netty version is 4.1.6.Final
@normanmaurer What would you want? my project is so large. maybe i can give u a demo when i free.
@lsqlebai I want you to upgrade to latest 4.1. version and try again if it happens. If it still happens provide me some way to reproduce.
> my project is so large. maybe i can give u a demo when i free.

Ideally folks that file issues will provide a minimal/complete (so we can run it, but don't have to understand your entire application logic) ... the easier it is to reproduce the more likely we will be able to figure out the issue (assuming its a Netty issue) and fix it.
@Scottmitch @normanmaurer i wrote a demo, how can i give you? by email or???
@Scottmitch @normanmaurer would u please try this code: https://github.com/lsqlebai/testzlib
that can be reproduce .
I took a brief look. It seems like the issue is in this case the `JdkZlibDecoder` happens to break the decode up into 2 buffers and the `JZlibDecoder` decodes into a single buffer, and the `ProtobufDecoder#decoder` assumes all content is in the same `ByteBuf`. See https://github.com/netty/netty/pull/6816 for a PR which makes the two decoders consistent.

Pull request review comments:

Patch that resolved the issue:
diff --git a/codec/src/main/java/io/netty/handler/codec/compression/JZlibDecoder.java b/codec/src/main/java/io/netty/handler/codec/compression/JZlibDecoder.java
index ad2ae299044..5d23bb8bd6f 100644
--- a/codec/src/main/java/io/netty/handler/codec/compression/JZlibDecoder.java
+++ b/codec/src/main/java/io/netty/handler/codec/compression/JZlibDecoder.java
@@ -107,16 +107,15 @@ protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) t
                 z.next_in = array;
                 z.next_in_index = 0;
             }
-            int oldNextInIndex = z.next_in_index;
+            final int oldNextInIndex = z.next_in_index;
 
             // Configure output.
-            int maxOutputLength = inputLength << 1;
-            ByteBuf decompressed = ctx.alloc().heapBuffer(maxOutputLength);
+            ByteBuf decompressed = ctx.alloc().heapBuffer(inputLength << 1);
 
             try {
                 loop: for (;;) {
-                    z.avail_out = maxOutputLength;
-                    decompressed.ensureWritable(maxOutputLength);
+                    decompressed.ensureWritable(z.avail_in << 1);
+                    z.avail_out = decompressed.writableBytes();
                     z.next_out = decompressed.array();
                     z.next_out_index = decompressed.arrayOffset() + decompressed.writerIndex();
                     int oldNextOutIndex = z.next_out_index;
diff --git a/codec/src/main/java/io/netty/handler/codec/compression/JdkZlibDecoder.java b/codec/src/main/java/io/netty/handler/codec/compression/JdkZlibDecoder.java
index 3594685a242..d9b4b914f19 100644
--- a/codec/src/main/java/io/netty/handler/codec/compression/JdkZlibDecoder.java
+++ b/codec/src/main/java/io/netty/handler/codec/compression/JdkZlibDecoder.java
@@ -167,25 +167,14 @@ protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) t
             inflater.setInput(array);
         }
 
-        int maxOutputLength = inflater.getRemaining() << 1;
-        ByteBuf decompressed = ctx.alloc().heapBuffer(maxOutputLength);
+        ByteBuf decompressed = ctx.alloc().heapBuffer(inflater.getRemaining() << 1);
         try {
             boolean readFooter = false;
-            byte[] outArray = decompressed.array();
             while (!inflater.needsInput()) {
+                byte[] outArray = decompressed.array();
                 int writerIndex = decompressed.writerIndex();
                 int outIndex = decompressed.arrayOffset() + writerIndex;
-                int length = decompressed.writableBytes();
-
-                if (length == 0) {
-                    // completely filled the buffer allocate a new one and start to fill it
-                    out.add(decompressed);
-                    decompressed = ctx.alloc().heapBuffer(maxOutputLength);
-                    outArray = decompressed.array();
-                    continue;
-                }
-
-                int outputLength = inflater.inflate(outArray, outIndex, length);
+                int outputLength = inflater.inflate(outArray, outIndex, decompressed.writableBytes());
                 if (outputLength > 0) {
                     decompressed.writerIndex(writerIndex + outputLength);
                     if (crc != null) {
@@ -208,6 +197,8 @@ protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) t
                         readFooter = true;
                     }
                     break;
+                } else {
+                    decompressed.ensureWritable(inflater.getRemaining() << 1);
                 }
             }
 


Test patch: