Repository: netty/netty
Issue: Consider making Snappy class public
Labels: feature

Maybe useful for users who want to use Snappy compression without handlers? (like Base64 in Netty)

https://groups.google.com/d/topic/netty/I8-M02b2ghg/discussion


Issue comments:
@trustin will take care.

Thanks, @normanmaurer 

Thank you very much!

@trustin and @normanmaurer I was trying to use the Snappy codec and it seems to me that the limit to the input `ByteBuf` is 36KB. Beyond this, the codec throws an `IndexOutOfBoundsException`. Is there an undocumented limit to the input size?

``` java
java.lang.ArrayIndexOutOfBoundsException: -30567

    at io.netty.buffer.HeapByteBufUtil.getInt(HeapByteBufUtil.java:48)
    at io.netty.buffer.UnpooledHeapByteBuf._getInt(UnpooledHeapByteBuf.java:378)
    at io.netty.buffer.UnpooledHeapByteBuf.getInt(UnpooledHeapByteBuf.java:373)
    at io.netty.handler.codec.compression.Snappy.encode(Snappy.java:103)
    at io.jarasandha.util.io.NettySnappyBug.compressUsingNetty(NettySnappyBug.java:51)
    at io.jarasandha.util.io.NettySnappyBug.testSnappy(NettySnappyBug.java:30)
```

``` java
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.buffer.Unpooled;
import io.netty.handler.codec.compression.Snappy;
import org.junit.Test;

import java.util.Random;

import static java.lang.String.format;

/**
 * Created by ashwin.jayaprakash on 9/11/16.
 */
public class NettySnappyBug {
    @Test
    public void testSnappy() {
        for (int multiplier = 1; multiplier < 125; multiplier++) {
            System.out.println("Trying out input size [" + multiplier + "]");

            byte[] bytes = new byte[multiplier * 1024];
            Random random = new Random();
            random.nextBytes(bytes);

            ByteBuf uncompressedByteBuf = Unpooled.wrappedBuffer(bytes);
            ByteBuf compressedByteBuf = null;
            try {
                compressedByteBuf = compressUsingNetty(ByteBufAllocator.DEFAULT, uncompressedByteBuf);
                System.out.println(format("Raw bytes [%s], compressed [%s]",
                        bytes.length, compressedByteBuf.readableBytes()));
            } finally {
                try {
                    uncompressedByteBuf.release();
                } finally {
                    if (compressedByteBuf != null) {
                        compressedByteBuf.release();
                    }
                }
            }

            System.out.println();
        }
    }

    private static ByteBuf compressUsingNetty(ByteBufAllocator allocator, ByteBuf rawByteBuf) {
        final Snappy snappy = new Snappy();
        final ByteBuf compressedByteBuf = allocator.buffer();
        try {
            snappy.encode(rawByteBuf, compressedByteBuf, rawByteBuf.readableBytes());
        } catch (Throwable t) {
            compressedByteBuf.release();
            throw t;
        }
        return compressedByteBuf;
    }
}
```

Have I made a wrong assumption about the API?

Thanks.

I guess I can answer my own question. I see here that the slice length is always at Short.MAX_VALUE:
https://github.com/netty/netty/blob/83c349ffa94d3992c4ee511d3625afc0c97c12bb/codec/src/main/java/io/netty/handler/codec/compression/SnappyFrameEncoder.java#L71


Pull request review comments:

Patch that resolved the issue:
diff --git a/codec/src/main/java/io/netty/handler/codec/compression/Snappy.java b/codec/src/main/java/io/netty/handler/codec/compression/Snappy.java
index df4e88a0821..9a02f3a0f3f 100644
--- a/codec/src/main/java/io/netty/handler/codec/compression/Snappy.java
+++ b/codec/src/main/java/io/netty/handler/codec/compression/Snappy.java
@@ -21,9 +21,9 @@
  * Uncompresses an input {@link ByteBuf} encoded with Snappy compression into an
  * output {@link ByteBuf}.
  *
- * See http://code.google.com/p/snappy/source/browse/trunk/format_description.txt
+ * See <a href="http://code.google.com/p/snappy/source/browse/trunk/format_description.txt">snappy format</a>.
  */
-class Snappy {
+public final class Snappy {
 
     private static final int MAX_HT_SIZE = 1 << 14;
     private static final int MIN_COMPRESSIBLE_BYTES = 15;
@@ -598,7 +598,7 @@ private static void validateOffset(int offset, int chunkSizeSoFar) {
      *
      * @param data The input data to calculate the CRC32C checksum of
      */
-    public static int calculateChecksum(ByteBuf data) {
+    static int calculateChecksum(ByteBuf data) {
         return calculateChecksum(data, data.readerIndex(), data.readableBytes());
     }
 
@@ -608,7 +608,7 @@ public static int calculateChecksum(ByteBuf data) {
      *
      * @param data The input data to calculate the CRC32C checksum of
      */
-    public static int calculateChecksum(ByteBuf data, int offset, int length) {
+    static int calculateChecksum(ByteBuf data, int offset, int length) {
         Crc32c crc32 = new Crc32c();
         try {
             if (data.hasArray()) {


Test patch: