Repository: netty/netty
Issue: OutOfDirectMemoryError for large uploads using HttpPostMultipartRequestDecoder

### Expected behavior

With a `HttpDataFactory` that has `useDisk=true`, I thought files of any size could potentially be uploaded.

### Actual behavior

Example error from below unit test:

```
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 4194304 byte(s) of direct memory (used: 62914560, max: 64487424)
	at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:775)
	at io.netty.util.internal.PlatformDependent.reallocateDirectNoCleaner(PlatformDependent.java:748)
	at io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.reallocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:34)
	at io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.reallocateDirect(UnpooledByteBufAllocator.java:194)
	at io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.capacity(UnpooledUnsafeNoCleanerDirectByteBuf.java:52)
	at io.netty.buffer.AbstractByteBuf.ensureWritable0(AbstractByteBuf.java:307)
	at io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:282)
	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1105)
	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1098)
	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1089)
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.offer(HttpPostMultipartRequestDecoder.java:351)
	at NettyUploadTest.itCanProcessLargeFiles(NettyUploadTest.java:46)
// snip
```

### Steps to reproduce

Set `-Xmx64m` and run below unit test.

### Minimal yet complete reproducer code (or URL to code)

```java
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.handler.codec.http.*;
import io.netty.handler.codec.http.multipart.DefaultHttpDataFactory;
import io.netty.handler.codec.http.multipart.FileUpload;
import io.netty.handler.codec.http.multipart.HttpDataFactory;
import io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder;
import org.junit.Test;

import java.nio.charset.StandardCharsets;
import java.util.Arrays;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;

public class NettyUploadTest {

    @Test
    public void itCanProcessLargeFiles() throws Exception {

        int fileSize = 100_000_000; // set Xmx to a number lower than this and it crashes
        int bytesPerChunk = 1_000_000;

        String prefix = "--861fbeab-cd20-470c-9609-d40a0f704466\n" +
            "Content-Disposition: form-data; name=\"image\"; filename=\"guangzhou.jpeg\"\n" +
            "Content-Type: image/jpeg\n" +
            "Content-Length: " + fileSize + "\n" +
            "\n";

        String suffix = "\n" +
            "--861fbeab-cd20-470c-9609-d40a0f704466--\n";

        HttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.POST, "/upload");
        request.headers().set("content-type", "multipart/form-data; boundary=861fbeab-cd20-470c-9609-d40a0f704466");
        request.headers().set("content-length", prefix.length() + fileSize + suffix.length());

        HttpDataFactory factory = new DefaultHttpDataFactory(true);
        HttpPostMultipartRequestDecoder decoder = new HttpPostMultipartRequestDecoder(factory, request);
        decoder.offer(new DefaultHttpContent(Unpooled.wrappedBuffer(prefix.getBytes(StandardCharsets.UTF_8))));

        byte[] body = new byte[bytesPerChunk];
        Arrays.fill(body, (byte)1);
        for (int i = 0; i < fileSize / bytesPerChunk; i++) {
            ByteBuf content = Unpooled.wrappedBuffer(body, 0, bytesPerChunk);
            decoder.offer(new DefaultHttpContent(content)); // **OutOfMemory here**
            content.release();
        }

        decoder.offer(new DefaultHttpContent(Unpooled.wrappedBuffer(suffix.getBytes(StandardCharsets.UTF_8))));
        decoder.offer(new DefaultLastHttpContent());
        FileUpload data = (FileUpload) decoder.getBodyHttpDatas().get(0);
        assertThat((int)data.length(), is(fileSize));
        assertThat(data.get().length, is(fileSize));

        factory.cleanAllHttpData();

    }

}
```

### Netty version

Tested on `4.1.56` and `4.1.58`.

### JVM version (e.g. `java -version`)

jdk1.8.0_162 and 12

### OS version (e.g. `uname -a`)

Windows 10

Issue comments:
@fredericBregier  It could be related to https://github.com/netty/netty/pull/10623 given that right now file isn't written until a delimiter is found?
I think so too... I think we should never the commit in question for now as while it fixes some "perf issues" when running in with paranoid leak detection I don't think it has any other benefits in real world use-cases. @fredericBregier WDYT ?
Hi, the issue is partially related but yet an issue. 
- the file is totally in the buffer while reading: previously it could be in memory but by chunk
- once the file is over, if the "disk" based HttpData is used, the buffer is written to a temprary file, therefore leaving the memory free
- Note that another bug was fixed in the same time since before buffers were free (`discardReadBytes`) wrongly

I agree that this should be changed to adapt the solution with the new way to find the delimiter. Perhaps this?
Currently:
- The content is added to the HttpData along slices (see https://github.com/netty/netty/blob/5c522916f796aef7a9bf7a7b234bfb81232b52d0/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java#L1191 ).
- But only once all slices are found (https://github.com/netty/netty/blob/5c522916f796aef7a9bf7a7b234bfb81232b52d0/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java#L1171)

If it is written (in Disk mode or Mixed mode and if size is greater than limit), the buffer might be cleared (free), so one could try to change this such as:
- each time a slice is found (even if the delimiter is not found), one could add the content to the HttpData, but carefully take into consideration that perhaps not all bufer shall be taken (the end of the buffer could be in the middle of the delimiter, to not take into account)
- therefore, try to release the memory (careful, since if Memory based, must not be released at all or bad content will occur) as in https://github.com/netty/netty/blob/5c522916f796aef7a9bf7a7b234bfb81232b52d0/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java#L348 https://github.com/netty/netty/blob/5c522916f796aef7a9bf7a7b234bfb81232b52d0/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java#L349) (we cannot simply reset the writerIndex and readerIndex to the beginning for this HttpData since the HttpData could be in Memory only)

Not sure it will work or easy to implement, but I think the idea is there...

Hi all, I propose a fix for this (adding a test inspired from the one given, testing both in Memory and on Disk behaviors)
Thanks to both of you. The new version is working well now.

The performance degradation with paranoid detection is very noticeable (e.g. a test with a lot of data moving was taking 1 second in `4.1.58` takes 30 seconds in `4.1.59`). For a while I thought there was a performance issue in the new version before I remembered what Frederic said about leak detection performance.

So if anyone else is experiencing this and wondering what is going on, try disabling PARANOID for the slow tests.
@danielflower or maybe wait for #11001 which fixes also this PARANOID issue, and improves without this level also performances (about 4 times)... ;-)
The #11001 PR is merged now.

Pull request review comments:
Calling this method `findLineBreak` would read better with the capitalisation.

I think calling it "line break" also makes sense for the parameters and variables in `findDelimiter`.
```suggestion
     * Try to find the delimiter, with LF or CRLF in front of it (added as delimiters) if needed
```
As for how this algorithm is implemented… check out the tools in `AbstractSearchProcessorFactory`. You could use them to search for the delimiter directly, then check if they are preceded by a line break or not. That should be less code to write, and probably be faster too.
Should we check if we really get an LF here?
Do we need the `while (undecodedChunk.isReadable())`, or can it be an `if`? Looks like we always either throw or return, if we have undecoded data.
We should give `getBytes` a `Charset` to work with, otherwise it might pick some random default platform encoding.
I wouldn't mind if this explanation was elaborated a little bit.
Shouldn't the "delimiter found" aka. "last" be `true` here?
The test data is 100 MB. Can't we get meaningful coverage with something smaller?
I think it should be possible to extract more common code. And perhaps even make each test case its own test method.
Same here with duplication and splitting into separate test cases.
Agree
Agree
I will check, but not so sure: I need to check that this is not doing getByte method call or equivalent.
You're probably right. Let me check
Since this is LF or CRLF, we need to check if CR at first position, which means LF is following.
And we need to get after line break, whatever the form.
Originally, there were none. I was surprised, but it was.
Of course I may use Charset, but hoping it will not break anything.
I agree. What about:

Rewite buffer in order to skip lengthToSkip bytes from current readerIndex, such that any readable bytes available after readerIndex + lengthToSkip (so before writerIndex) are moved at readerIndex position, therefore decreasing writerIndex of lengthToSkip at the end of the process.
The delimiter is not found yet. We are in the middle:
- one CRLF is found but not delimiter
- therefore we can at least add the content up to CRLF (not included) but we can't ensure the delimiter is not at all there (could be in the next chunk, using previous CRLF still available)
So it has to be false
It was the original code provided by issue submitter.
But I agree, we can limit the size for something like 10 MB (using 100KB chunk size)
Not really, since the ending asserts are differents. Only the releasing are the same (due to my fix).

I can split into several tests of course, one per Factory type.
Again asserts are differents.
But agree to split into different tests using each a factory type.
@chrisvest I test your proposal. The results are not that good using the benchmark (almost 3 times slower).

First, in order to see if what I am doing is correct, here is a copy of the method:

     static int findDelimiter(ByteBuf buffer, int index, SearchProcessorFactory factory, int delimiterLength,
                             boolean precededByLineBreak) {
        SearchProcessor processor = factory.newSearchProcessor();
        final int readerIndex = buffer.readerIndex();
        final int writerIndex = buffer.writerIndex();
        int length = writerIndex - index;
        while (delimiterLength <= length) {
            int startDelimiter = buffer.forEachByte(index, length, processor);
            if (startDelimiter < 0) {
                // No delimiter found
                return -1;
            }
            startDelimiter -= delimiterLength - 1;
            if (precededByLineBreak) {
                if (startDelimiter > readerIndex && buffer.getByte(startDelimiter - 1) == HttpConstants.LF) {
                    startDelimiter--;
                    if (startDelimiter > readerIndex && buffer.getByte(startDelimiter - 1) == HttpConstants.CR) {
                        startDelimiter--;
                    }
                } else {
                    // A delimiter could be found further, so iterate
                    index = startDelimiter + 1;
                    length = writerIndex - index;
                    continue;
                }
            }
            return startDelimiter - readerIndex;
        }
        return -1;
    }

I set the searchProcessorFactory once for the request and reuse it as per the doc said.

I try both KMP and newBitap. Here are the results compared to the previous one. 

Note that your indication makes me change a bit the algorithm also using `bytesBefore` by first searching the delimiter then check if any Line Break is preceding if asked for. 
The results are almost the same (very small improvement), so I will however update using this new algorithm but globally the same (just search are reversed, first delimiter then line break if needed).

Here are the results:

Original algorithm using `bytesBefore`

     Benchmark                                                                           Mode  Cnt  Score   Error   Units
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigAdvancedLevel   thrpt    6  4,253 ± 0,333  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigDisabledLevel   thrpt    6  4,422 ± 0,250  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigParanoidLevel   thrpt    6  0,877 ± 0,014  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigSimpleLevel     thrpt    6  4,151 ± 0,481  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighAdvancedLevel  thrpt    6  2,167 ± 0,098  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighDisabledLevel  thrpt    6  2,520 ± 0,043  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighParanoidLevel  thrpt    6  0,177 ± 0,003  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighSimpleLevel    thrpt    6  2,419 ± 0,061  ops/ms

SearchProcessor version reuse KMP

     Benchmark                                                                           Mode  Cnt  Score   Error   Units
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigAdvancedLevel   thrpt    6  1,309 ± 0,015  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigDisabledLevel   thrpt    6  1,740 ± 0,029  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigParanoidLevel   thrpt    6  0,710 ± 0,008  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigSimpleLevel     thrpt    6  1,233 ± 0,703  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighAdvancedLevel  thrpt    6  1,045 ± 0,008  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighDisabledLevel  thrpt    6  1,544 ± 0,604  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighParanoidLevel  thrpt    6  0,309 ± 0,012  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighSimpleLevel    thrpt    6  1,104 ± 0,097  ops/ms

SearchProcessor version reuse newBitap

     Benchmark                                                                           Mode  Cnt  Score   Error   Units
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigAdvancedLevel   thrpt    6  1,029 ± 0,002  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigDisabledLevel   thrpt    6  1,828 ± 0,052  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigParanoidLevel   thrpt    6  0,703 ± 0,084  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderBigSimpleLevel     thrpt    6  1,033 ± 0,003  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighAdvancedLevel  thrpt    6  0,872 ± 0,016  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighDisabledLevel  thrpt    6  1,417 ± 0,027  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighParanoidLevel  thrpt    6  0,300 ± 0,009  ops/ms
     HttpPostMultipartRequestDecoderBenchmark.multipartRequestDecoderHighSimpleLevel    thrpt    6  0,873 ± 0,011  ops/ms

I split into several tests, refactoring code.
You were right!
Done and all tests passed
Done: split and refactoring tests
Done tests are ok
@chrisvest I looked into the `forEachByte` method implementation and I understand the reason:
- there is a loop using `_getByte()` for each byte
- while `byteBefore` might use the same `_getByte()` for each byte but might also try to use optimized access using long pattern (8 times reading operation less at most)

That's the only clue I can think of to explain the difference between the 2 implementations.
bytesBefore, if is going forward should use the SWAR indexOf I have referenced in another comment AFAIK https://github.com/netty/netty/commit/9a02832fdb10afbc09f144a462d604176fb11049
It's branchless and perform batch read so should be pretty fast..
Happy to see that is working as designed :)
And, as said, I am sure stomp text can benefit a lot by this 
@franz1981 So I think I've got the current fastest implementation using bytesBefore and the underlying SWAR indexOf, thanks to you! ;-)

I see… we only get here because `findLineBreak` told us there was a line break at this position, so if we read CR then the next byte will necessarily be an LF.
Looking at this one… both `DiskAttribute` and `MemoryAttribute` already do this check themselves, so I wonder if we can just remove the check here and let the delegate take care of it. Then we won't compute the byte array twice.
Will have a look to it to check
You're right! I change it
Done

Patch that resolved the issue:
diff --git a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/AbstractMemoryHttpData.java b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/AbstractMemoryHttpData.java
index 5cc6ec834ef..14071dad606 100644
--- a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/AbstractMemoryHttpData.java
+++ b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/AbstractMemoryHttpData.java
@@ -106,6 +106,9 @@ public void addContent(ByteBuf buffer, boolean last)
             size += localsize;
             if (byteBuf == null) {
                 byteBuf = buffer;
+            } else if (localsize == 0) {
+                // Nothing to add and byteBuf already exists
+                buffer.release();
             } else if (byteBuf instanceof CompositeByteBuf) {
                 CompositeByteBuf cbb = (CompositeByteBuf) byteBuf;
                 cbb.addComponent(true, buffer);
diff --git a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostBodyUtil.java b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostBodyUtil.java
index 18a1686635e..13f75f55bf2 100644
--- a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostBodyUtil.java
+++ b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostBodyUtil.java
@@ -16,6 +16,7 @@
 package io.netty.handler.codec.http.multipart;
 
 import io.netty.buffer.ByteBuf;
+import io.netty.handler.codec.http.HttpConstants;
 
 /**
  * Shared Static object between HttpMessageDecoder, HttpPostRequestDecoder and HttpPostRequestEncoder
@@ -150,4 +151,85 @@ static int findEndOfString(String sb) {
         return result;
     }
 
+    /**
+     * Try to find LF or CRLF as Line Breaking
+     *
+     * @param buffer the buffer to search in
+     * @param index the index to start from in the buffer
+     * @return a relative position from index > 0 if LF or CRLF is found
+     *         or < 0 if not found
+     */
+    static int findLineBreak(ByteBuf buffer, int index) {
+        int toRead = buffer.readableBytes() - (index - buffer.readerIndex());
+        int posFirstChar = buffer.bytesBefore(index, toRead, HttpConstants.LF);
+        if (posFirstChar == -1) {
+            // No LF, so neither CRLF
+            return  -1;
+        }
+        if (posFirstChar > 0 && buffer.getByte(index + posFirstChar - 1) == HttpConstants.CR) {
+            posFirstChar--;
+        }
+        return posFirstChar;
+    }
+
+    /**
+     * Try to find the delimiter, with LF or CRLF in front of it (added as delimiters) if needed
+     *
+     * @param buffer the buffer to search in
+     * @param index the index to start from in the buffer
+     * @param delimiter the delimiter as byte array
+     * @param precededByLineBreak true if it must be preceded by LF or CRLF, else false
+     * @return a relative position from index > 0 if delimiter found designing the start of it
+     *         (including LF or CRLF is asked)
+     *         or a number < 0 if delimiter is not found
+     * @throws IndexOutOfBoundsException
+     *         if {@code offset + delimiter.length} is greater than {@code buffer.capacity}
+     */
+    static int findDelimiter(ByteBuf buffer, int index, byte[] delimiter, boolean precededByLineBreak) {
+        final int delimiterLength = delimiter.length;
+        final int readerIndex = buffer.readerIndex();
+        final int writerIndex = buffer.writerIndex();
+        int toRead = writerIndex - index;
+        int newOffset = index;
+        boolean delimiterNotFound = true;
+        while (delimiterNotFound && delimiterLength <= toRead) {
+            // Find first position: delimiter
+            int posDelimiter = buffer.bytesBefore(newOffset, toRead, delimiter[0]);
+            if (posDelimiter < 0) {
+                return -1;
+            }
+            newOffset += posDelimiter;
+            toRead -= posDelimiter;
+            // Now check for delimiter
+            delimiterNotFound = false;
+            for (int i = 0; i < delimiterLength; i++) {
+                if (buffer.getByte(newOffset + i) != delimiter[i]) {
+                    newOffset++;
+                    toRead--;
+                    delimiterNotFound = true;
+                    break;
+                }
+            }
+            if (!delimiterNotFound) {
+                // Delimiter found, find if necessary: LF or CRLF
+                if (precededByLineBreak && newOffset > readerIndex) {
+                    if (buffer.getByte(newOffset - 1) == HttpConstants.LF) {
+                        newOffset--;
+                        // Check if CR before: not mandatory to be there
+                        if (newOffset > readerIndex && buffer.getByte(newOffset - 1) == HttpConstants.CR) {
+                            newOffset--;
+                        }
+                    } else {
+                        // Delimiter with Line Break could be further: iterate after first char of delimiter
+                        newOffset++;
+                        toRead--;
+                        delimiterNotFound = true;
+                        continue;
+                    }
+                }
+                return newOffset - readerIndex;
+            }
+        }
+        return -1;
+    }
 }
diff --git a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java
index e67c0103d8e..ebf88f23360 100644
--- a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java
+++ b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoder.java
@@ -551,7 +551,7 @@ private InterfaceHttpData decodeMultipart(MultiPartStatus state) {
                 }
             }
             // load data
-            if (!loadDataMultipart(undecodedChunk, multipartDataBoundary, currentAttribute)) {
+            if (!loadDataMultipartOptimized(undecodedChunk, multipartDataBoundary, currentAttribute)) {
                 // Delimiter is not found. Need more chunks.
                 return null;
             }
@@ -648,7 +648,7 @@ private InterfaceHttpData findMultipartDelimiter(String delimiter, MultiPartStat
         skipOneLine();
         String newline;
         try {
-            newline = readDelimiter(undecodedChunk, delimiter);
+            newline = readDelimiterOptimized(undecodedChunk, delimiter, charset);
         } catch (NotEnoughDataDecoderException ignored) {
             undecodedChunk.readerIndex(readerIndex);
             return null;
@@ -688,7 +688,7 @@ private InterfaceHttpData findMultipartDisposition() {
             String newline;
             try {
                 skipControlCharacters(undecodedChunk);
-                newline = readLine(undecodedChunk, charset);
+                newline = readLineOptimized(undecodedChunk, charset);
             } catch (NotEnoughDataDecoderException ignored) {
                 undecodedChunk.readerIndex(readerIndex);
                 return null;
@@ -918,7 +918,7 @@ protected InterfaceHttpData getFileUpload(String delimiter) {
             }
         }
         // load data as much as possible
-        if (!loadDataMultipart(undecodedChunk, delimiter, currentFileUpload)) {
+        if (!loadDataMultipartOptimized(undecodedChunk, delimiter, currentFileUpload)) {
             // Delimiter is not found. Need more chunks.
             return null;
         }
@@ -998,83 +998,32 @@ private void cleanMixedAttributes() {
      *             Need more chunks and reset the {@code readerIndex} to the previous
      *             value
      */
-    private static String readLineStandard(ByteBuf undecodedChunk, Charset charset) {
+    private static String readLineOptimized(ByteBuf undecodedChunk, Charset charset) {
         int readerIndex = undecodedChunk.readerIndex();
-        ByteBuf line = undecodedChunk.alloc().heapBuffer(64);
+        ByteBuf line = null;
         try {
-            while (undecodedChunk.isReadable()) {
-                byte nextByte = undecodedChunk.readByte();
-                if (nextByte == HttpConstants.CR) {
-                    // check but do not changed readerIndex
-                    nextByte = undecodedChunk.getByte(undecodedChunk.readerIndex());
-                    if (nextByte == HttpConstants.LF) {
-                        // force read
-                        undecodedChunk.readByte();
-                        return line.toString(charset);
-                    } else {
-                        // Write CR (not followed by LF)
-                        line.writeByte(HttpConstants.CR);
-                    }
-                } else if (nextByte == HttpConstants.LF) {
-                    return line.toString(charset);
-                } else {
-                    line.writeByte(nextByte);
+            if (undecodedChunk.isReadable()) {
+                int posLfOrCrLf = HttpPostBodyUtil.findLineBreak(undecodedChunk, undecodedChunk.readerIndex());
+                if (posLfOrCrLf <= 0) {
+                    throw new NotEnoughDataDecoderException();
                 }
-            }
-        } catch (IndexOutOfBoundsException e) {
-            undecodedChunk.readerIndex(readerIndex);
-            throw new NotEnoughDataDecoderException(e);
-        } finally {
-            line.release();
-        }
-        undecodedChunk.readerIndex(readerIndex);
-        throw new NotEnoughDataDecoderException();
-    }
+                try {
+                    line = undecodedChunk.alloc().heapBuffer(posLfOrCrLf);
+                    line.writeBytes(undecodedChunk, posLfOrCrLf);
 
-    /**
-     * Read one line up to the CRLF or LF
-     *
-     * @return the String from one line
-     * @throws NotEnoughDataDecoderException
-     *             Need more chunks and reset the {@code readerIndex} to the previous
-     *             value
-     */
-    private static String readLine(ByteBuf undecodedChunk, Charset charset) {
-        if (!undecodedChunk.hasArray()) {
-            return readLineStandard(undecodedChunk, charset);
-        }
-        SeekAheadOptimize sao = new SeekAheadOptimize(undecodedChunk);
-        int readerIndex = undecodedChunk.readerIndex();
-        ByteBuf line = undecodedChunk.alloc().heapBuffer(64);
-        try {
-            while (sao.pos < sao.limit) {
-                byte nextByte = sao.bytes[sao.pos++];
-                if (nextByte == HttpConstants.CR) {
-                    if (sao.pos < sao.limit) {
-                        nextByte = sao.bytes[sao.pos++];
-                        if (nextByte == HttpConstants.LF) {
-                            sao.setReadPosition(0);
-                            return line.toString(charset);
-                        } else {
-                            // Write CR (not followed by LF)
-                            sao.pos--;
-                            line.writeByte(HttpConstants.CR);
-                        }
-                    } else {
-                        line.writeByte(nextByte);
+                    byte nextByte = undecodedChunk.readByte();
+                    if (nextByte == HttpConstants.CR) {
+                        // force read next byte since LF is the following one
+                        undecodedChunk.readByte();
                     }
-                } else if (nextByte == HttpConstants.LF) {
-                    sao.setReadPosition(0);
                     return line.toString(charset);
-                } else {
-                    line.writeByte(nextByte);
+                } finally {
+                    line.release();
                 }
             }
         } catch (IndexOutOfBoundsException e) {
             undecodedChunk.readerIndex(readerIndex);
             throw new NotEnoughDataDecoderException(e);
-        } finally {
-            line.release();
         }
         undecodedChunk.readerIndex(readerIndex);
         throw new NotEnoughDataDecoderException();
@@ -1095,23 +1044,19 @@ private static String readLine(ByteBuf undecodedChunk, Charset charset) {
      *             Need more chunks and reset the {@code readerIndex} to the previous
      *             value
      */
-    private static String readDelimiterStandard(ByteBuf undecodedChunk, String delimiter) {
-        int readerIndex = undecodedChunk.readerIndex();
+    private static String readDelimiterOptimized(ByteBuf undecodedChunk, String delimiter, Charset charset) {
+        final int readerIndex = undecodedChunk.readerIndex();
+        final byte[] bdelimiter = delimiter.getBytes(charset);
+        final int delimiterLength = bdelimiter.length;
         try {
-            StringBuilder sb = new StringBuilder(64);
-            int delimiterPos = 0;
-            int len = delimiter.length();
-            while (undecodedChunk.isReadable() && delimiterPos < len) {
-                byte nextByte = undecodedChunk.readByte();
-                if (nextByte == delimiter.charAt(delimiterPos)) {
-                    delimiterPos++;
-                    sb.append((char) nextByte);
-                } else {
-                    // delimiter not found so break here !
-                    undecodedChunk.readerIndex(readerIndex);
-                    throw new NotEnoughDataDecoderException();
-                }
+            int delimiterPos = HttpPostBodyUtil.findDelimiter(undecodedChunk, readerIndex, bdelimiter, false);
+            if (delimiterPos < 0) {
+                // delimiter not found so break here !
+                undecodedChunk.readerIndex(readerIndex);
+                throw new NotEnoughDataDecoderException();
             }
+            StringBuilder sb = new StringBuilder(delimiter);
+            undecodedChunk.readerIndex(readerIndex + delimiterPos + delimiterLength);
             // Now check if either opening delimiter or closing delimiter
             if (undecodedChunk.isReadable()) {
                 byte nextByte = undecodedChunk.readByte();
@@ -1176,127 +1121,28 @@ private static String readDelimiterStandard(ByteBuf undecodedChunk, String delim
     }
 
     /**
-     * Read one line up to --delimiter or --delimiter-- and if existing the CRLF
-     * or LF. Note that CRLF or LF are mandatory for opening delimiter
-     * (--delimiter) but not for closing delimiter (--delimiter--) since some
-     * clients does not include CRLF in this case.
+     * Rewrite buffer in order to skip lengthToSkip bytes from current readerIndex,
+     * such that any readable bytes available after readerIndex + lengthToSkip (so before writerIndex)
+     * are moved at readerIndex position,
+     * therefore decreasing writerIndex of lengthToSkip at the end of the process.
      *
-     * @param delimiter
-     *            of the form --string, such that '--' is already included
-     * @return the String from one line as the delimiter searched (opening or
-     *         closing)
-     * @throws NotEnoughDataDecoderException
-     *             Need more chunks and reset the readerInder to the previous
-     *             value
+     * @param buffer the buffer to rewrite from current readerIndex
+     * @param lengthToSkip the size to skip from readerIndex
      */
-    private static String readDelimiter(ByteBuf undecodedChunk, String delimiter) {
-        if (!undecodedChunk.hasArray()) {
-            return readDelimiterStandard(undecodedChunk, delimiter);
+    private static void rewriteCurrentBuffer(ByteBuf buffer, int lengthToSkip) {
+        if (lengthToSkip == 0) {
+            return;
         }
-        SeekAheadOptimize sao = new SeekAheadOptimize(undecodedChunk);
-        int readerIndex = undecodedChunk.readerIndex();
-        int delimiterPos = 0;
-        int len = delimiter.length();
-        try {
-            StringBuilder sb = new StringBuilder(64);
-            // check conformity with delimiter
-            while (sao.pos < sao.limit && delimiterPos < len) {
-                byte nextByte = sao.bytes[sao.pos++];
-                if (nextByte == delimiter.charAt(delimiterPos)) {
-                    delimiterPos++;
-                    sb.append((char) nextByte);
-                } else {
-                    // delimiter not found so break here !
-                    undecodedChunk.readerIndex(readerIndex);
-                    throw new NotEnoughDataDecoderException();
-                }
-            }
-            // Now check if either opening delimiter or closing delimiter
-            if (sao.pos < sao.limit) {
-                byte nextByte = sao.bytes[sao.pos++];
-                if (nextByte == HttpConstants.CR) {
-                    // first check for opening delimiter
-                    if (sao.pos < sao.limit) {
-                        nextByte = sao.bytes[sao.pos++];
-                        if (nextByte == HttpConstants.LF) {
-                            sao.setReadPosition(0);
-                            return sb.toString();
-                        } else {
-                            // error CR without LF
-                            // delimiter not found so break here !
-                            undecodedChunk.readerIndex(readerIndex);
-                            throw new NotEnoughDataDecoderException();
-                        }
-                    } else {
-                        // error since CR must be followed by LF
-                        // delimiter not found so break here !
-                        undecodedChunk.readerIndex(readerIndex);
-                        throw new NotEnoughDataDecoderException();
-                    }
-                } else if (nextByte == HttpConstants.LF) {
-                    // same first check for opening delimiter where LF used with
-                    // no CR
-                    sao.setReadPosition(0);
-                    return sb.toString();
-                } else if (nextByte == '-') {
-                    sb.append('-');
-                    // second check for closing delimiter
-                    if (sao.pos < sao.limit) {
-                        nextByte = sao.bytes[sao.pos++];
-                        if (nextByte == '-') {
-                            sb.append('-');
-                            // now try to find if CRLF or LF there
-                            if (sao.pos < sao.limit) {
-                                nextByte = sao.bytes[sao.pos++];
-                                if (nextByte == HttpConstants.CR) {
-                                    if (sao.pos < sao.limit) {
-                                        nextByte = sao.bytes[sao.pos++];
-                                        if (nextByte == HttpConstants.LF) {
-                                            sao.setReadPosition(0);
-                                            return sb.toString();
-                                        } else {
-                                            // error CR without LF
-                                            // delimiter not found so break here !
-                                            undecodedChunk.readerIndex(readerIndex);
-                                            throw new NotEnoughDataDecoderException();
-                                        }
-                                    } else {
-                                        // error CR without LF
-                                        // delimiter not found so break here !
-                                        undecodedChunk.readerIndex(readerIndex);
-                                        throw new NotEnoughDataDecoderException();
-                                    }
-                                } else if (nextByte == HttpConstants.LF) {
-                                    sao.setReadPosition(0);
-                                    return sb.toString();
-                                } else {
-                                    // No CRLF but ok however (Adobe Flash
-                                    // uploader)
-                                    // minus 1 since we read one char ahead but
-                                    // should not
-                                    sao.setReadPosition(1);
-                                    return sb.toString();
-                                }
-                            }
-                            // FIXME what do we do here?
-                            // either considering it is fine, either waiting for
-                            // more data to come?
-                            // lets try considering it is fine...
-                            sao.setReadPosition(0);
-                            return sb.toString();
-                        }
-                        // whatever now => error since incomplete
-                        // only one '-' => not enough or whatever not enough
-                        // element
-                    }
-                }
-            }
-        } catch (IndexOutOfBoundsException e) {
-            undecodedChunk.readerIndex(readerIndex);
-            throw new NotEnoughDataDecoderException(e);
+        final int readerIndex = buffer.readerIndex();
+        final int readableBytes = buffer.readableBytes();
+        if (readableBytes == lengthToSkip) {
+            buffer.readerIndex(readerIndex);
+            buffer.writerIndex(readerIndex);
+            return;
         }
-        undecodedChunk.readerIndex(readerIndex);
-        throw new NotEnoughDataDecoderException();
+        buffer.setBytes(readerIndex, buffer, readerIndex + lengthToSkip, readableBytes - lengthToSkip);
+        buffer.readerIndex(readerIndex);
+        buffer.writerIndex(readerIndex + readableBytes - lengthToSkip);
     }
 
     /**
@@ -1305,91 +1151,50 @@ private static String readDelimiter(ByteBuf undecodedChunk, String delimiter) {
      * @return {@code true} if the last chunk is loaded (boundary delimiter found), {@code false} if need more chunks
      * @throws ErrorDataDecoderException
      */
-    private static boolean loadDataMultipartStandard(ByteBuf undecodedChunk, String delimiter, HttpData httpData) {
-        final int startReaderIndex = undecodedChunk.readerIndex();
-        final int delimeterLength = delimiter.length();
-        int index = 0;
-        int lastPosition = startReaderIndex;
-        byte prevByte = HttpConstants.LF;
-        boolean delimiterFound = false;
-        while (undecodedChunk.isReadable()) {
-            final byte nextByte = undecodedChunk.readByte();
-            // Check the delimiter
-            if (prevByte == HttpConstants.LF && nextByte == delimiter.codePointAt(index)) {
-                index++;
-                if (delimeterLength == index) {
-                    delimiterFound = true;
-                    break;
-                }
-                continue;
-            }
-            lastPosition = undecodedChunk.readerIndex();
-            if (nextByte == HttpConstants.LF) {
-                index = 0;
-                lastPosition -= (prevByte == HttpConstants.CR)? 2 : 1;
-            }
-            prevByte = nextByte;
-        }
-        if (prevByte == HttpConstants.CR) {
-            lastPosition--;
-        }
-        ByteBuf content = undecodedChunk.retainedSlice(startReaderIndex, lastPosition - startReaderIndex);
-        try {
-            httpData.addContent(content, delimiterFound);
-        } catch (IOException e) {
-            throw new ErrorDataDecoderException(e);
-        }
-        undecodedChunk.readerIndex(lastPosition);
-        return delimiterFound;
-    }
-
-    /**
-     * Load the field value from a Multipart request
-     *
-     * @return {@code true} if the last chunk is loaded (boundary delimiter found), {@code false} if need more chunks
-     * @throws ErrorDataDecoderException
-     */
-    private static boolean loadDataMultipart(ByteBuf undecodedChunk, String delimiter, HttpData httpData) {
-        if (!undecodedChunk.hasArray()) {
-            return loadDataMultipartStandard(undecodedChunk, delimiter, httpData);
+    private static boolean loadDataMultipartOptimized(ByteBuf undecodedChunk, String delimiter, HttpData httpData) {
+        if (!undecodedChunk.isReadable()) {
+            return false;
         }
-        final SeekAheadOptimize sao = new SeekAheadOptimize(undecodedChunk);
         final int startReaderIndex = undecodedChunk.readerIndex();
-        final int delimeterLength = delimiter.length();
-        int index = 0;
-        int lastRealPos = sao.pos;
-        byte prevByte = HttpConstants.LF;
-        boolean delimiterFound = false;
-        while (sao.pos < sao.limit) {
-            final byte nextByte = sao.bytes[sao.pos++];
-            // Check the delimiter
-            if (prevByte == HttpConstants.LF && nextByte == delimiter.codePointAt(index)) {
-                index++;
-                if (delimeterLength == index) {
-                    delimiterFound = true;
-                    break;
+        final byte[] bdelimiter = delimiter.getBytes(httpData.getCharset());
+        int posDelimiter = HttpPostBodyUtil.findDelimiter(undecodedChunk, startReaderIndex, bdelimiter, true);
+        if (posDelimiter < 0) {
+            // Not found but however perhaps because incomplete so search LF or CRLF
+            posDelimiter = HttpPostBodyUtil.findLineBreak(undecodedChunk, startReaderIndex);
+            if (posDelimiter < 0) {
+                // not found so this chunk can be fully added
+                ByteBuf content = undecodedChunk.copy();
+                try {
+                    httpData.addContent(content, false);
+                } catch (IOException e) {
+                    throw new ErrorDataDecoderException(e);
                 }
-                continue;
-            }
-            lastRealPos = sao.pos;
-            if (nextByte == HttpConstants.LF) {
-                index = 0;
-                lastRealPos -= (prevByte == HttpConstants.CR)? 2 : 1;
+                undecodedChunk.readerIndex(startReaderIndex);
+                undecodedChunk.writerIndex(startReaderIndex);
+                return false;
+            } else if (posDelimiter > 0) {
+                // Not fully but still some bytes to provide: httpData is not yet finished since delimiter not found
+                ByteBuf content = undecodedChunk.copy(startReaderIndex, posDelimiter);
+                try {
+                    httpData.addContent(content, false);
+                } catch (IOException e) {
+                    throw new ErrorDataDecoderException(e);
+                }
+                rewriteCurrentBuffer(undecodedChunk, posDelimiter);
+                return false;
             }
-            prevByte = nextByte;
-        }
-        if (prevByte == HttpConstants.CR) {
-            lastRealPos--;
+            // Empty chunk or so
+            return false;
         }
-        final int lastPosition = sao.getReadPosition(lastRealPos);
-        final ByteBuf content = undecodedChunk.retainedSlice(startReaderIndex, lastPosition - startReaderIndex);
+        // Delimiter found at posDelimiter, including LF or CRLF, so httpData has its last chunk
+        ByteBuf content = undecodedChunk.copy(startReaderIndex, posDelimiter);
         try {
-            httpData.addContent(content, delimiterFound);
+            httpData.addContent(content, true);
         } catch (IOException e) {
             throw new ErrorDataDecoderException(e);
         }
-        undecodedChunk.readerIndex(lastPosition);
-        return delimiterFound;
+        rewriteCurrentBuffer(undecodedChunk, posDelimiter);
+        return true;
     }
 
     /**
@@ -1529,4 +1334,15 @@ private static String[] splitMultipartHeaderValues(String svalue) {
         values.add(svalue.substring(start));
         return values.toArray(new String[0]);
     }
+
+    /**
+     * This method is package private intentionally in order to allow during tests
+     * to access to the amount of memory allocated (capacity) within the private
+     * ByteBuf undecodedChunk
+     *
+     * @return the number of bytes the internal buffer can contain
+     */
+    int getCurrentAllocatedCapacity() {
+        return undecodedChunk.capacity();
+    }
 }
diff --git a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostRequestEncoder.java b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostRequestEncoder.java
index 18d0a969b5e..8921fca24f6 100755
--- a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostRequestEncoder.java
+++ b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostRequestEncoder.java
@@ -943,12 +943,12 @@ private HttpContent encodeNextChunkUrlEncoded(int sizeleft) throws ErrorDataEnco
         // Set name=
         if (isKey) {
             String key = currentData.getName();
-            buffer = wrappedBuffer(key.getBytes());
+            buffer = wrappedBuffer(key.getBytes(charset));
             isKey = false;
             if (currentBuffer == null) {
-                currentBuffer = wrappedBuffer(buffer, wrappedBuffer("=".getBytes()));
+                currentBuffer = wrappedBuffer(buffer, wrappedBuffer("=".getBytes(charset)));
             } else {
-                currentBuffer = wrappedBuffer(currentBuffer, buffer, wrappedBuffer("=".getBytes()));
+                currentBuffer = wrappedBuffer(currentBuffer, buffer, wrappedBuffer("=".getBytes(charset)));
             }
             // continue
             size -= buffer.readableBytes() + 1;
@@ -969,7 +969,7 @@ private HttpContent encodeNextChunkUrlEncoded(int sizeleft) throws ErrorDataEnco
         ByteBuf delimiter = null;
         if (buffer.readableBytes() < size) {
             isKey = true;
-            delimiter = iterator.hasNext() ? wrappedBuffer("&".getBytes()) : null;
+            delimiter = iterator.hasNext() ? wrappedBuffer("&".getBytes(charset)) : null;
         }
 
         // End for current InterfaceHttpData, need potentially more data
diff --git a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostStandardRequestDecoder.java b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostStandardRequestDecoder.java
index 37e41fa9176..d2ef677dbf0 100644
--- a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostStandardRequestDecoder.java
+++ b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/HttpPostStandardRequestDecoder.java
@@ -303,7 +303,17 @@ public HttpPostStandardRequestDecoder offer(HttpContent content) {
         }
         parseBody();
         if (undecodedChunk != null && undecodedChunk.writerIndex() > discardThreshold) {
-            undecodedChunk.discardReadBytes();
+            if (undecodedChunk.refCnt() == 1) {
+                // It's safe to call discardBytes() as we are the only owner of the buffer.
+                undecodedChunk.discardReadBytes();
+            } else {
+                // There seems to be multiple references of the buffer. Let's copy the data and release the buffer to
+                // ensure we can give back memory to the system.
+                ByteBuf buffer = undecodedChunk.alloc().buffer(undecodedChunk.readableBytes());
+                buffer.writeBytes(undecodedChunk);
+                undecodedChunk.release();
+                undecodedChunk = buffer;
+            }
         }
         return this;
     }
diff --git a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/MixedAttribute.java b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/MixedAttribute.java
index ed1e37434a5..9b89af43fa7 100644
--- a/codec-http/src/main/java/io/netty/handler/codec/http/multipart/MixedAttribute.java
+++ b/codec-http/src/main/java/io/netty/handler/codec/http/multipart/MixedAttribute.java
@@ -270,9 +270,6 @@ public String getValue() throws IOException {
 
     @Override
     public void setValue(String value) throws IOException {
-        if (value != null) {
-            checkSize(value.getBytes().length);
-        }
         attribute.setValue(value);
     }
 
diff --git a/microbench/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoderBenchmark.java b/microbench/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoderBenchmark.java
index c570cd67455..29d1013923b 100644
--- a/microbench/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoderBenchmark.java
+++ b/microbench/src/main/java/io/netty/handler/codec/http/multipart/HttpPostMultipartRequestDecoderBenchmark.java
@@ -24,6 +24,7 @@
 import io.netty.handler.codec.http.HttpMethod;
 import io.netty.handler.codec.http.HttpVersion;
 import io.netty.microbench.util.AbstractMicrobenchmark;
+import io.netty.util.CharsetUtil;
 import io.netty.util.ResourceLeakDetector;
 import io.netty.util.ResourceLeakDetector.Level;
 import org.openjdk.jmh.annotations.Benchmark;
@@ -54,12 +55,12 @@ public double testHighNumberChunks(boolean big, boolean noDisk) {
                                  "Content-Disposition: form-data; name=\"msg_id\"\n\n15200\n--" +
                                  BOUNDARY +
                                  "\nContent-Disposition: form-data; name=\"msg1\"; filename=\"file1.txt\"\n\n" +
-                                 data).getBytes();
-        byte[] bodyPartBigBytes = data.getBytes();
+                                 data).getBytes(CharsetUtil.UTF_8);
+        byte[] bodyPartBigBytes = data.getBytes(CharsetUtil.UTF_8);
         byte[] intermediaryBytes = ("\n--" + BOUNDARY +
                                     "\nContent-Disposition: form-data; name=\"msg2\"; filename=\"file2.txt\"\n\n" +
-                                    data).getBytes();
-        byte[] finalBigBytes = ("\n" + "--" + BOUNDARY + "--\n").getBytes();
+                                    data).getBytes(CharsetUtil.UTF_8);
+        byte[] finalBigBytes = ("\n" + "--" + BOUNDARY + "--\n").getBytes(CharsetUtil.UTF_8);
         ByteBuf firstBuf = Unpooled.wrappedBuffer(bodyStartBytes);
         ByteBuf finalBuf = Unpooled.wrappedBuffer(finalBigBytes);
         ByteBuf nextBuf;


Test patch:
diff --git a/codec-http/src/test/java/io/netty/handler/codec/http/multipart/HttpPostRequestDecoderTest.java b/codec-http/src/test/java/io/netty/handler/codec/http/multipart/HttpPostRequestDecoderTest.java
index acbcf1219d3..93991c375a5 100644
--- a/codec-http/src/test/java/io/netty/handler/codec/http/multipart/HttpPostRequestDecoderTest.java
+++ b/codec-http/src/test/java/io/netty/handler/codec/http/multipart/HttpPostRequestDecoderTest.java
@@ -34,6 +34,7 @@
 import io.netty.util.CharsetUtil;
 import org.junit.Test;
 
+import java.io.IOException;
 import java.net.URLEncoder;
 import java.nio.charset.UnsupportedCharsetException;
 import java.util.Arrays;
@@ -961,6 +962,7 @@ public void testDecodeMultipartRequest() {
 
         try {
             HttpPostRequestDecoder decoder = new HttpPostRequestDecoder(new DefaultHttpDataFactory(false), req);
+            assertEquals(2, decoder.getBodyHttpDatas().size());
             InterfaceHttpData data = decoder.getBodyHttpData("title");
             assertTrue(data instanceof MemoryAttribute);
             assertEquals("bar-stream", ((MemoryAttribute) data).getString());
@@ -976,4 +978,177 @@ public void testDecodeMultipartRequest() {
             assertTrue(req.release());
         }
     }
+
+    private void commonTestBigFileDelimiterInMiddleChunk(HttpDataFactory factory, boolean inMemory)
+            throws IOException {
+        int nbChunks = 100;
+        int bytesPerChunk = 100000;
+        int bytesLastChunk = 10000;
+        int fileSize = bytesPerChunk * nbChunks + bytesLastChunk; // set Xmx to a number lower than this and it crashes
+
+        String prefix = "--861fbeab-cd20-470c-9609-d40a0f704466\n" +
+                "Content-Disposition: form-data; name=\"image\"; filename=\"guangzhou.jpeg\"\n" +
+                "Content-Type: image/jpeg\n" +
+                "Content-Length: " + fileSize + "\n" +
+                "\n";
+
+        String suffix1 = "\n" +
+                "--861fbeab-";
+        String suffix2 = "cd20-470c-9609-d40a0f704466--\n";
+        String suffix = suffix1 + suffix2;
+
+        HttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.POST, "/upload");
+        request.headers().set("content-type", "multipart/form-data; boundary=861fbeab-cd20-470c-9609-d40a0f704466");
+        request.headers().set("content-length", prefix.length() + fileSize + suffix.length());
+
+        HttpPostMultipartRequestDecoder decoder = new HttpPostMultipartRequestDecoder(factory, request);
+        decoder.offer(new DefaultHttpContent(Unpooled.wrappedBuffer(prefix.getBytes(CharsetUtil.UTF_8))));
+
+        byte[] body = new byte[bytesPerChunk];
+        Arrays.fill(body, (byte) 1);
+        for (int i = 0; i < nbChunks; i++) {
+            ByteBuf content = Unpooled.wrappedBuffer(body, 0, bytesPerChunk);
+            decoder.offer(new DefaultHttpContent(content)); // **OutOfMemory here**
+            content.release();
+        }
+
+        byte[] bsuffix1 = suffix1.getBytes(CharsetUtil.UTF_8);
+        byte[] lastbody = new byte[bytesLastChunk + bsuffix1.length];
+        Arrays.fill(body, (byte) 1);
+        for (int i = 0; i < bsuffix1.length; i++) {
+            lastbody[bytesLastChunk + i] = bsuffix1[i];
+        }
+
+        ByteBuf content2 = Unpooled.wrappedBuffer(lastbody, 0, lastbody.length);
+        decoder.offer(new DefaultHttpContent(content2));
+        content2.release();
+        content2 = Unpooled.wrappedBuffer(suffix2.getBytes(CharsetUtil.UTF_8));
+        decoder.offer(new DefaultHttpContent(content2));
+        content2.release();
+        decoder.offer(new DefaultLastHttpContent());
+
+        FileUpload data = (FileUpload) decoder.getBodyHttpDatas().get(0);
+        assertEquals(data.length(), fileSize);
+        assertEquals(inMemory, data.isInMemory());
+        if (data.isInMemory()) {
+            // To be done only if not inMemory: assertEquals(data.get().length, fileSize);
+            assertFalse("Capacity should be higher than 1M", data.getByteBuf().capacity()
+                    < 1024 * 1024);
+        }
+        assertTrue("Capacity should be less than 1M", decoder.getCurrentAllocatedCapacity()
+                < 1024 * 1024);
+        for (InterfaceHttpData httpData: decoder.getBodyHttpDatas()) {
+            httpData.release();
+            factory.removeHttpDataFromClean(request, httpData);
+        }
+        factory.cleanAllHttpData();
+        decoder.destroy();
+    }
+
+    @Test
+    public void testBIgFileUploadDelimiterInMiddleChunkDecoderDiskFactory() throws IOException {
+        // Factory using Disk mode
+        HttpDataFactory factory = new DefaultHttpDataFactory(true);
+
+        commonTestBigFileDelimiterInMiddleChunk(factory, false);
+    }
+
+    @Test
+    public void testBIgFileUploadDelimiterInMiddleChunkDecoderMemoryFactory() throws IOException {
+        // Factory using Memory mode
+        HttpDataFactory factory = new DefaultHttpDataFactory(false);
+
+        commonTestBigFileDelimiterInMiddleChunk(factory, true);
+    }
+
+    @Test
+    public void testBIgFileUploadDelimiterInMiddleChunkDecoderMixedFactory() throws IOException {
+        // Factory using Mixed mode, where file shall be on Disk
+        HttpDataFactory factory = new DefaultHttpDataFactory(10000);
+
+        commonTestBigFileDelimiterInMiddleChunk(factory, false);
+    }
+
+    private void commonNotBadReleaseBuffersDuringDecoding(HttpDataFactory factory, boolean inMemory)
+            throws IOException {
+        int nbItems = 20;
+        int bytesPerItem = 1000;
+        int maxMemory = 500;
+
+        String prefix1 = "\n--861fbeab-cd20-470c-9609-d40a0f704466\n" +
+                "Content-Disposition: form-data; name=\"image";
+        String prefix2 =
+                "\"; filename=\"guangzhou.jpeg\"\n" +
+                        "Content-Type: image/jpeg\n" +
+                        "Content-Length: " + bytesPerItem + "\n" + "\n";
+
+        String suffix = "\n--861fbeab-cd20-470c-9609-d40a0f704466--\n";
+
+        HttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.POST, "/upload");
+        request.headers().set("content-type", "multipart/form-data; boundary=861fbeab-cd20-470c-9609-d40a0f704466");
+        request.headers().set("content-length", nbItems * (prefix1.length() + prefix2.length() + 2 + bytesPerItem)
+                + suffix.length());
+        HttpPostMultipartRequestDecoder decoder = new HttpPostMultipartRequestDecoder(factory, request);
+        decoder.setDiscardThreshold(maxMemory);
+        for (int rank = 0; rank < nbItems; rank++) {
+            byte[] bp1 = prefix1.getBytes(CharsetUtil.UTF_8);
+            byte[] bp2 = prefix2.getBytes(CharsetUtil.UTF_8);
+            byte[] prefix = new byte[bp1.length + 2 + bp2.length];
+            for (int i = 0; i < bp1.length; i++) {
+                prefix[i] = bp1[i];
+            }
+            byte[] brank = Integer.toString(10 + rank).getBytes(CharsetUtil.UTF_8);
+            prefix[bp1.length] = brank[0];
+            prefix[bp1.length + 1] = brank[1];
+            for (int i = 0; i < bp2.length; i++) {
+                prefix[bp1.length + 2 + i] = bp2[i];
+            }
+            decoder.offer(new DefaultHttpContent(Unpooled.wrappedBuffer(prefix)));
+            byte[] body = new byte[bytesPerItem];
+            Arrays.fill(body, (byte) rank);
+            ByteBuf content = Unpooled.wrappedBuffer(body, 0, bytesPerItem);
+            decoder.offer(new DefaultHttpContent(content));
+            content.release();
+        }
+        byte[] lastbody = suffix.getBytes(CharsetUtil.UTF_8);
+        ByteBuf content2 = Unpooled.wrappedBuffer(lastbody, 0, lastbody.length);
+        decoder.offer(new DefaultHttpContent(content2));
+        content2.release();
+        decoder.offer(new DefaultLastHttpContent());
+
+        for (int rank = 0; rank < nbItems; rank++) {
+            FileUpload data = (FileUpload) decoder.getBodyHttpData("image" + (10 + rank));
+            assertEquals(data.length(), bytesPerItem);
+            assertEquals(inMemory, data.isInMemory());
+            byte[] body = new byte[bytesPerItem];
+            Arrays.fill(body, (byte) rank);
+            assertTrue(Arrays.equals(body, data.get()));
+        }
+        // To not be done since will load full file on memory: assertEquals(data.get().length, fileSize);
+        // Not mandatory since implicitely called during destroy of decoder
+        for (InterfaceHttpData httpData: decoder.getBodyHttpDatas()) {
+            httpData.release();
+            factory.removeHttpDataFromClean(request, httpData);
+        }
+        factory.cleanAllHttpData();
+        decoder.destroy();
+    }
+    @Test
+    public void testNotBadReleaseBuffersDuringDecodingDiskFactory() throws IOException {
+        // Using Disk Factory
+        HttpDataFactory factory = new DefaultHttpDataFactory(true);
+        commonNotBadReleaseBuffersDuringDecoding(factory, false);
+    }
+    @Test
+    public void testNotBadReleaseBuffersDuringDecodingMemoryFactory() throws IOException {
+        // Using Memory Factory
+        HttpDataFactory factory = new DefaultHttpDataFactory(false);
+        commonNotBadReleaseBuffersDuringDecoding(factory, true);
+    }
+    @Test
+    public void testNotBadReleaseBuffersDuringDecodingMixedFactory() throws IOException {
+        // Using Mixed Factory
+        HttpDataFactory factory = new DefaultHttpDataFactory(100);
+        commonNotBadReleaseBuffersDuringDecoding(factory, false);
+    }
 }