opened
nelusnegur
username_0
When I use an HTTP client configured with the GZip client middleware to perform a HEAD request, and provided the HTTP server can return a compressed response, the GZip middleware crashes during the processing of the response with the following error:

```
fs2.compress$NonProgressiveDecompressionException: buffer size 32768 is too small; gunzip cannot make progress
  fs2.compress$.$anonfun$gunzip$3(compress.scala:239)
  fs2.compress$.$anonfun$gunzip$3$adapted(compress.scala:221)
  fs2.Pull$.$anonfun$flatMap$1(Pull.scala:51)
  fs2.internal.FreeC$$anon$1.cont(FreeC.scala:29)
  fs2.internal.FreeC$ViewL$$anon$9$$anon$10.cont(FreeC.scala:196)
  fs2.internal.FreeC$ViewL$.mk(FreeC.scala:185)
  fs2.internal.FreeC$ViewL$.apply(FreeC.scala:176)
  fs2.internal.FreeC.viewL(FreeC.scala:79)
  fs2.internal.Algebra$.go$1(Algebra.scala:178)
  fs2.internal.Algebra$.$anonfun$compileLoop$7(Algebra.scala:219)
  fs2.internal.Algebra$.$anonfun$compileLoop$1(Algebra.scala:197)
  monix.eval.internal.TaskRunLoop$.startFull(TaskRunLoop.scala:170)
  monix.eval.internal.TaskRestartCallback.syncOnSuccess(TaskRestartCallback.scala:101)
  monix.eval.internal.TaskRestartCallback$$anon$1.run(TaskRestartCallback.scala:118)
  ...
```
Here is small Scala ammonite script, which defines an HTTP client configured with GZip middleware, executes a HEAD request and crashes with the error mentioned above:

```scala
import $ivy.`org.http4s::http4s-blaze-client:0.21.4`
import $ivy.`io.monix::monix:3.2.1`
import $ivy.`co.fs2::fs2-core:2.3.0`

import monix.eval.Task
import monix.eval.Task._
import monix.execution.Scheduler
import monix.execution.Scheduler.Implicits.global
import org.http4s._
import org.http4s.client.blaze._
import org.http4s.client._
import org.http4s.client.dsl.io._
import org.http4s.client.blaze.BlazeClientBuilder
import org.http4s.client.middleware.GZip

import scala.concurrent.Await
import scala.concurrent.duration._

val (httpClient, shutdown) =
  BlazeClientBuilder(global)
    .withMaxTotalConnections(3)
    .withMaxWaitQueueLimit(5)
    .allocated
    .map { case (client, shutdown) => GZip()(client) -> shutdown }
    .runSyncUnsafe(10.seconds)

val headRequest = Request[Task](Method.HEAD).withUri(Uri.uri("https://doc.akka.io"))
val task = httpClient.expect[String](headRequest)
val future = task.runToFuture

Await.result(future, 3.seconds)
```

The error message is a bit misleading, because an HTTP HEAD response doesn't contain a response body regardless what is contained in the headers response. I tried to put the whole implementation of the [GZip client middleware](https://github.com/http4s/http4s/blob/master/client/src/main/scala/org/http4s/client/middleware/GZip.scala) in the above script, and I get the following exception:
```
java.io.EOFException
  fs2.compression$.$anonfun$gunzip$6(compression.scala:379)
  fs2.compression$.$anonfun$gunzip$6$adapted(compression.scala:368)
  fs2.Pull$.$anonfun$flatMap$1(Pull.scala:51)
  fs2.internal.FreeC$$anon$1.cont(FreeC.scala:29)
  fs2.internal.FreeC$ViewL$$anon$9$$anon$10.<init>(FreeC.scala:194)
  ...
```
According to this stack trace, when GZip middleware tries to decompress the response body of a HEAD request, it fails because the response body stream is empty; there are no gzip header bytes which the `fs2.compression.gunzip` method expects. 

The [RFC 7230, section 3.3.3](https://tools.ietf.org/html/rfc7230#section-3.3.3) says that the response body of a HEAD request is empty regardless of the headers present in the response (in our case, `Content-Encoding`), because they show what headers will contain the GET request on the same URI. So, the GZip client middleware should not decompress a HEAD response body even when the `Content-Encoding: gzip` header is present. I will open a PR to fix this.
GZip client middleware crashes when processing an HTTP HEAD response
issue
created
rossabaker
username_1
Leaving this one open to track the [related empty body issue](https://github.com/http4s/http4s/pull/3476#discussion_r438229941), even though #3476 covers the common cause.  The HEAD will be fixed in 0.21.5.

Assigning to @username_0, since he expressed interest in the follow-up work as well.
comment
created
rossabaker
username_1
Note that this is still not properly fixed in 0.21.5.  We'll try again.  See #3476.
comment
closed
rossabaker
username_1
issue
reopened
rossabaker
username_1
When I use an HTTP client configured with the GZip client middleware to perform a HEAD request, and provided the HTTP server can return a compressed response, the GZip middleware crashes during the processing of the response with the following error:

```
fs2.compress$NonProgressiveDecompressionException: buffer size 32768 is too small; gunzip cannot make progress
  fs2.compress$.$anonfun$gunzip$3(compress.scala:239)
  fs2.compress$.$anonfun$gunzip$3$adapted(compress.scala:221)
  fs2.Pull$.$anonfun$flatMap$1(Pull.scala:51)
  fs2.internal.FreeC$$anon$1.cont(FreeC.scala:29)
  fs2.internal.FreeC$ViewL$$anon$9$$anon$10.cont(FreeC.scala:196)
  fs2.internal.FreeC$ViewL$.mk(FreeC.scala:185)
  fs2.internal.FreeC$ViewL$.apply(FreeC.scala:176)
  fs2.internal.FreeC.viewL(FreeC.scala:79)
  fs2.internal.Algebra$.go$1(Algebra.scala:178)
  fs2.internal.Algebra$.$anonfun$compileLoop$7(Algebra.scala:219)
  fs2.internal.Algebra$.$anonfun$compileLoop$1(Algebra.scala:197)
  monix.eval.internal.TaskRunLoop$.startFull(TaskRunLoop.scala:170)
  monix.eval.internal.TaskRestartCallback.syncOnSuccess(TaskRestartCallback.scala:101)
  monix.eval.internal.TaskRestartCallback$$anon$1.run(TaskRestartCallback.scala:118)
  ...
```
Here is small Scala ammonite script, which defines an HTTP client configured with GZip middleware, executes a HEAD request and crashes with the error mentioned above:

```scala
import $ivy.`org.http4s::http4s-blaze-client:0.21.4`
import $ivy.`io.monix::monix:3.2.1`
import $ivy.`co.fs2::fs2-core:2.3.0`

import monix.eval.Task
import monix.eval.Task._
import monix.execution.Scheduler
import monix.execution.Scheduler.Implicits.global
import org.http4s._
import org.http4s.client.blaze._
import org.http4s.client._
import org.http4s.client.dsl.io._
import org.http4s.client.blaze.BlazeClientBuilder
import org.http4s.client.middleware.GZip

import scala.concurrent.Await
import scala.concurrent.duration._

val (httpClient, shutdown) =
  BlazeClientBuilder(global)
    .withMaxTotalConnections(3)
    .withMaxWaitQueueLimit(5)
    .allocated
    .map { case (client, shutdown) => GZip()(client) -> shutdown }
    .runSyncUnsafe(10.seconds)

val headRequest = Request[Task](Method.HEAD).withUri(Uri.uri("https://doc.akka.io"))
val task = httpClient.expect[String](headRequest)
val future = task.runToFuture

Await.result(future, 3.seconds)
```

The error message is a bit misleading, because an HTTP HEAD response doesn't contain a response body regardless what is contained in the headers response. I tried to put the whole implementation of the [GZip client middleware](https://github.com/http4s/http4s/blob/master/client/src/main/scala/org/http4s/client/middleware/GZip.scala) in the above script, and I get the following exception:
```
java.io.EOFException
  fs2.compression$.$anonfun$gunzip$6(compression.scala:379)
  fs2.compression$.$anonfun$gunzip$6$adapted(compression.scala:368)
  fs2.Pull$.$anonfun$flatMap$1(Pull.scala:51)
  fs2.internal.FreeC$$anon$1.cont(FreeC.scala:29)
  fs2.internal.FreeC$ViewL$$anon$9$$anon$10.<init>(FreeC.scala:194)
  ...
```
According to this stack trace, when GZip middleware tries to decompress the response body of a HEAD request, it fails because the response body stream is empty; there are no gzip header bytes which the `fs2.compression.gunzip` method expects. 

The [RFC 7230, section 3.3.3](https://tools.ietf.org/html/rfc7230#section-3.3.3) says that the response body of a HEAD request is empty regardless of the headers present in the response (in our case, `Content-Encoding`), because they show what headers will contain the GET request on the same URI. So, the GZip client middleware should not decompress a HEAD response body even when the `Content-Encoding: gzip` header is present. I will open a PR to fix this.
GZip client middleware crashes when processing an HTTP HEAD response
issue
closed
rossabaker
username_1
issue
created
rossabaker
username_1
Oh, this looks like it was fixed. :tada:
comment