`Can not decode content-encoding: gzip` when loading `scitldr` dataset with streaming
State: closed | Author: SBrandeis (CONTRIBUTOR)
Labels: name: bug

name: streaming

## Describe the bug

Trying to load the `"FullText"` config of the `"scitldr"` dataset with `streaming=True` raises an error from `aiohttp`:
```python
ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

cc @lhoestq 

## Steps to reproduce the bug
```python
from datasets import load_dataset

iter_dset = iter(
    load_dataset("scitldr", name="FullText", split="test", streaming=True)
)

next(iter_dset)
```

## Expected results
Returns the first sample of the dataset

## Actual results
Calling `__next__` crashes with the following Traceback:

```python
----> 1 next(dset_iter)

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
    339
    340     def __iter__(self):
--> 341         for key, example in self._iter():
    342             if self.features:
    343                 # we encode the example for ClassLabel feature types for example

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in _iter(self)
    336         else:
    337             ex_iterable = self._ex_iterable
--> 338         yield from ex_iterable
    339
    340     def __iter__(self):

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
     76
     77     def __iter__(self):
---> 78         for key, example in self.generate_examples_fn(**self.kwargs):
     79             yield key, example
     80

~\.cache\huggingface\modules\datasets_modules\datasets\scitldr\72d6e2195786c57e1d343066fb2cc4f93ea39c5e381e53e6ae7c44bbfd1f05ef\scitldr.py in _generate_examples(self, filepath, split)
    162
    163         with open(filepath, encoding="utf-8") as f:
--> 164             for id_, row in enumerate(f):
    165                 data = json.loads(row)
    166                 if self.config.name == "AIC":

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in read(self, length)
    496         else:
    497             length = min(self.size - self.loc, length)
--> 498         return super().read(length)
    499
    500     async def async_fetch_all(self):

~\miniconda3\envs\datasets\lib\site-packages\fsspec\spec.py in read(self, length)
   1481             # don't even bother calling fetch
   1482             return b""
-> 1483         out = self.cache._fetch(self.loc, self.loc + length)
   1484         self.loc += len(out)
   1485         return out

~\miniconda3\envs\datasets\lib\site-packages\fsspec\caching.py in _fetch(self, start, end)
    378         elif start < self.start:
    379             if self.end - end > self.blocksize:
--> 380                 self.cache = self.fetcher(start, bend)
    381                 self.start = start
    382             else:

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in wrapper(*args, **kwargs)
     86     def wrapper(*args, **kwargs):
     87         self = obj or args[0]
---> 88         return sync(self.loop, func, *args, **kwargs)
     89
     90     return wrapper

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in sync(loop, func, timeout, *args, **kwargs)
     67         raise FSTimeoutError
     68     if isinstance(result[0], BaseException):
---> 69         raise result[0]
     70     return result[0]
     71

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in _runner(event, coro, result, timeout)
     23         coro = asyncio.wait_for(coro, timeout=timeout)
     24     try:
---> 25         result[0] = await coro
     26     except Exception as ex:
     27         result[0] = ex

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in async_fetch_range(self, start, end)
    538             if r.status == 206:
    539                 # partial content, as expected
--> 540                 out = await r.read()
    541             elif "Content-Length" in r.headers:
    542                 cl = int(r.headers["Content-Length"])

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\client_reqrep.py in read(self)
   1030         if self._body is None:
   1031             try:
-> 1032                 self._body = await self.content.read()
   1033                 for trace in self._traces:
   1034                     await trace.send_response_chunk_received(

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\streams.py in read(self, n)
    342     async def read(self, n: int = -1) -> bytes:
    343         if self._exception is not None:
--> 344             raise self._exception
    345
    346         # migration problem; with DataQueue you have to catch

ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

## Environment info

- `datasets` version: 1.12.0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyArrow version: 2.0.0
- aiohttp version: 3.7.4.post0


Comments: Hi @SBrandeis, thanks for reporting! ^^

I think this is an issue with `fsspec`: https://github.com/intake/filesystem_spec/issues/389

I will ask them if they are planning to fix it...

Code to reproduce the bug: `ClientPayloadError: 400, message='Can not decode content-encoding: gzip'`
```python
In [1]: import fsspec

In [2]: import json

In [3]: with fsspec.open('https://raw.githubusercontent.com/allenai/scitldr/master/SciTLDR-Data/SciTLDR-FullText/test.jsonl', encoding="utf-8") as f:
   ...:     for row in f:
   ...:         data = json.loads(row)
   ...:
---------------------------------------------------------------------------
ClientPayloadError                        Traceback (most recent call last)
```

Thanks for investigating @albertvillanova ! 🤗