Unable to view the dataset and loading the same dataset throws the error - ArrowInvalid: Exceeded maximum rows
Opened by Pavithree (closed)

## Dataset viewer issue - -ArrowInvalid: Exceeded maximum rows

**Link:** *https://huggingface.co/datasets/Pavithree/explainLikeImFive*

*This is the subset of original eli5 dataset  https://huggingface.co/datasets/vblagoje/lfqa. I just filtered the data samples which belongs to one particular subreddit thread. However, the dataset preview for train split  returns  the below mentioned error:
Status code:   400
Exception:     ArrowInvalid
Message:       Exceeded maximum rows
When I try to load the same dataset it returns ArrowInvalid: Exceeded maximum rows error*

Am I the one who added this dataset ? Yes 


Comments:
Thanks for reporting. I'm looking at it

  It's not related to the dataset viewer in itself. I can replicate the error with:

```
>>> import datasets as ds
>>> d = ds.load_dataset('Pavithree/explainLikeImFive')
Using custom data configuration Pavithree--explainLikeImFive-b68b6d8112cd8a51
Downloading and preparing dataset json/Pavithree--explainLikeImFive to /home/slesage/.cache/huggingface/datasets/json/Pavithree--explainLikeImFive-b68b6d8112cd8a51/0.0.0/ac0ca5f5289a6cf108e706efcf040422dbbfa8e658dee6a819f20d76bb84d26b...
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 305M/305M [00:03<00:00, 98.6MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 17.9M/17.9M [00:00<00:00, 75.7MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11.9M/11.9M [00:00<00:00, 70.6MB/s]
Downloading data files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:05<00:00,  1.92s/it]
Extracting data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 1948.42it/s]
Failed to read file '/home/slesage/.cache/huggingface/datasets/downloads/5fee9c8819754df277aee6f252e4db6897d785231c21938407b8862ca871d246' with error <class 'pyarrow.lib.ArrowInvalid'>: Exceeded maximum rows
Traceback (most recent call last):
  File "/home/slesage/hf/datasets/src/datasets/packaged_modules/json/json.py", line 144, in _generate_tables
    dataset = json.load(f)
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 916 (char 915)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets/src/datasets/load.py", line 1691, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets/src/datasets/builder.py", line 605, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets/src/datasets/builder.py", line 694, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets/src/datasets/builder.py", line 1151, in _prepare_split
    for key, table in logging.tqdm(
  File "/home/slesage/.pyenv/versions/datasets/lib/python3.8/site-packages/tqdm/std.py", line 1168, in __iter__
    for obj in iterable:
  File "/home/slesage/hf/datasets/src/datasets/packaged_modules/json/json.py", line 146, in _generate_tables
    raise e
  File "/home/slesage/hf/datasets/src/datasets/packaged_modules/json/json.py", line 122, in _generate_tables
    pa_table = paj.read_json(
  File "pyarrow/_json.pyx", line 246, in pyarrow._json.read_json
  File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Exceeded maximum rows
```

cc @lhoestq @albertvillanova @mariosasko 

It seems that train.json is not a valid JSON Lines file: it has several JSON objects in the first line (the 915th character in the first line starts a new object, and there's no "\n")

You need to have one JSON object per line

I'm closing this issue.

@Pavithree, please, feel free to re-open it if fixing the JSON file does not solve it.

Thank you! that fixes the issue.