[TypeError: Couldn't cast array of type] Can only load a subset of the dataset
Opened by loubnabnl (closed)

### Describe the bug

I'm trying to load this [dataset](https://huggingface.co/datasets/bigcode-data/the-stack-gh-issues) which consists of jsonl files and I get the following error:
```
casted_values = _c(array.values, feature[0])
  File "/opt/conda/lib/python3.7/site-packages/datasets/table.py", line 1839, in wrapper
    return func(array, *args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/datasets/table.py", line 2132, in cast_array_to_feature
    raise TypeError(f"Couldn't cast array of type\n{array.type}\nto\n{feature}")
TypeError: Couldn't cast array of type
struct<type: string, action: string, datetime: timestamp[s], author: string, title: string, description: string, comment_id: int64, comment: string, labels: list<item: string>>
to
{'type': Value(dtype='string', id=None), 'action': Value(dtype='string', id=None), 'datetime': Value(dtype='timestamp[s]', id=None), 'author': Value(dtype='string', id=None), 'title': Value(dtype='string', id=None), 'description': Value(dtype='string', id=None), 'comment_id': Value(dtype='int64', id=None), 'comment': Value(dtype='string', id=None)}
```

But I can succesfully load a subset of the dataset, for example this works:
```python
ds = load_dataset('bigcode-data/the-stack-gh-issues', split="train", data_files=[f"data/data-{x}.jsonl" for x in range(10)])
```
and `ds.features` returns:
```
{'repo': Value(dtype='string', id=None),
 'org': Value(dtype='string', id=None),
 'issue_id': Value(dtype='int64', id=None),
 'issue_number': Value(dtype='int64', id=None),
 'pull_request': {'user_login': Value(dtype='string', id=None),
  'repo': Value(dtype='string', id=None),
  'number': Value(dtype='int64', id=None)},
 'events': [{'type': Value(dtype='string', id=None),
   'action': Value(dtype='string', id=None),
   'datetime': Value(dtype='timestamp[s]', id=None),
   'author': Value(dtype='string', id=None),
   'title': Value(dtype='string', id=None),
   'description': Value(dtype='string', id=None),
   'comment_id': Value(dtype='int64', id=None),
   'comment': Value(dtype='string', id=None)}]}
  ```
  So I'm not sure if there's an issue with just some of the files. Grateful if you have any suggestions to fix the issue.
  
  Side note: 
  I saw this related [issue](https://github.com/huggingface/datasets/issues/3637) and tried to write a loading script to have `events` as a `Sequence` and not `list` [here](https://huggingface.co/datasets/bigcode-data/the-stack-gh-issues/blob/main/loading.py) (the script was renamed). It worked with a subset locally but doesn't for the remote dataset  it can't find https://huggingface.co/datasets/bigcode-data/the-stack-gh-issues/resolve/main/data.

### Steps to reproduce the bug

```python
from datasets import load_dataset

ds = load_dataset('bigcode-data/the-stack-gh-issues', split="train")
```

### Expected behavior

Load the entire dataset succesfully.

### Environment info

- `datasets` version: 2.10.1
- Platform: Linux-4.19.0-23-cloud-amd64-x86_64-with-debian-10.13
- Python version: 3.7.12
- PyArrow version: 9.0.0
- Pandas version: 1.3.4

Comments:
Apparently some JSON objects have a `"labels"` field. Since this field is not present in every object, you must specify all the fields types in the README.md

EDIT: actually specifying the feature types doesn’t solve the issue, it raises an error because “labels” is missing in the data

We've updated the dataset to remove the extra `labels` field from some files, closing this issue. Thanks!

A similar error occurs in the Pile dataset (EleutherAI/the_pile)

Loading the dataset produces the following error.

```
TypeError: Couldn't cast array of type
struct<file: string, id: string>
to
{'id': Value(dtype='string', id=None)}
```


I think this was fixed in https://huggingface.co/datasets/EleutherAI/the_pile/discussions/11

i have the same problem ，how to solve :
    raise TypeError(f"Couldn't cast array of type\n{array.type}\nto\n{feature}")
TypeError: Couldn't cast array of type
list<item: string>
to
{'content': Value(dtype='string', id=None), 'role': Value(dtype='string', id=None)}