Split is inferred from filename and overrides metadata.jsonl
Opened by float-trip (closed)

## Describe the bug

Including the strings "test" or "train" anywhere in a filename causes `datasets` to infer the split and silently ignore all other files.

This behavior is documented for directory names but not filenames: https://huggingface.co/docs/datasets/image_dataset#imagefolder

## Steps to reproduce the bug
`metadata.jsonl`
```json
{"file_name": "photo of a cat.jpg", "text": "a photo of a cat"}
{"file_name": "photo of a dog.jpg", "text": "a photo of a dog"}
{"file_name": "photo of a train.jpg", "text": "a photo of a train"}
{"file_name": "photo of test tubes.jpg", "text": "a photo of test tubes"}
```

`bug.py`
```python
from datasets import load_dataset

dataset = load_dataset("dataset")

print(dataset)
# DatasetDict({
#     train: Dataset({
#         features: ['image', 'text'],
#         num_rows: 1
#     })
#     test: Dataset({
#         features: ['image', 'text'],
#         num_rows: 1
#     })
# })

for split in dataset:
    for n in dataset[split]:
        print(n['text'])
        # a photo of a train
        # a photo of test tubes
```

## Expected results
One single dataset with all four images / a warning for unused files / documentation of this behavior

## Actual results
Only the images with "test" or "train" in the name are loaded

## Environment info
- `datasets` version: 2.5.1
- Platform: macOS-12.5.1-x86_64-i386-64bit
- Python version: 3.10.4
- PyArrow version: 9.0.0
- Pandas version: 1.5.0

Comments:
Hi! What's the structure of your image folder? `datasets` by default tries to infer to what split each file belongs based on directory/file names. If it's OK to load all the images inside the `dataset` folder in the `train` split, you can do the following:
```python
dataset = load_dataset("imagefolder", data_files="dataset/**")
```

Thanks! Specifying `data_files` worked for that case.

I'm new to the library, so let me try rephrasing the issue. If there's no actual bug here, sorry for the trouble.

I've uploaded an example [here](https://files.catbox.moe/nfj2pd.zip) with the following files: 

```
.
├── bug.py
└── imagefolder
   ├── test
   │  ├── metadata.jsonl
   │  ├── dog.jpg
   │  └── personal trainer.jpg
   └── train
      ├── metadata.jsonl
      ├── cat.jpg
      └── testing center.jpg
```

`bug.py`
```
from datasets import load_dataset

dataset = load_dataset("imagefolder")

print(dataset)
# DatasetDict({
#     test: Dataset({
#         features: ['image', 'text'],
#         num_rows: 1
#     })
# })

for split in dataset:
    print("Split:", split)
    for n in dataset[split]:
        print(n['text'])


# Split: test
# testing center
```

As far as I can tell, this conforms with the example given here: https://huggingface.co/docs/datasets/image_dataset#imagefolder. It appears to me that, even though `metadata.jsonl` is present, the inferred labels from the path are taking precedent. Does this sound like a bug/undocumented behavior?

This looks like a duplicate of https://github.com/huggingface/datasets/issues/4895 (the problem is explained in this comment: https://github.com/huggingface/datasets/issues/4895#issuecomment-1248269550).

In the meantime, you can do the following to fetch all the splits:
```python
dataset = load_dataset("imagefolder", data_files={"train": "imagefolder/train/**", "test": "imagefolder/test/**"})
```