EmptyDatasetError: The directory at /metadata.jsonl doesn't contain any data files
Opened by Aki1991 (closed)

### Describe the bug

while trying to load custom dataset from jsonl file, I get the error: "metadata.jsonl doesn't contain any data files"

### Steps to reproduce the bug

This is my [metadata_v2.jsonl](https://github.com/user-attachments/files/16016011/metadata_v2.json) file.  I have this file in the folder with all images mentioned in that json(l) file.

Through below mentioned command I am trying to load_dataset so that I can upload it as mentioned here on the [official website](https://huggingface.co/docs/datasets/en/image_dataset#upload-dataset-to-the-hub).

````
from datasets import load_dataset

dataset = load_dataset("imagefolder", data_dir="path/to/jsonl/metadata.jsonl")
````

error:

````
EmptyDatasetError                         Traceback (most recent call last)
Cell In[18], line 3
      1 from datasets import load_dataset
----> 3 dataset = load_dataset("imagefolder", 
      4                        data_dir="path/to/jsonl/file/metadata.jsonl")
      5 dataset[0]["objects"]

File ~/anaconda3/envs/lvis/lib/python3.11/site-packages/datasets/load.py:2594, in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, verification_mode, ignore_verifications, keep_in_memory, save_infos, revision, token, use_auth_token, task, streaming, num_proc, storage_options, trust_remote_code, **config_kwargs)
   2589 verification_mode = VerificationMode(
   2590     (verification_mode or VerificationMode.BASIC_CHECKS) if not save_infos else VerificationMode.ALL_CHECKS
   2591 )
   2593 # Create a dataset builder
-> 2594 builder_instance = load_dataset_builder(
   2595     path=path,
   2596     name=name,
   2597     data_dir=data_dir,
   2598     data_files=data_files,
   2599     cache_dir=cache_dir,
   2600     features=features,
   2601     download_config=download_config,
   2602     download_mode=download_mode,
   2603     revision=revision,
   2604     token=token,
   2605     storage_options=storage_options,
   2606     trust_remote_code=trust_remote_code,
   2607     _require_default_config_name=name is None,
   2608     **config_kwargs,
   2609 )
   2611 # Return iterable dataset in case of streaming
   2612 if streaming:

File ~/anaconda3/envs/lvis/lib/python3.11/site-packages/datasets/load.py:2266, in load_dataset_builder(path, name, data_dir, data_files, cache_dir, features, download_config, download_mode, revision, token, use_auth_token, storage_options, trust_remote_code, _require_default_config_name, **config_kwargs)
   2264     download_config = download_config.copy() if download_config else DownloadConfig()
   2265     download_config.storage_options.update(storage_options)
-> 2266 dataset_module = dataset_module_factory(
   2267     path,
   2268     revision=revision,
   2269     download_config=download_config,
   2270     download_mode=download_mode,
   2271     data_dir=data_dir,
   2272     data_files=data_files,
   2273     cache_dir=cache_dir,
   2274     trust_remote_code=trust_remote_code,
   2275     _require_default_config_name=_require_default_config_name,
   2276     _require_custom_configs=bool(config_kwargs),
   2277 )
   2278 # Get dataset builder class from the processing script
   2279 builder_kwargs = dataset_module.builder_kwargs

File ~/anaconda3/envs/lvis/lib/python3.11/site-packages/datasets/load.py:1805, in dataset_module_factory(path, revision, download_config, download_mode, dynamic_modules_path, data_dir, data_files, cache_dir, trust_remote_code, _require_default_config_name, _require_custom_configs, **download_kwargs)
   1782 # We have several ways to get a dataset builder:
   1783 #
   1784 # - if path is the name of a packaged dataset module
   (...)
   1796 
   1797 # Try packaged
   1798 if path in _PACKAGED_DATASETS_MODULES:
   1799     return PackagedDatasetModuleFactory(
   1800         path,
   1801         data_dir=data_dir,
   1802         data_files=data_files,
   1803         download_config=download_config,
   1804         download_mode=download_mode,
-> 1805     ).get_module()
   1806 # Try locally
   1807 elif path.endswith(filename):

File ~/anaconda3/envs/lvis/lib/python3.11/site-packages/datasets/load.py:1140, in PackagedDatasetModuleFactory.get_module(self)
   1135 def get_module(self) -> DatasetModule:
   1136     base_path = Path(self.data_dir or "").expanduser().resolve().as_posix()
   1137     patterns = (
   1138         sanitize_patterns(self.data_files)
   1139         if self.data_files is not None
-> 1140         else get_data_patterns(base_path, download_config=self.download_config)
   1141     )
   1142     data_files = DataFilesDict.from_patterns(
   1143         patterns,
   1144         download_config=self.download_config,
   1145         base_path=base_path,
   1146     )
   1147     supports_metadata = self.name in _MODULE_SUPPORTS_METADATA

File ~/anaconda3/envs/lvis/lib/python3.11/site-packages/datasets/data_files.py:503, in get_data_patterns(base_path, download_config)
    501     return _get_data_files_patterns(resolver)
    502 except FileNotFoundError:
--> 503     raise EmptyDatasetError(f"The directory at {base_path} doesn't contain any data files") from None

EmptyDatasetError: The directory at path/to/jsonl/file/metadata.jsonl doesn't contain any data files`
```

### Expected behavior

It should be able load the whole file in a format of "dataset" inside the dataset variable. But it gives error "The directory at "path/to/jsonl/metadata.jsonl" doesn't contain any data files."

### Environment info

I am using conda environment.

Comments:
Hi ! `data_dir=` is for directories, can you try using `data_files=` instead ?

If you are trying to load your image dataset from a local folder, you should replace "data_dir=path/to/jsonl/metadata.jsonl" with the real folder path in your computer.

https://huggingface.co/docs/datasets/en/image_load#imagefolder

Ah yes. My bad. I was giving file name. I should have given the folder directory as the path. That solved my issue. Thank you @albertvillanova and @lhoestq.