GitHub issue #6073: version2.3.2   load_dataset()data_files can't include .xxxx in path
State: closed | Created: 2023-07-26T11:09:31Z

### Describe the bug

First, I cd workdir.
Then, I just use load_dataset("json", data_file={"train":"/a/b/c/.d/train/train.json", "test":"/a/b/c/.d/train/test.json"})
that couldn't work and 
<FileNotFoundError: Unable to find 
'/a/b/c/.d/train/train.jsonl' at 
/a/b/c/.d/>

And I debug, it is fine in version2.1.2

So there maybe a bug in path join.

Here is the whole bug report:
/x/datasets/loa │
│ d.py:1656 in load_dataset                                                    │
│                                                                              │
│   1653 │   ignore_verifications = ignore_verifications or save_infos         │
│   1654 │                                                                     │
│   1655 │   # Create a dataset builder                                        │
│ ❱ 1656 │   builder_instance = load_dataset_builder(                          │
│   1657 │   │   path=path,                                                    │
│   1658 │   │   name=name,                                                    │
│   1659 │   │   data_dir=data_dir,                                            │
│                                                                              │
│ x/datasets/loa │
│ d.py:1439 in load_dataset_builder                                            │
│                                                                              │
│   1436 │   if use_auth_token is not None:                                    │
│   1437 │   │   download_config = download_config.copy() if download_config e │
│   1438 │   │   download_config.use_auth_token = use_auth_token               │
│ ❱ 1439 │   dataset_module = dataset_module_factory(                          │
│   1440 │   │   path,                                                         │
│   1441 │   │   revision=revision,                                            │
│   1442 │   │   download_config=download_config,                              │
│                                                                              │
│ x/datasets/loa │
│ d.py:1097 in dataset_module_factory                                          │
│                                                                              │
│   1094 │                                                                     │
│   1095 │   # Try packaged                                                    │
│   1096 │   if path in _PACKAGED_DATASETS_MODULES:                            │
│ ❱ 1097 │   │   return PackagedDatasetModuleFactory(                          │
│   1098 │   │   │   path,                                                     │
│   1099 │   │   │   data_dir=data_dir,                                        │
│   1100 │   │   │   data_files=data_files,                                    │
│                                                                              │
│x/datasets/loa │
│ d.py:743 in get_module                                                       │
│                                                                              │
│    740 │   │   │   if self.data_dir is not None                              │
│    741 │   │   │   else get_patterns_locally(str(Path().resolve()))          │
│    742 │   │   )                                                             │
│ ❱  743 │   │   data_files = DataFilesDict.from_local_or_remote(              │
│    744 │   │   │   patterns,                                                 │
│    745 │   │   │   use_auth_token=self.download_config.use_auth_token,       │
│    746 │   │   │   base_path=str(Path(self.data_dir).resolve()) if self.data │
│                                                                              │
│ x/datasets/dat │
│ a_files.py:590 in from_local_or_remote                                       │
│                                                                              │
│   587 │   │   out = cls()                                                    │
│   588 │   │   for key, patterns_for_key in patterns.items():                 │
│   589 │   │   │   out[key] = (                                               │
│ ❱ 590 │   │   │   │   DataFilesList.from_local_or_remote(                    │
│   591 │   │   │   │   │   patterns_for_key,                                  │
│   592 │   │   │   │   │   base_path=base_path,                               │
│   593 │   │   │   │   │   allowed_extensions=allowed_extensions,             │
│                                                                              │
│ /x/datasets/dat │
│ a_files.py:558 in from_local_or_remote                                       │
│                                                                              │
│   555 │   │   use_auth_token: Optional[Union[bool, str]] = None,             │
│   556 │   ) -> "DataFilesList":                                              │
│   557 │   │   base_path = base_path if base_path is not None else str(Path() │
│ ❱ 558 │   │   data_files = resolve_patterns_locally_or_by_urls(base_path, pa │
│   559 │   │   origin_metadata = _get_origin_metadata_locally_or_by_urls(data │
│   560 │   │   return cls(data_files, origin_metadata)                        │
│   561                                                                        │
│                                                                              │
│ /x/datasets/dat │
│ a_files.py:195 in resolve_patterns_locally_or_by_urls                        │
│                                                                              │
│   192 │   │   if is_remote_url(pattern):                                     │
│   193 │   │   │   data_files.append(Url(pattern))                            │
│   194 │   │   else:                                                          │
│ ❱ 195 │   │   │   for path in _resolve_single_pattern_locally(base_path, pat │
│   196 │   │   │   │   data_files.append(path)                                │
│   197 │                                                                      │
│   198 │   if not data_files:                                                 │
│                                                                              │
│ /x/datasets/dat │
│ a_files.py:145 in _resolve_single_pattern_locally                            │
│                                                                              │
│   142 │   │   error_msg = f"Unable to find '{pattern}' at {Path(base_path).r │
│   143 │   │   if allowed_extensions is not None:                             │
│   144 │   │   │   error_msg += f" with any supported extension {list(allowed │
│ ❱ 145 │   │   raise FileNotFoundError(error_msg)                             │
│   146 │   return sorted(out)                                                 │
│   147                         

### Steps to reproduce the bug

1. Version=2.3.2
2. In shell, cd workdir.(cd /a/b/c/.d/)
3. load_dataset("json", data_file={"train":"/a/b/c/.d/train/train.json", "test":"/a/b/c/.d/train/test.json"})

### Expected behavior

fix it please~

### Environment info

2.3.2