Repository: open-compass/opencompass
Issue: [Bug] The error occurred during evaluating hellaswag dataset
Root cause: Configuration Error

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

The data file of hellaswag dataset is **"/data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/data/hellaswag/hellaswag.jsonl"**

When start the evaluation script, an error occurred.
The detail of the error are as follows:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/run.py: │
│ 517 in <module>                                                                                  │
│                                                                                                  │
│   514                                                                                            │
│   515                                                                                            │
│   516 if __name__ == '__main__':                                                                 │
│ ❱ 517 │   main()                                                                                 │
│   518                                                                                            │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/run.py: │
│ 388 in main                                                                                      │
│                                                                                                  │
│   385 │   │   │   │   osp.join(cfg['work_dir'], 'predictions/'),                                 │
│   386 │   │   │   │   max_task_size=args.max_partition_size,                                     │
│   387 │   │   │   │   gen_task_coef=args.gen_task_coef)                                          │
│ ❱ 388 │   │   │   tasks = partitioner(cfg)                                                       │
│   389 │   │   │   if args.dry_run:                                                               │
│   390 │   │   │   │   return                                                                     │
│   391 │   │   │   # execute the infer subtasks                                                   │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/opencom │
│ pass/partitioners/base.py:48 in __call__                                                         │
│                                                                                                  │
│   45 │   │   datasets = cfg['datasets']                                                          │
│   46 │   │   work_dir = cfg['work_dir']                                                          │
│   47 │   │                                                                                       │
│ ❱ 48 │   │   tasks = self.partition(models, datasets, work_dir, self.out_dir)                    │
│   49 │   │                                                                                       │
│   50 │   │   self.logger.info(f'Partitioned into {len(tasks)} tasks.')                           │
│   51 │   │   for i, task in enumerate(tasks):                                                    │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/opencom │
│ pass/partitioners/size.py:67 in partition                                                        │
│                                                                                                  │
│    64 │   │   │   List[ConfigDict]: A list of tasks.                                             │
│    65 │   │   """                                                                                │
│    66 │   │                                                                                      │
│ ❱  67 │   │   datasets = sorted(datasets,                                                        │
│    68 │   │   │   │   │   │     key=lambda x: self.get_cost(x),                                  │
│    69 │   │   │   │   │   │     reverse=True)                                                    │
│    70 │   │   tasks = []                                                                         │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/opencom │
│ pass/partitioners/size.py:68 in <lambda>                                                         │
│                                                                                                  │
│    65 │   │   """                                                                                │
│    66 │   │                                                                                      │
│    67 │   │   datasets = sorted(datasets,                                                        │
│ ❱  68 │   │   │   │   │   │     key=lambda x: self.get_cost(x),                                  │
│    69 │   │   │   │   │   │     reverse=True)                                                    │
│    70 │   │   tasks = []                                                                         │
│    71 │   │   for model in models:                                                               │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/opencom │
│ pass/partitioners/size.py:188 in get_cost                                                        │
│                                                                                                  │
│   185 │   │   │   │   return actual_size, factor                                                 │
│   186 │   │   │   return factor * actual_size                                                    │
│   187 │   │                                                                                      │
│ ❱ 188 │   │   dataset = build_dataset_from_cfg(dataset)                                          │
│   189 │   │   self.dataset_size[dataset_abbr] = len(dataset.test)                                │
│   190 │   │                                                                                      │
│   191 │   │   mmengine.mkdir_or_exist('.cache/')                                                 │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/opencom │
│ pass/utils/build.py:13 in build_dataset_from_cfg                                                 │
│                                                                                                  │
│   10 │   dataset_cfg.pop('infer_cfg', None)                                                      │
│   11 │   dataset_cfg.pop('eval_cfg', None)                                                       │
│   12 │   dataset_cfg.pop('abbr', None)                                                           │
│ ❱ 13 │   return LOAD_DATASET.build(dataset_cfg)                                                  │
│   14                                                                                             │
│   15                                                                                             │
│   16 def build_model_from_cfg(model_cfg: ConfigDict) -> ConfigDict:                              │
│                                                                                                  │
│ /home/liugf/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/mmengine/registry/registry.py: │
│ 570 in build                                                                                     │
│                                                                                                  │
│   567 │   │   │   >>> cfg = dict(type='ResNet', depth=50)                                        │
│   568 │   │   │   >>> model = MODELS.build(cfg)                                                  │
│   569 │   │   """                                                                                │
│ ❱ 570 │   │   return self.build_func(cfg, *args, **kwargs, registry=self)                        │
│   571 │                                                                                          │
│   572 │   def _add_child(self, registry: 'Registry') -> None:                                    │
│   573 │   │   """Add a child for a registry.                                                     │
│                                                                                                  │
│ /home/liugf/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/mmengine/registry/build_functi │
│ ons.py:121 in build_from_cfg                                                                     │
│                                                                                                  │
│   118 │   │   │   │   issubclass(obj_cls, ManagerMixin):  # type: ignore                         │
│   119 │   │   │   obj = obj_cls.get_instance(**args)  # type: ignore                             │
│   120 │   │   else:                                                                              │
│ ❱ 121 │   │   │   obj = obj_cls(**args)  # type: ignore                                          │
│   122 │   │                                                                                      │
│   123 │   │   if (inspect.isclass(obj_cls) or inspect.isfunction(obj_cls)                        │
│   124 │   │   │   │   or inspect.ismethod(obj_cls)):                                             │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/opencom │
│ pass/datasets/base.py:12 in __init__                                                             │
│                                                                                                  │
│    9 class BaseDataset:                                                                          │
│   10 │                                                                                           │
│   11 │   def __init__(self, reader_cfg: Optional[Dict] = {}, **kwargs):                          │
│ ❱ 12 │   │   self.dataset = self.load(**kwargs)                                                  │
│   13 │   │   self._init_reader(**reader_cfg)                                                     │
│   14 │                                                                                           │
│   15 │   def _init_reader(self, **kwargs):                                                       │
│                                                                                                  │
│ /data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/opencom │
│ pass/datasets/hellaswag.py:13 in load                                                            │
│                                                                                                  │
│   10 │                                                                                           │
│   11 │   @staticmethod                                                                           │
│   12 │   def load(**kwargs):                                                                     │
│ ❱ 13 │   │   dataset = load_dataset(**kwargs)                                                    │
│   14 │   │                                                                                       │
│   15 │   │   def preprocess(example):                                                            │
│   16 │   │   │   for i in range(4):                                                              │
│                                                                                                  │
│ /home/liugf/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/datasets/load.py:1773 in       │
│ load_dataset                                                                                     │
│                                                                                                  │
│   1770 │   )                                                                                     │
│   1771 │                                                                                         │
│   1772 │   # Create a dataset builder                                                            │
│ ❱ 1773 │   builder_instance = load_dataset_builder(                                              │
│   1774 │   │   path=path,                                                                        │
│   1775 │   │   name=name,                                                                        │
│   1776 │   │   data_dir=data_dir,                                                                │
│                                                                                                  │
│ /home/liugf/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/datasets/load.py:1502 in       │
│ load_dataset_builder                                                                             │
│                                                                                                  │
│   1499 │   if use_auth_token is not None:                                                        │
│   1500 │   │   download_config = download_config.copy() if download_config else DownloadConfig(  │
│   1501 │   │   download_config.use_auth_token = use_auth_token                                   │
│ ❱ 1502 │   dataset_module = dataset_module_factory(                                              │
│   1503 │   │   path,                                                                             │
│   1504 │   │   revision=revision,                                                                │
│   1505 │   │   download_config=download_config,                                                  │
│                                                                                                  │
│ /home/liugf/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/datasets/load.py:1221 in       │
│ dataset_module_factory                                                                           │
│                                                                                                  │
│   1218 │   │   │   │   │   ) from None                                                           │
│   1219 │   │   │   │   raise e1 from None                                                        │
│   1220 │   else:                                                                                 │
│ ❱ 1221 │   │   raise FileNotFoundError(                                                          │
│   1222 │   │   │   f"Couldn't find a dataset script at {relative_to_absolute_path(combined_path  │
│   1223 │   │   )                                                                                 │
│   1224                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
**FileNotFoundError: Couldn't find a dataset script at
/data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/data/hellaswag/hellaswag.jsonl/hellaswag.jsonl.py or any data
file in the same directory.**


### Reproduces the problem - code/configuration sample

None

### Reproduces the problem - command or script

  None

### Reproduces the problem - error message

  None

### Other information

None

#404 可以看下这个

> #404 可以看下这个
报错原因不太一样，这个数据集文件在
/data/AutoEval/News-Eval/code/evaluation_scripts/public_dataset_eval_scripts/opencompass/data/hellaswag/hellaswag.jsonl这个目录下存在，就是hellaswag.jsonl，但是报错提示没有找到这个目录下一级的hellaswag.jsonl.py这个文件

Feel free to re-open if needed.

遇到了同样的问题，请问解决了吗