Repository: open-compass/opencompass
Issue: [Bug] Can NOT find valid BAAI/TACO dataset
Root cause: Configuration Error

### Prerequisite

- [x] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [x] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

```python
{'CUDA available': True,
 'CUDA_HOME': '/usr/local/cuda',
 'GCC': 'gcc (GCC) 11.4.0',
 'GPU 0,1,2,3,4,5,6,7': 'NVIDIA H20',
 'MMEngine': '0.10.7',
 'MUSA available': False,
 'NVCC': 'Cuda compilation tools, release 12.8, V12.8.93',
 'PyTorch': '2.7.1+cu126',
 'PyTorch compiling details': 'PyTorch built with:\n'
                              '  - GCC 11.2\n'
                              '  - C++ Version: 201703\n'
                              '  - Intel(R) oneAPI Math Kernel Library Version '
                              '2024.2-Product Build 20240605 for Intel(R) 64 '
                              'architecture applications\n'
                              '  - Intel(R) MKL-DNN v3.7.1 (Git Hash '
                              '8d263e693366ef8db40acc569cc7d8edf644556d)\n'
                              '  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
                              '  - LAPACK is enabled (usually provided by '
                              'MKL)\n'
                              '  - NNPACK is enabled\n'
                              '  - CPU capability usage: AVX512\n'
                              '  - CUDA Runtime 12.6\n'
                              '  - NVCC architecture flags: '
                              '-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n'       
                              '  - CuDNN 90.5.1\n'
                              '  - Magma 2.6.1\n'
                              '  - Build settings: BLAS_INFO=mkl, '
                              'BUILD_TYPE=Release, '
                              'COMMIT_SHA=e2d141dbde55c2a4370fac5165b0561b6af4798b, '
                              'CUDA_VERSION=12.6, CUDNN_VERSION=9.5.1, '
                              'CXX_COMPILER=/opt/rh/gcc-toolset-11/root/usr/bin/c++, '
                              'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 '
                              '-fvisibility-inlines-hidden -DUSE_PTHREADPOOL '
                              '-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER '
                              '-DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM '
                              '-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK '
                              '-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC '
                              '-Wall -Wextra -Werror=return-type '
                              '-Werror=non-virtual-dtor '
                              '-Werror=range-loop-construct '
                              '-Werror=bool-operation -Wnarrowing '
                              '-Wno-missing-field-initializers '
                              '-Wno-unknown-pragmas -Wno-unused-parameter '
                              '-Wno-strict-overflow -Wno-strict-aliasing '
                              '-Wno-stringop-overflow -Wsuggest-override '
                              '-Wno-psabi -Wno-error=old-style-cast '
                              '-fdiagnostics-color=always -faligned-new '
                              '-Wno-maybe-uninitialized -fno-math-errno '
                              '-fno-trapping-math -Werror=format '
                              '-Wno-stringop-overflow, LAPACK_INFO=mkl, '
                              'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
                              'TORCH_VERSION=2.7.1, USE_CUDA=ON, USE_CUDNN=ON, '
                              'USE_CUSPARSELT=1, USE_GFLAGS=OFF, USE_GLOG=OFF, '
                              'USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, '
                              'USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, '
                              'USE_OPENMP=ON, USE_ROCM=OFF, '
                              'USE_ROCM_KERNEL_ASSERT=OFF, \n',
 'Python': '3.10.18 (main, Jun  5 2025, 13:14:17) [GCC 11.2.0]',
 'TorchVision': '0.22.1+cu126',
 'lmdeploy': "not installed:No module named 'lmdeploy'",
 'numpy_random_seed': 2147483648,
 'opencompass': '0.4.2+6eee45e',
 'sys.platform': 'linux',
 'transformers': '4.56.0'}
```

### Reproduces the problem - code/configuration sample

```python
from mmengine.config import read_base

from opencompass.models import VLLMwithChatTemplate, OpenAISDK
from opencompass.partitioners import NaivePartitioner, NumWorkerPartitioner
from opencompass.runners import LocalRunner
from opencompass.tasks import OpenICLInferTask, OpenICLEvalTask
from opencompass.utils.text_postprocessors import extract_non_reasoning_content


#######################################################################
#                          PART 1  Datasets List                      #
#######################################################################

with read_base():
    from opencompass.configs.datasets.taco.taco_gen_c7893a import TACO_datasets


datasets = sum(
    (v for k, v in locals().items() if k.endswith("_datasets")),
    [],
)

#######################################################################
#                          PART 2  Model List                         #
#######################################################################

models = [
    dict(
        type=VLLMwithChatTemplate,
        abbr="deepseek-r1-distill-qwen-1.5b-vllm",
        path="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
        max_out_len=32768,
        batch_size=16,
        generation_kwargs=dict(temperature=0.6, top_p=0.95, do_sample=True),
        model_kwargs=dict(tensor_parallel_size=1),
        run_cfg=dict(num_gpus=1),
        pred_postprocessor=dict(type=extract_non_reasoning_content)
    )
]


#######################################################################
#                          PART 3  Inference/Evaluation               #
#######################################################################

# Inference configuration
infer = dict(
    partitioner=dict(
        type=NumWorkerPartitioner,
        num_worker=8
    ),
    runner=dict(
        type=LocalRunner,
        task=dict(type=OpenICLInferTask)
    ),
)

# Evaluation configuration
eval = dict(
    partitioner=dict(
        type=NaivePartitioner, n=8
    ),
    runner=dict(
        type=LocalRunner,
        task=dict(
            type=OpenICLEvalTask)
    ),
)


#######################################################################
#                          PART 4  Utils                              #
#######################################################################

work_dir = "outputs/deepseek_r1_distill_qwen_1p5b_taco"
```

### Reproduces the problem - command or script

```shell
python run.py eval_deepseek_r1_distill_qwen_1p5b_taco.py --debug
```

### Reproduces the problem - error message

```
INFO 08-31 23:20:47 [__init__.py:241] Automatically detected platform cuda.
08/31 23:20:48 - OpenCompass - INFO - Current exp folder: outputs/deepseek_r1_distill_qwen_1p5b_taco/20250831_232048
08/31 23:20:48 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
08/31 23:20:48 - OpenCompass - INFO - ./data/BAAI-TACO does not exist!Start Download data automatically!If you have downloaded the data before,You can specific `COMPASS_DATA_CACHE` to avoid downloading~
Traceback (most recent call last):
  File "/root/opencompass/run.py", line 4, in <module>
    main()
  File "/root/opencompass/opencompass/cli/main.py", line 345, in main
    tasks = partitioner(cfg)
  File "/root/opencompass/opencompass/partitioners/base.py", line 83, in __call__
    tasks = self.partition(**model_and_dataset_args,
  File "/root/opencompass/opencompass/partitioners/num_worker.py", line 72, in partition
    dataset_size = self.get_size(dataset)
  File "/root/opencompass/opencompass/partitioners/num_worker.py", line 145, in get_size
    dataset = build_dataset_from_cfg(dataset)
  File "/root/opencompass/opencompass/utils/build.py", line 12, in build_dataset_from_cfg
    return LOAD_DATASET.build(dataset_cfg)
  File "/root/miniconda3/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/root/miniconda3/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/root/opencompass/opencompass/datasets/base.py", line 19, in __init__
    dataset = self.load(**kwargs)
  File "/root/opencompass/opencompass/datasets/taco.py", line 42, in load
    path = get_data_path(path, local_mode=True)
  File "/root/opencompass/opencompass/utils/datasets.py", line 30, in get_data_path
    return download_dataset(local_path, cache_dir)
  File "/root/opencompass/opencompass/utils/datasets.py", line 90, in download_dataset
    assert dataset_name, f'No valid url for {data_path}!\n' + \
AssertionError: No valid url for ./data/BAAI-TACO!
Please make sure  `./data/BAAI-TACO` is correct
```

### Other information

data_path is a local path:
https://github.com/open-compass/opencompass/blob/main/opencompass/configs/datasets/taco/taco_gen_c7893a.py#L22

https://github.com/open-compass/opencompass/blob/main/opencompass/datasets/taco.py#L42-L43

but I cannot find a valid dataset id or url in:
https://github.com/open-compass/opencompass/blob/main/opencompass/utils/datasets_info.py

hi, @ShaneTian , thank you for providing this problem. We will make repairs in the future.

Temporarily, you can make the following modifications in the project to ensure your evaluation runs.

1) replace the path in `opencompass/configs/datasets/taco/taco_gen_c7893a.py` with a HF path:

```
TACO_datasets = [
    dict(
        type=TACODataset,
        abbr='TACO',
        path='BAAI/TACO',
        num_repeats = 1,
        reader_cfg=TACO_reader_cfg,
        infer_cfg=TACO_infer_cfg,
        eval_cfg=TACO_eval_cfg,
    )
]
```

2) replace the dataset load mode in `opencompass/datasets/taco.py` with `load_dataset`:

```
class TACODataset(BaseDataset):

    @staticmethod
    def load(path: str, num_repeats: int = 1, difficulty='ALL'):
        # path = get_data_path(path, local_mode=True)
        dataset = load_dataset(path)
        new_dataset = DatasetDict()
        ...
```