Repository: modelscope/evalscope
Issue: evalscope 1.0版本 对于mmlu这种多个sub_list的评测集，会有一个默认值all，报错，且官网没有具体的使用方法
Root cause: Configuration Error

self = <evalscope.benchmarks.mmlu.mmlu_adapter.MMLUAdapter object at 0x7f6ce13b38f0>
load_func = functools.partial(<bound method DefaultDataAdapter.load_subset of <evalscope.benchmarks.mmlu.mmlu_adapter.MMLUAdapter object at 0x7f6ce13b38f0>>, data_loader=<class 'evalscope.api.dataset.loader.RemoteDataLoader'>)
is_fewshot = False
(Pdb) l
198                 load_func (Callable[[str], Dataset]): Function to load individual subsets
199  
200             Returns:
201                 DatasetDict: Dictionary containing all loaded subsets
202             """
203  ->         if self.reformat_subset:
204                 # Load only the default subset
205                 subset_data = load_func(self.default_subset)
206                 # Reformat the subset to create multiple subsets based on sample keys
207                 # NOTE: subset_list and limit is applied here if specified
208                 limit = self.few_shot_num if is_fewshot else self.limit
(Pdb) n
> /usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py(205)load_subsets()
-> subset_data = load_func(self.default_subset)
(Pdb) self.default_subset
'all'
(Pdb) 
'all'
(Pdb) c
2025-10-15 20:21:24 - evalscope - INFO: Loading dataset modelscope/mmlu from modelscope > subset: all > split: test ...
Traceback (most recent call last):
  File "/mnt/workspace/E2Eeval/eval_public.py", line 68, in <module>
    run_task(task_cfg=task_cfg)
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 28, in run_task
    return run_single_task(task_cfg, run_time)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 41, in run_single_task
    result = evaluate_model(task_cfg, outputs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 144, in evaluate_model
    res_dict = evaluator.eval()
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/evaluator/evaluator.py", line 94, in eval
    dataset_dict = self.benchmark.load_dataset()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 62, in load_dataset
    self.test_dataset, self.fewshot_dataset = self.load()
                                              ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 81, in load
    return self.load_from_remote()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 93, in load_from_remote
    test_dataset = self.load_subsets(test_load_func)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 205, in load_subsets
    subset_data = load_func(self.default_subset)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 252, in load_subset
    dataset = loader.load()
              ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/dataset/loader.py", line 88, in load
    dataset = MsDataset.load(
              ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/ms_dataset.py", line 308, in load
    with load_dataset_with_ctx(
  File "/usr/local/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1475, in load_dataset_with_ctx
    dataset_res = DatasetsWrapperHF.load_dataset(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 991, in load_dataset
    builder_instance = DatasetsWrapperHF.load_dataset_builder(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1164, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
                                       ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/datasets/builder.py", line 343, in __init__
    self.config, self.config_id = self._create_builder_config(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/datasets/builder.py", line 570, in _create_builder_config
    raise ValueError(
ValueError: BuilderConfig 'all' not found. Available: ['high_school_european_history', 'business_ethics', 'clinical_knowledge', 'medical_genetics', 'high_school_us_history', 'high_school_physics', 'high_school_world_history', 'virology', 'high_school_microeconomics', 'econometrics', 'college_computer_science', 'high_school_biology', 'abstract_algebra', 'professional_accounting', 'philosophy', 'professional_medicine', 'nutrition', 'global_facts', 'machine_learning', 'security_studies', 'public_relations', 'professional_psychology', 'prehistory', 'anatomy', 'human_sexuality', 'college_medicine', 'high_school_government_and_politics', 'college_chemistry', 'logical_fallacies', 'high_school_geography', 'elementary_mathematics', 'human_aging', 'college_mathematics', 'high_school_psychology', 'formal_logic', 'high_school_statistics', 'international_law', 'high_school_mathematics', 'high_school_computer_science', 'conceptual_physics', 'miscellaneous', 'high_school_chemistry', 'marketing', 'professional_law', 'management', 'college_physics', 'jurisprudence', 'world_religions', 'sociology', 'us_foreign_policy', 'high_school_macroeconomics', 'computer_security', 'moral_scenarios', 'moral_disputes', 'electrical_engineering', 'astronomy', 'college_biology']


真不知道这个all是从哪里来了，为什么官网文档又不能很好维护，怎么用呢。。。

请提供一下完整的评测命令，使用的数据是有`all`这个子集的：

<img width="1924" height="1050" alt="Image" src="https://github.com/user-attachments/assets/8f88fa0d-a0a0-4586-971c-b8f915a4a69a" />

task_cfg = TaskConfig(
    # model='Qwen3-32B',
    # api_url='http://127.0.0.1:8996/v1/chat/completions',
    model=args.model,
    api_url='http://127.0.0.1:' + str(args.port) + '/v1/chat/completions',
    eval_type=args.eval_type,
    datasets=[
        f'{args.dataset}'
        # 'gsm8k','iquiz', 'aime24','mmlu','super_gpqa', 'bbh'
    ],
    dataset_args={
        'aime24': {
            'dataset_id': 'HuggingFaceH4/aime_2024',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            },
        'iquiz': {
            'dataset_id': 'AI-ModelScope/IQuiz',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            },
        'mmlu': {
            'dataset_id': 'modelscope/mmlu',
            'filters': {'remove_until': '</think>'},
            },
        'super_gpqa': {
            'dataset_id': 'm-a-p/SuperGPQA',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            },
        'bbh': {
            'dataset_id': 'modelscope/bbh',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            }, 
        'gsm8k': {
            'dataset_id': 'modelscope/gsm8k',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
        },
    },
    eval_batch_size=args.batch_size,
    generation_config={
        'max_tokens': 30000,  # Max number of generated tokens, suggested to set a large value to avoid output truncation
        'temperature': args.temperature,  # Sampling temperature (recommended value per Qwen report)
        'top_p': args.top_p,  # top-p sampling (recommended value per Qwen report)
        'top_k': args.top_k,  # top-k sampling (recommended value per Qwen report)
        'n': 1,  # Number of replies generated per request
    },
    timeout=70000,  # Timeout
    stream=False,  # Use streaming output
    limit=args.sample_size,
    ignore_errors=True# Set to 100 samples for testing
)

run_task(task_cfg=task_cfg)

上面是评测脚本的主要部分
评测命令是： python3 eval_public2.py --model Qwen3-30B-A3B --dataset mmlu --sample_size 120 --eval_type server

输出是：
root@dsw-640862-7ffbf6d56-27h7l:/mnt/workspace/E2Eeval# python3 eval_public2.py --model Qwen3-30B-A3B --dataset mmlu --sample_size 120 --eval_type server
2025-10-16 19:17:14,936 - numexpr.utils - INFO: Note: NumExpr detected 44 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16.
2025-10-16 19:17:14,936 - numexpr.utils - INFO: NumExpr defaulting to 16 threads.
usage: eval_public2.py [-h] --model MODEL --dataset DATASET --port PORT [--eval_type EVAL_TYPE] [--sample_size SAMPLE_SIZE] [--batch_size BATCH_SIZE]
                       [--temperature TEMPERATURE] [--top_p TOP_P] [--top_k TOP_K]
eval_public2.py: error: the following arguments are required: --port
root@dsw-640862-7ffbf6d56-27h7l:/mnt/workspace/E2Eeval# python3 eval_public2.py --model Qwen3-30B-A3B --dataset mmlu --sample_size 120 --eval_type server --port 8000
2025-10-16 19:17:21,486 - numexpr.utils - INFO: Note: NumExpr detected 44 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16.
2025-10-16 19:17:21,486 - numexpr.utils - INFO: NumExpr defaulting to 16 threads.
In evalscope Current temperature:  0.6
In evalscope Current top_p:  0.95
In evalscope Current top_k:  20
2025-10-16 19:17:21 - evalscope - WARNING: Deprecated: The `timeout` parameter is deprecated and will be removed in v1.1.0. Use `generation_config.timeout` instead.
2025-10-16 19:17:21 - evalscope - WARNING: Deprecated: The `stream` parameter is deprecated and will be removed in v1.1.0. Use `generation_config.stream` instead.
2025-10-16 19:17:21 - evalscope - INFO: Args: Task config is provided with TaskConfig type.
2025-10-16 19:17:22 - evalscope - INFO: Creating model Qwen3-30B-A3B with eval_type=server base_url=http://127.0.0.1:8000/v1/chat/completions, config={'timeout': 70000, 'batch_size': 10, 'stream': False, 'max_tokens': 30000, 'top_p': 0.95, 'temperature': 0.6, 'top_k': 20, 'n': 1}, model_args={}
2025-10-16 19:17:22 - evalscope - WARNING: server in model_apis.py has been deprecated since version 1.0.0. and will be removed in version 1.1.0. Use openai_api instead
2025-10-16 19:17:22 - evalscope - INFO: Dump task config to ./outputs/20251016_191721/configs/task_config_c3b3a0.yaml
2025-10-16 19:17:22 - evalscope - INFO: {
    "model": "Qwen3-30B-A3B",
    "model_id": "Qwen3-30B-A3B",
    "model_args": {},
    "model_task": "text_generation",
    "chat_template": null,
    "datasets": [
        "mmlu"
    ],
    "dataset_args": {
        "aime24": {
            "dataset_id": "HuggingFaceH4/aime_2024",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "iquiz": {
            "dataset_id": "AI-ModelScope/IQuiz",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "mmlu": {
            "name": "mmlu",
            "dataset_id": "modelscope/mmlu",
            "output_types": [
                "generation"
            ],
            "subset_list": [
                "abstract_algebra",
                "anatomy",
                "astronomy",
                "business_ethics",
                "clinical_knowledge",
                "college_biology",
                "college_chemistry",
                "college_computer_science",
                "college_mathematics",
                "college_medicine",
                "college_physics",
                "computer_security",
                "conceptual_physics",
                "econometrics",
                "electrical_engineering",
                "elementary_mathematics",
                "formal_logic",
                "global_facts",
                "high_school_biology",
                "high_school_chemistry",
                "high_school_computer_science",
                "high_school_european_history",
                "high_school_geography",
                "high_school_government_and_politics",
                "high_school_macroeconomics",
                "high_school_mathematics",
                "high_school_microeconomics",
                "high_school_physics",
                "high_school_psychology",
                "high_school_statistics",
                "high_school_us_history",
                "high_school_world_history",
                "human_aging",
                "human_sexuality",
                "international_law",
                "jurisprudence",
                "logical_fallacies",
                "machine_learning",
                "management",
                "marketing",
                "medical_genetics",
                "miscellaneous",
                "moral_disputes",
                "moral_scenarios",
                "nutrition",
                "philosophy",
                "prehistory",
                "professional_accounting",
                "professional_law",
                "professional_medicine",
                "professional_psychology",
                "public_relations",
                "security_studies",
                "sociology",
                "us_foreign_policy",
                "virology",
                "world_religions"
            ],
            "default_subset": "all",
            "few_shot_num": 5,
            "few_shot_random": false,
            "train_split": "dev",
            "eval_split": "test",
            "prompt_template": "Answer the following multiple choice question. The last line of your response should be of the following format: 'ANSWER: $LETTER' (without quotes) where LETTER is one of {letters}. Think step by step before answering.\n\n{question}\n\n{choices}",
            "few_shot_prompt_template": null,
            "system_prompt": null,
            "query_template": null,
            "pretty_name": "MMLU",
            "description": "The MMLU (Massive Multitask Language Understanding) benchmark is a comprehensive evaluation suite designed to assess the performance of language models across a wide range of subjects and tasks. It includes multiple-choice questions from various domains, such as history, science, mathematics, and more, providing a robust measure of a model's understanding and reasoning capabilities.",
            "tags": [
                "Knowledge",
                "MCQ"
            ],
            "filters": {
                "remove_until": "</think>"
            },
            "metric_list": [
                "acc"
            ],
            "aggregation": "mean",
            "shuffle": false,
            "shuffle_choices": false,
            "review_timeout": null,
            "extra_params": {}
        },
        "super_gpqa": {
            "dataset_id": "m-a-p/SuperGPQA",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "bbh": {
            "dataset_id": "modelscope/bbh",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "gsm8k": {
            "dataset_id": "modelscope/gsm8k",
            "filters": {
                "remove_until": "</think>"
            }
        }
    },
    "dataset_dir": "/root/.cache/modelscope/hub/datasets",
    "dataset_hub": "modelscope",
    "repeats": 1,
    "generation_config": {
        "timeout": 70000,
        "batch_size": 10,
        "stream": false,
        "max_tokens": 30000,
        "top_p": 0.95,
        "temperature": 0.6,
        "top_k": 20,
        "n": 1
    },
    "eval_type": "server",
    "eval_backend": "Native",
    "eval_config": null,
    "limit": 120,
    "eval_batch_size": 10,
    "use_cache": null,
    "rerun_review": false,
    "work_dir": "./outputs/20251016_191721",
    "ignore_errors": true,
    "debug": false,
    "seed": 42,
    "api_url": "http://127.0.0.1:8000/v1/chat/completions",
    "timeout": 70000,
    "stream": false,
    "judge_strategy": "auto",
    "judge_worker_num": 1,
    "judge_model_args": {},
    "analysis_report": false,
    "use_sandbox": false,
    "sandbox_type": "docker",
    "sandbox_manager_config": {},
    "sandbox_config": {}
}
2025-10-16 19:17:22 - evalscope - INFO: Loading dataset modelscope/mmlu from modelscope > subset: all > split: test ...
Downloading [README.md]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 503/503 [00:00<00:00, 6.74MB/s]
Downloading [README.md]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.26k/5.26k [00:00<00:00, 39.0MB/s]
Traceback (most recent call last):
  File "/mnt/workspace/E2Eeval/eval_public2.py", line 78, in <module>
    run_task(task_cfg=task_cfg)
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 28, in run_task
    return run_single_task(task_cfg, run_time)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 41, in run_single_task
    result = evaluate_model(task_cfg, outputs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 144, in evaluate_model
    res_dict = evaluator.eval()
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/evaluator/evaluator.py", line 94, in eval
    dataset_dict = self.benchmark.load_dataset()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 62, in load_dataset
    self.test_dataset, self.fewshot_dataset = self.load()
                                              ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 81, in load
    return self.load_from_remote()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 89, in load_from_remote
    test_dataset = self.load_subsets(test_load_func)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 201, in load_subsets
    subset_data = load_func(self.default_subset)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 248, in load_subset
    dataset = loader.load()
              ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/dataset/loader.py", line 88, in load
    dataset = MsDataset.load(
              ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/ms_dataset.py", line 308, in load
    with load_dataset_with_ctx(
  File "/usr/local/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1475, in load_dataset_with_ctx
    dataset_res = DatasetsWrapperHF.load_dataset(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 991, in load_dataset
    builder_instance = DatasetsWrapperHF.load_dataset_builder(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1164, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
                                       ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/datasets/builder.py", line 343, in __init__
    self.config, self.config_id = self._create_builder_config(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/datasets/builder.py", line 570, in _create_builder_config
    raise ValueError(
ValueError: BuilderConfig 'all' not found. Available: ['high_school_european_history', 'business_ethics', 'clinical_knowledge', 'medical_genetics', 'high_school_us_history', 'high_school_physics', 'high_school_world_history', 'virology', 'high_school_microeconomics', 'econometrics', 'college_computer_science', 'high_school_biology', 'abstract_algebra', 'professional_accounting', 'philosophy', 'professional_medicine', 'nutrition', 'global_facts', 'machine_learning', 'security_studies', 'public_relations', 'professional_psychology', 'prehistory', 'anatomy', 'human_sexuality', 'college_medicine', 'high_school_government_and_politics', 'college_chemistry', 'logical_fallacies', 'high_school_geography', 'elementary_mathematics', 'human_aging', 'college_mathematics', 'high_school_psychology', 'formal_logic', 'high_school_statistics', 'international_law', 'high_school_mathematics', 'high_school_computer_science', 'conceptual_physics', 'miscellaneous', 'high_school_chemistry', 'marketing', 'professional_law', 'management', 'college_physics', 'jurisprudence', 'world_religions', 'sociology', 'us_foreign_policy', 'high_school_macroeconomics', 'computer_security', 'moral_scenarios', 'moral_disputes', 'electrical_engineering', 'astronomy', 'college_biology']


脚本中不需要设置dataset id，会自动寻找兼容版本的数据集，你手动设置数据集的并不兼容

ok，get，再多问一句，是不是所有的 评测集的dataset id 都不需要设置，比如iquiz 这些

> ok，get，再多问一句，是不是所有的 评测集的dataset id 都不需要设置，比如iquiz 这些

都不需要手动设置，除非你需要从本地进行加载数据集

    ],
    dataset_args={
        'aime24': {
            'dataset_id': 'HuggingFaceH4/aime_2024',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            },
        'iquiz': {
            # 'local_path': "/usr/local/lib/python3.12/site-packages/evalscope/custom_eval/",
            # 'subset_list':['IQ', 'EQ'],
            # 'dataset_id': 'AI-ModelScope/IQuiz',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            },
        'mmlu': {
            # 'dataset_id': 'modelscope/mmlu',
            'dataset_id': 'cais/mmlu',   #1.0.2版本不需要dataset_id, 默认的是cais/mmlu
            'filters': {'remove_until': '</think>'},
            # 'subset_list':mmlu_subset_list
            },
        'super_gpqa': {
            'dataset_id': 'm-a-p/SuperGPQA',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            },
        'bbh': {
            'dataset_id': 'modelscope/bbh',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
            }, 
        'gsm8k': {
            'dataset_id': 'modelscope/gsm8k',
            'filters': {'remove_until': '</think>'}  # Filter out the content of thinking
        },
    },


同样的运行命令：
python3 eval_public.py --model Qwen3-30B-A3B --port 8000 --eval_type server --sample_size 10 --dataset iquiz


在iquiz上报错
    builder_instance = DatasetsWrapperHF.load_dataset_builder(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1125, in load_dataset_builder
    dataset_module = DatasetsWrapperHF.dataset_module_factory(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1425, in dataset_module_factory
    raise FileNotFoundError(
FileNotFoundError: Couldn't find a dataset script at /mnt/workspace/E2Eeval/AI-ModelScope/IQuiz/IQuiz.py or any data file in the same directory. Couldn't find 'AI-ModelScope/IQuiz' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']

补充一下，试了一下 去掉dataset_id, mmlu去掉dataset_id是ok的，iquiz 不 ok


你本地的这个路径是否存在`/mnt/workspace/E2Eeval/AI-ModelScope/IQuiz`，若存在需要把它删掉

root@dsw-640862-5df5bf847b-rb8vg:/mnt/workspace/E2Eeval# ls AI-ModelScope/IQuiz
ls: cannot access 'AI-ModelScope/IQuiz': No such file or directory
root@dsw-640862-5df5bf847b-rb8vg:/mnt/workspace/E2Eeval# ls AI-ModelScope
ls: cannot access 'AI-ModelScope': No such file or directory

没有这两个路径

我这边无法复现这个问题，你可以尝试单独运行一下iquiz的评测，看看是否能成功；此外看一下本地是否存在缓存`~/.cache/modelscope/hub/datasets`，清空后再尝试运行。

可以 上面这个问题，我明天验证一下

关于第893个issue，已经被关闭了，下午有些改动么有save，我其实只是想问问，根本的rootcause是不是：016版本，可能因 token 限制、stop token 设置等原因截断了答案部分；  谢谢！

我今天把~/.cache/modelscope 下的所有datasets都删掉了。运行


我看  AI-ModelScope/IQuiz 是在 /usr/local/lib/python3.12/site-packages/evalscope/benchmarks/iquiz/iquiz_adapter.py文件中定义好的。

如果 这个文件中定义了  AI-ModelScope/IQuiz 命名空间，那它确定没有问题的话，剩下的就是从哪里获取数据的问题：
 Couldn't find 'AI-ModelScope/IQuiz' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl'  这个问题就是 从hf上没有拿到了。 这个hf的路径是对的么？


下面就是全部的运行打印信息了


root@dsw-646931-6b7bd7df9-tp2jj:/mnt/data/E2Eeval#  python3 eval_public.py --model Qwen3-30B-A3B --port 8000 --eval_type server --sample_size 10 --dataset iquiz
2025-10-21 13:33:08,479 - numexpr.utils - INFO: Note: NumExpr detected 44 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16.
2025-10-21 13:33:08,479 - numexpr.utils - INFO: NumExpr defaulting to 16 threads.
In evalscope Current temperature:  0.6
In evalscope Current top_p:  0.95
In evalscope Current top_k:  20
2025-10-21 13:33:08 - evalscope - WARNING: Deprecated: The `timeout` parameter is deprecated and will be removed in v1.1.0. Use `generation_config.timeout` instead.
2025-10-21 13:33:08 - evalscope - WARNING: Deprecated: The `stream` parameter is deprecated and will be removed in v1.1.0. Use `generation_config.stream` instead.
2025-10-21 13:33:08 - evalscope - INFO: Args: Task config is provided with TaskConfig type.
2025-10-21 13:33:09 - evalscope - INFO: Creating model Qwen3-30B-A3B with eval_type=server base_url=http://127.0.0.1:8000/v1/chat/completions, config={'timeout': 70000, 'batch_size': 10, 'stream': False, 'max_tokens': 30000, 'top_p': 0.95, 'temperature': 0.6, 'top_k': 20, 'n': 1}, model_args={}
2025-10-21 13:33:09 - evalscope - WARNING: server in model_apis.py has been deprecated since version 1.0.0. and will be removed in version 1.1.0. Use openai_api instead
2025-10-21 13:33:09 - evalscope - INFO: Dump task config to ./outputs/20251021_133308/configs/task_config_9d3981.yaml
2025-10-21 13:33:09 - evalscope - INFO: {
    "model": "Qwen3-30B-A3B",
    "model_id": "Qwen3-30B-A3B",
    "model_args": {},
    "model_task": "text_generation",
    "chat_template": null,
    "datasets": [
        "iquiz"
    ],
    "dataset_args": {
        "aime24": {
            "dataset_id": "HuggingFaceH4/aime_2024",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "iquiz": {
            "name": "iquiz",
            "dataset_id": "AI-ModelScope/IQuiz",    ## 源自python3.12/site-packages/evalscope/benchmarks/iquiz/iquiz_adapter.py
            "output_types": [
                "generation"
            ],
            "subset_list": [
                "IQ",
                "EQ"
            ],
            "default_subset": "default",
            "few_shot_num": 0,
            "few_shot_random": false,
            "train_split": null,
            "eval_split": "test",
            "prompt_template": "回答下面的单项选择题，请选出其中的正确答案。你的回答的最后一行应该是这样的格式：\"LETTER\"（不带引号），其中 LETTER 是 {letters} 中的一个。请在回答前进行一步步思考。\n\n问题：{question}\n选项：\n{choices}\n",
            "few_shot_prompt_template": null,
            "system_prompt": null,
            "query_template": null,
            "pretty_name": "IQuiz",
            "description": "IQuiz is a benchmark for evaluating AI models on IQ and EQ questions. It consists of multiple-choice questions where the model must select the correct answer and provide an explanation.",
            "tags": [
                "Knowledge",
                "MCQ",
                "Chinese"
            ],
            "filters": {
                "remove_until": "</think>"
            },
            "metric_list": [
                "acc"
            ],
            "aggregation": "mean",
            "shuffle": false,
            "shuffle_choices": false,
            "review_timeout": null,
            "extra_params": {}
        },
        "mmlu": {
            "dataset_id": "cais/mmlu",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "super_gpqa": {
            "dataset_id": "m-a-p/SuperGPQA",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "bbh": {
            "dataset_id": "modelscope/bbh",
            "filters": {
                "remove_until": "</think>"
            }
        },
        "gsm8k": {
            "dataset_id": "modelscope/gsm8k",
            "filters": {
                "remove_until": "</think>"
            }
        }
    },
    "dataset_dir": "/root/.cache/modelscope/hub/datasets",
    "dataset_hub": "modelscope",
    "repeats": 1,
    "generation_config": {
        "timeout": 70000,
        "batch_size": 10,
        "stream": false,
        "max_tokens": 30000,
        "top_p": 0.95,
        "temperature": 0.6,
        "top_k": 20,
        "n": 1
    },
    "eval_type": "server",
    "eval_backend": "Native",
    "eval_config": null,
    "limit": 10,
    "eval_batch_size": 10,
    "use_cache": null,
    "rerun_review": false,
    "work_dir": "./outputs/20251021_133308",
    "ignore_errors": true,
    "debug": false,
    "seed": 42,
    "api_url": "http://127.0.0.1:8000/v1/chat/completions",
    "timeout": 70000,
    "stream": false,
    "judge_strategy": "auto",
    "judge_worker_num": 1,
    "judge_model_args": {},
    "analysis_report": false,
    "use_sandbox": false,
    "sandbox_type": "docker",
    "sandbox_manager_config": {},
    "sandbox_config": {}
}
2025-10-21 13:33:10 - evalscope - INFO: Loading dataset AI-ModelScope/IQuiz from modelscope > subset: IQ > split: test ...
Downloading [README.md]: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.33k/3.33k [00:00<00:00, 28.7MB/s]
2025-10-21 13:33:17,843 - modelscope - ERROR - >> Error loading AI-ModelScope/IQuiz: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']
Traceback (most recent call last):
  File "/mnt/data/E2Eeval/eval_public.py", line 75, in <module>
    run_task(task_cfg=task_cfg)
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 28, in run_task
    return run_single_task(task_cfg, run_time)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 41, in run_single_task
    result = evaluate_model(task_cfg, outputs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/run.py", line 144, in evaluate_model
    res_dict = evaluator.eval()
               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/evaluator/evaluator.py", line 94, in eval
    dataset_dict = self.benchmark.load_dataset()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 62, in load_dataset
    self.test_dataset, self.fewshot_dataset = self.load()
                                              ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 81, in load
    return self.load_from_remote()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 89, in load_from_remote
    test_dataset = self.load_subsets(test_load_func)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 215, in load_subsets
    subset_data = load_func(subset)
                  ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 248, in load_subset
    dataset = loader.load()
              ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/evalscope/api/dataset/loader.py", line 88, in load
    dataset = MsDataset.load(
              ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/ms_dataset.py", line 308, in load
    with load_dataset_with_ctx(
  File "/usr/local/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1475, in load_dataset_with_ctx
    dataset_res = DatasetsWrapperHF.load_dataset(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 991, in load_dataset
    builder_instance = DatasetsWrapperHF.load_dataset_builder(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1125, in load_dataset_builder
    dataset_module = DatasetsWrapperHF.dataset_module_factory(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1425, in dataset_module_factory
    raise FileNotFoundError(
FileNotFoundError: Couldn't find a dataset script at /mnt/data/E2Eeval/AI-ModelScope/IQuiz/IQuiz.py or any data file in the same directory. Couldn't find 'AI-ModelScope/IQuiz' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']


请问你环境中的datasets库版本是多少？建议使用datasets==3.6.0，直接运行下面这段代码能否成功：
```python
from modelscope.msdatasets import MsDataset
ds =  MsDataset.load('AI-ModelScope/IQuiz', subset_name='IQ', split='test')
```

之前版本是3.2.0，升级了一下：

root@dsw-646931-6b7bd7df9-tp2jj:/mnt/data/E2Eeval# pip install datasets==3.6.0 -i https://mirrors.aliyun.com/pypi/simple/
aiext pypi plugin has been loaded.
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collecting datasets==3.6.0
  Downloading https://mirrors.aliyun.com/pypi/packages/20/34/a08b0ee99715eaba118cbe19a71f7b5e2425c2718ef96007c325944a1152/datasets-3.6.0-py3-none-any.whl (491 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 491.5/491.5 kB 885.8 kB/s eta 0:00:00
Requirement already satisfied: filelock in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (3.19.1)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (1.26.0)
Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (20.0.0)
Requirement already satisfied: dill<0.3.9,>=0.3.0 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (0.3.6)
Requirement already satisfied: pandas in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (2.2.3)
Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (2.32.5)
Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (4.67.1)
Requirement already satisfied: xxhash in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (3.2.0)
Requirement already satisfied: multiprocess<0.70.17 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (0.70.14)
Requirement already satisfied: fsspec<=2025.3.0,>=2023.1.0 in /usr/local/lib/python3.12/site-packages (from fsspec[http]<=2025.3.0,>=2023.1.0->datasets==3.6.0) (2023.6.0)
Requirement already satisfied: huggingface-hub>=0.24.0 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (0.30.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (24.2)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.12/site-packages (from datasets==3.6.0) (6.0.2)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.12/site-packages (from fsspec[http]<=2025.3.0,>=2023.1.0->datasets==3.6.0) (3.9.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.12/site-packages (from huggingface-hub>=0.24.0->datasets==3.6.0) (4.15.0)
Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/site-packages (from requests>=2.32.2->datasets==3.6.0) (2.0.12)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.12/site-packages (from requests>=2.32.2->datasets==3.6.0) (3.3)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/site-packages (from requests>=2.32.2->datasets==3.6.0) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.12/site-packages (from requests>=2.32.2->datasets==3.6.0) (2022.9.24)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/site-packages (from pandas->datasets==3.6.0) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/site-packages (from pandas->datasets==3.6.0) (2022.1)
Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/site-packages (from pandas->datasets==3.6.0) (2022.7)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.12/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets==3.6.0) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.12/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets==3.6.0) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.12/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets==3.6.0) (1.9.2)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.12/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets==3.6.0) (1.4.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.12/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets==3.6.0) (1.3.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas->datasets==3.6.0) (1.16.0)
Installing collected packages: datasets
  Attempting uninstall: datasets
    Found existing installation: datasets 3.2.0
    Uninstalling datasets-3.2.0:
      Successfully uninstalled datasets-3.2.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
evalscope 0.17.0 requires datasets==3.2.0, but you have datasets 3.6.0 which is incompatible.
Successfully installed datasets-3.6.0

运行您这边刚刚给的load方法：
root@dsw-646931-6b7bd7df9-tp2jj:/mnt/data/E2Eeval# python
Python 3.12.3 (main, Aug  1 2025, 10:18:36) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from modelscope.msdatasets import MsDataset
>>> ds =  MsDataset.load('AI-ModelScope/IQuiz', subset_name='IQ', split='test')
2025-10-21 14:04:10,228 - modelscope - ERROR - >> Error loading AI-ModelScope/IQuiz: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/ms_dataset.py", line 308, in load
    with load_dataset_with_ctx(
  File "/usr/local/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1475, in load_dataset_with_ctx
    dataset_res = DatasetsWrapperHF.load_dataset(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 991, in load_dataset
    builder_instance = DatasetsWrapperHF.load_dataset_builder(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1125, in load_dataset_builder
    dataset_module = DatasetsWrapperHF.dataset_module_factory(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1425, in dataset_module_factory
    raise FileNotFoundError(
FileNotFoundError: Couldn't find a dataset script at /mnt/data/E2Eeval/AI-ModelScope/IQuiz/IQuiz.py or any data file in the same directory. Couldn't find 'AI-ModelScope/IQuiz' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']
>>> 

我在想是不是还有其他的包导致的？？？？

modelscope这个库也建议升级一下到最新版本1.31.0再尝试运行一下，如果还是不行的话，建议新建一个干净的环境再测试一下

还是不行啊 我不明白为什么0.17.0可以的环境

升级了evalscope 到1.1.0  modelscope 和datasets 都升级到你刚刚说的最新版本
就还是无法从huggingfacehub上拉到 iquiz

0.17.0的和1.1.0的diff 是哪里啊，好诡异啊。。。

2025-10-21 15:03:55 - evalscope - INFO: Loading dataset AI-ModelScope/IQuiz from modelscope > subset: IQ > split: test ...
2025-10-21 15:04:01,730 - modelscope - ERROR - >> Error loading AI-ModelScope/IQuiz: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']
Using the latest cached version of the dataset since AI-ModelScope/IQuiz couldn't be found on the Hugging Face Hub

huggingface datasets 上好像确实没有IQuiz，modelscope datasets上才有IQuiz，它为什么回去到hf上去找IQuiz:
 Couldn't find 'AI-ModelScope/IQuiz' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension


默认就是在modelscope上查找数据集，不会去huggingface的

尝试以下配置： 

1.  versions
```python
modelscope==1.31.0
datasets==3.6.0
```

2. usage
```python
from modelscope import MsDataset
from modelscope.utils.constant import DownloadMode

ds = MsDataset.load(
    'AI-ModelScope/IQuiz',
    split='test',
    download_mode=DownloadMode.FORCE_REDOWNLOAD
)

# 备注：正常使用的时候可以不需加download_mode=DownloadMode.FORCE_REDOWNLOAD，这里用于调试

print(next(iter(ds)))
```

本地执行结果： 

<img width="3024" height="1052" alt="Image" src="https://github.com/user-attachments/assets/01e64840-0b55-4176-9e11-8f08832f6023" />

按照大佬的方式 run了一下，还是不行
我比较好奇的是 用0.17.0的那一套包 就能run ，实在是不知道差异在什么地方

root@dsw-646931-7fd849b86c-2hn48:/mnt/workspace# pip list | grep datasets
datasets                          3.6.0
root@dsw-646931-7fd849b86c-2hn48:/mnt/workspace# pip list | grep modelscope
modelscope                        1.31.0

root@dsw-646931-7fd849b86c-2hn48:/mnt/workspace# python3 test.py 
Downloading [README.md]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.33k/3.33k [00:00<00:00, 32.6MB/s]
2025-10-22 08:55:05,788 - modelscope - INFO - storing https://www.modelscope.cn/api/v1/datasets/AI-ModelScope/IQuiz/repo?Source=SDK&Revision=master&FilePath=README.md&View=False in cache at /root/.cache/modelscope/hub/datasets/455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d
2025-10-22 08:55:05,789 - modelscope - INFO - creating metadata file for /root/.cache/modelscope/hub/datasets/455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d
Downloading [README.md]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.33k/3.33k [00:00<00:00, 39.3MB/s]
2025-10-22 08:55:08,308 - modelscope - INFO - storing https://www.modelscope.cn/api/v1/datasets/AI-ModelScope/IQuiz/repo?Source=SDK&Revision=master&FilePath=README.md&View=False in cache at /root/.cache/modelscope/hub/datasets/455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d
2025-10-22 08:55:08,308 - modelscope - INFO - creating metadata file for /root/.cache/modelscope/hub/datasets/455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d
2025-10-22 08:55:10,583 - modelscope - ERROR - >> Error loading AI-ModelScope/IQuiz: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']
Traceback (most recent call last):
  File "/mnt/workspace/test.py", line 4, in <module>
    ds = MsDataset.load(
         ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/ms_dataset.py", line 308, in load
    with load_dataset_with_ctx(
  File "/usr/local/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1475, in load_dataset_with_ctx
    dataset_res = DatasetsWrapperHF.load_dataset(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 991, in load_dataset
    builder_instance = DatasetsWrapperHF.load_dataset_builder(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1125, in load_dataset_builder
    dataset_module = DatasetsWrapperHF.dataset_module_factory(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/modelscope/msdatasets/utils/hf_datasets_util.py", line 1425, in dataset_module_factory
    raise FileNotFoundError(
FileNotFoundError: Couldn't find a dataset script at /mnt/workspace/AI-ModelScope/IQuiz/IQuiz.py or any data file in the same directory. Couldn't find 'AI-ModelScope/IQuiz' on the Hugging Face Hub either: FileNotFoundError: Unable to find 'hf://datasets/AI-ModelScope/IQuiz@master/IQ.jsonl' with any supported extension ['.csv', '.tsv', '.json', '.jsonl', '.ndjson', '.parquet', '.geoparquet', '.gpq', '.arrow', '.txt', '.tar', '.xml', '.blp', '.bmp', '.dib', '.bufr', '.cur', '.pcx', '.dcx', '.dds', '.ps', '.eps', '.fit', '.fits', '.fli', '.flc', '.ftc', '.ftu', '.gbr', '.gif', '.grib', '.png', '.apng', '.jp2', '.j2k', '.jpc', '.jpf', '.jpx', '.j2c', '.icns', '.ico', '.im', '.iim', '.tif', '.tiff', '.jfif', '.jpe', '.jpg', '.jpeg', '.mpg', '.mpeg', '.msp', '.pcd', '.pxr', '.pbm', '.pgm', '.ppm', '.pnm', '.psd', '.bw', '.rgb', '.rgba', '.sgi', '.ras', '.tga', '.icb', '.vda', '.vst', '.webp', '.wmf', '.emf', '.xbm', '.xpm', '.BLP', '.BMP', '.DIB', '.BUFR', '.CUR', '.PCX', '.DCX', '.DDS', '.PS', '.EPS', '.FIT', '.FITS', '.FLI', '.FLC', '.FTC', '.FTU', '.GBR', '.GIF', '.GRIB', '.PNG', '.APNG', '.JP2', '.J2K', '.JPC', '.JPF', '.JPX', '.J2C', '.ICNS', '.ICO', '.IM', '.IIM', '.TIF', '.TIFF', '.JFIF', '.JPE', '.JPG', '.JPEG', '.MPG', '.MPEG', '.MSP', '.PCD', '.PXR', '.PBM', '.PGM', '.PPM', '.PNM', '.PSD', '.BW', '.RGB', '.RGBA', '.SGI', '.RAS', '.TGA', '.ICB', '.VDA', '.VST', '.WEBP', '.WMF', '.EMF', '.XBM', '.XPM', '.aiff', '.au', '.avr', '.caf', '.flac', '.htk', '.svx', '.mat4', '.mat5', '.mpc2k', '.ogg', '.paf', '.pvf', '.raw', '.rf64', '.sd2', '.sds', '.ircam', '.voc', '.w64', '.wav', '.nist', '.wavex', '.wve', '.xi', '.mp3', '.opus', '.AIFF', '.AU', '.AVR', '.CAF', '.FLAC', '.HTK', '.SVX', '.MAT4', '.MAT5', '.MPC2K', '.OGG', '.PAF', '.PVF', '.RAW', '.RF64', '.SD2', '.SDS', '.IRCAM', '.VOC', '.W64', '.WAV', '.NIST', '.WAVEX', '.WVE', '.XI', '.MP3', '.OPUS', '.mkv', '.mp4', '.avi', '.mov', '.MKV', '.MP4', '.AVI', '.MOV', '.pdf', '.PDF', '.zip']

root@dsw-646931-7fd849b86c-2hn48:/mnt/workspace# cat test.py 
from modelscope import MsDataset
from modelscope.utils.constant import DownloadMode

ds = MsDataset.load(
    'AI-ModelScope/IQuiz',
    split='test',
    download_mode=DownloadMode.FORCE_REDOWNLOAD
)

# 备注：正常使用的时候可以不需加download_mode=DownloadMode.FORCE_REDOWNLOAD，这里用于调试

print(next(iter(ds)))

那确实有点奇怪，不排除是其他依赖版本问题导致的，可以重新建一个conda env来验证一下： 

```markdown
1. conda create -n test_ds python=3.11
2. conda activate test_ds
3. pip install 'modelscope[datasets]' -U
```

然后执行如下代码： 

```python
from modelscope import MsDataset
from modelscope.utils.constant import DownloadMode

ds = MsDataset.load(
    'AI-ModelScope/IQuiz',
    split='test',
    subset_name='IQ',
    cache_dir='/path/to/your_test_dir'
)

print(next(iter(ds)))
```
注意这里`cache_dir`替换成自己的路径（系统默认会在~/.cache/modelscope/hub/datasets 文件夹下建立缓存，这里为了规避已经下载过的数据对测试的影响，单独指定`cache_dir`）


(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# python3 test.py 
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8.72k/8.72k [00:00<00:00, 41.1MB/s]
2025-10-22 20:23:57,588 - modelscope - INFO - storing https://www.modelscope.cn/api/v1/datasets/AI-ModelScope/IQuiz/repo?Source=SDK&Revision=master&FilePath=IQ.jsonl in cache at ./test/downloads/2f58ddb3473e913f97f5c2a7564d4197c6e8d0b9f6ff14342faa4dfae6ef1bb9
2025-10-22 20:23:57,597 - modelscope - INFO - creating metadata file for ./test/downloads/2f58ddb3473e913f97f5c2a7564d4197c6e8d0b9f6ff14342faa4dfae6ef1bb9
Generating test split: 40 examples [00:00, 2939.92 examples/s]
{'question': '天气预报说本周星期三会下雨，昨天果然下雨了，今天星期几？', 'choices': ['星期一', '星期二', '星期三', '星期四'], 'answer': 'D', 'level': 1}

=========================
(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# pip list | grep datasets
datasets               3.6.0
(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# pip list | grep modelscope
modelscope             1.31.0

=========================
(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# cat test.py 
from modelscope import MsDataset
from modelscope.utils.constant import DownloadMode

ds = MsDataset.load(
    'AI-ModelScope/IQuiz',
    split='test',
    subset_name='IQ',
    cache_dir='./test/'
)

print(next(iter(ds)))

搞了一个conda的环境，包的版本是与这个evalscope版本match的，现在结果是ok的。
从这个能大概定位到 rootcause的出处吗？目前看是原来的环境 的问题。

补充一下

我用了默认的.cache ，前提将cache目录下的modescope和hf的路径都rm掉了。
结果也没问题

(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# python3 test.py 
Downloading [README.md]: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 3.33k/3.33k [00:00<00:00, 22.2MB/s]
2025-10-22 20:28:47,966 - modelscope - INFO - storing https://www.modelscope.cn/api/v1/datasets/AI-ModelScope/IQuiz/repo?Source=SDK&Revision=master&FilePath=README.md&View=False in cache at /root/.cache/modelscope/hub/datasets/455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d
2025-10-22 20:28:47,966 - modelscope - INFO - creating metadata file for /root/.cache/modelscope/hub/datasets/455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d
Downloading data: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8.72k/8.72k [00:00<00:00, 43.1MB/s]
2025-10-22 20:28:53,691 - modelscope - INFO - storing https://www.modelscope.cn/api/v1/datasets/AI-ModelScope/IQuiz/repo?Source=SDK&Revision=master&FilePath=IQ.jsonl in cache at /root/.cache/modelscope/hub/datasets/downloads/2f58ddb3473e913f97f5c2a7564d4197c6e8d0b9f6ff14342faa4dfae6ef1bb9
2025-10-22 20:28:53,691 - modelscope - INFO - creating metadata file for /root/.cache/modelscope/hub/datasets/downloads/2f58ddb3473e913f97f5c2a7564d4197c6e8d0b9f6ff14342faa4dfae6ef1bb9
Generating test split: 40 examples [00:00, 11204.98 examples/s]
{'question': '天气预报说本周星期三会下雨，昨天果然下雨了，今天星期几？', 'choices': ['星期一', '星期二', '星期三', '星期四'], 'answer': 'D', 'level': 1}
(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# ls ~/.cache/
Microsoft  YAPF  conda  conda-anaconda-tos  evalscope  flashinfer  jedi  modelscope  pip  vllm
(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# ls ~/.cache/modelscope/
hub
(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# ls ~/.cache/modelscope/hub/
datasets
(test_ds) root@dsw-646931-79fdf65cd9-q57x9:/mnt/workspace# ls ~/.cache/modelscope/hub/datasets/
455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d       AI-ModelScope___i_quiz
455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d.json  _root_.cache_modelscope_hub_datasets_AI-ModelScope___i_quiz_IQ-08d7a10bc016ff0d_0.0.0_master.lock
455e4c972aa16ac253e89f5eba46344e4b48ed065dfe46d624a0d1be4fbaee6d.lock  downloads

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！