Repository: modelscope/evalscope
Issue: 按文档抽取数据集，运行时报错
Root cause: Environment Incompatibility

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述

请简要描述您遇到的问题。
这是文档代码：
from evalscope.collections import CollectionSchema, DatasetInfo, WeightedSampler
from evalscope.utils.io_utils import dump_jsonl_data

schema = CollectionSchema(name='Qwen3', datasets=[
    CollectionSchema(name='English', datasets=[
        DatasetInfo(name='mmlu_pro', weight=1, task_type='exam', tags=['en'], args={'few_shot_num': 0}),
        DatasetInfo(name='mmlu_redux', weight=1, task_type='exam', tags=['en'], args={'few_shot_num': 0}),
        DatasetInfo(name='ifeval', weight=1, task_type='instruction', tags=['en'], args={'few_shot_num': 0}),
    ]),
    CollectionSchema(name='Chinese', datasets=[
        DatasetInfo(name='ceval', weight=1, task_type='exam', tags=['zh'], args={'few_shot_num': 0}),
        DatasetInfo(name='iquiz', weight=1, task_type='exam', tags=['zh'], args={'few_shot_num': 0}),
    ]),
    CollectionSchema(name='Code', datasets=[
        DatasetInfo(name='live_code_bench', weight=1, task_type='code', tags=['en'], args={'few_shot_num': 0, 'subset_list': ['v5_v6'], 'extra_params': {'start_date': '2025-01-01', 'end_date': '2025-04-30'}}),
    ]),
    CollectionSchema(name='Math&Science', datasets=[
        DatasetInfo(name='math_500', weight=1, task_type='math', tags=['en'], args={'few_shot_num': 0}),
        DatasetInfo(name='aime24', weight=1, task_type='math', tags=['en'], args={'few_shot_num': 0}),
        DatasetInfo(name='aime25', weight=1, task_type='math', tags=['en'], args={'few_shot_num': 0}),
        DatasetInfo(name='gpqa_diamond', weight=1, task_type='knowledge', tags=['en'], args={'few_shot_num': 0})
    ])
])

# get the mixed data
mixed_data = WeightedSampler(schema).sample(100000000)  # set a large number to ensure all datasets are sampled
# dump the mixed data to a jsonl file
dump_jsonl_data(mixed_data, 'outputs/qwen3_test.jsonl')
我的代码不同点就是mixed_data = WeightedSampler(schema).sample(500)、DatasetInfo(name='gpqa', weight=1, task_type='knowledge', tags=['en'], args={'few_shot_num': 0})（因为原先运行时报错说gpqa_diamond没有，有gpqa）
抽取数据后，运行报错：
Traceback (most recent call last):
  File "/mnt/workspace/miniconda3/envs/evalscope/bin/evalscope", line 7, in <module>
    sys.exit(run_cmd())
             ^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/cli/cli.py", line 27, in run_cmd
    cmd.execute()
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/cli/start_eval.py", line 32, in execute
    run_task(self.args)
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/run.py", line 28, in run_task
    return run_single_task(task_cfg, run_time)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/run.py", line 41, in run_single_task
    result = evaluate_model(task_cfg, outputs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/run.py", line 144, in evaluate_model
    res_dict = evaluator.eval()
               ^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/evaluator/evaluator.py", line 94, in eval
    dataset_dict = self.benchmark.load_dataset()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 62, in load_dataset
    self.test_dataset, self.fewshot_dataset = self.load()
                                              ^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/benchmarks/data_collection/data_collection_adapter.py", line 64, in load
    ).load()
      ^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/api/dataset/loader.py", line 204, in load
    samples=data_to_samples(data=dataset, data_to_sample=data_to_sample),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/api/dataset/utils.py", line 53, in data_to_samples
    record_samples = as_sample_list(data_to_sample(record=record))
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/benchmarks/data_collection/data_collection_adapter.py", line 83, in record_to_sample
    sample = Sample.model_validate(entry.prompt)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/pydantic/main.py", line 716, in model_validate
    return cls.__pydantic_validator__.validate_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for Sample
input
  Field required [type=missing, input_value={'data': ['The following ...c': 'stemez-Economics'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.12/v/missing
问题是数据格式与 Sample 模型的字段要求不匹配，请问这是版本原因导致的吗？该怎么解决？谢谢！
## EvalScope 版本（必填）
v1.1.0

## 使用的工具
- [ ] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

请提供您执行的主要代码或指令。

## 错误日志

请粘贴完整的错误日志或控制台输出。

## 运行环境

- 操作系统：
- Python版本：

## 其他信息

如果有其他相关信息，请在此处提供。


v1.1.0 版本已经移除了gpqa数据集，更改为了gpqa_diamond，你运行环境的版本是否正确呢

> v1.1.0 版本已经移除了gpqa数据集，更改为了gpqa_diamond，你运行环境的版本是否正确呢

好的，谢谢，这个问题已经解决了，现在有个新问题，我eval跑完后用evalscope app查看时，报错（这个错误我搜过文档，问过大模型，搜过issue，都没找到解决方法，麻烦你帮忙看一下，谢谢！）：
Traceback (most recent call last):
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/gradio/queueing.py", line 624, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/gradio/route_utils.py", line 323, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/gradio/blocks.py", line 2018, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/gradio/blocks.py", line 1567, in call_function
    prediction = await anyio.to_thread.run_sync(  # type: ignore
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2485, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 976, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/gradio/utils.py", line 846, in wrapper
    response = f(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/app/ui/single_model.py", line 126, in update_single_report_score
    report_score_plot = plot_single_report_scores(report_score_df)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/evalscope/app/utils/visualization.py", line 22, in plot_single_report_scores
    plot = px.bar(df, x=df[ReportKey.dataset_name], y=df[ReportKey.score], text=df[ReportKey.score])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/express/_chart_types.py", line 373, in bar
    return make_figure(
           ^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/express/_core.py", line 2115, in make_figure
    apply_default_cascade(args)
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/express/_core.py", line 993, in apply_default_cascade
    args["pattern_shape_sequence"] = [
                                     ^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/express/_core.py", line 994, in <listcomp>
    bar.marker.pattern.shape for bar in args["template"].data.bar
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/graph_objs/bar/marker/_pattern.py", line 252, in shape
    return self["shape"]
           ~~~~^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4742, in __getitem__
    elif self._props is not None and prop in self._props:
         ^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4437, in _props
    return self.parent._get_child_props(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4451, in _get_child_props
    if self._props is None:
       ^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4437, in _props
    return self.parent._get_child_props(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4451, in _get_child_props
    if self._props is None:
       ^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4437, in _props
    return self.parent._get_child_props(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 4471, in _get_child_props
    child_ind = BaseFigure._index_is(children, child)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/workspace/miniconda3/envs/evalscope/lib/python3.11/site-packages/plotly/basedatatypes.py", line 3965, in _index_is
    raise ValueError("Invalid value")
ValueError: Invalid value

你刷新一下gradio网页再试试，有时可能是因为数据没加载完全

> v1.1.0 版本已经移除了gpqa数据集，更改为了gpqa_diamond，你运行环境的版本是否正确呢

请教一下，为啥要移除gpqa，只保留diamond呢？

@Oukaishen 主要是为了对齐主流评测方法，一般只使用 gpqa_diamond ，修改之后无需额外配置