Repository: modelscope/evalscope
Issue: omni_doc_bench数据测评报错
Root cause: Validation Gap

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述

- 模型：Qwen3-VL-30B-A3B-Instruct-FP8
- 任务：omni_doc_bench精度测评
- 现象： 数据集报错

## EvalScope 版本（必填）
Version: 0.0.0.dev0

## 使用的工具
- [x] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

```python
from evalscope import TaskConfig, run_task
from evalscope.constants import EvalType, JudgeStrategy

task_cfg = TaskConfig(
    model='Qwen3-VL-30B-A3B-Instruct-FP8',
    api_url='http://0.0.0.0:8090/v1',
    api_key='API_KEY',
    eval_type=EvalType.SERVICE,
    datasets=[
        'omni_doc_bench', # 异常
    ],

    eval_batch_size=8,
    generation_config={
        'max_tokens': 32768,  # 最大生成token数，建议设置为较大值避免输出截断
        'temperature': 0.7,  # 采样温度 (qwen 报告推荐值)
        'top_p': 0.8,  # top-p采样 (qwen 报告推荐值)
        'top_k': 20,  # top-k采样 (qwen 报告推荐值)
        'repetition_penalty': 1.0,  # 重复惩罚系数
        'presence_penalty': 1.5,  # 新内容惩罚系数
        'n': 1,  # 每个请求产生的回复数量
    },
    # limit=100,  # 设置为100条数据进行测试
    # use_cache='outputs/20251210_223837'
)

run_task(task_cfg=task_cfg)
```


## 错误日志

```python
2025-12-13 11:44:37 - evalscope - INFO: Start loading benchmark dataset: omni_doc_bench
Processing records:   0%|                                                                                                           | 0/4905 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/cx8k/fs101/wzp/code/0929/evalscope/examples/collection/qwen3_vl_vision.py", line 81, in <module>
    run_task(task_cfg=task_cfg)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/run.py", line 28, in run_task
    return run_single_task(task_cfg, run_time)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/run.py", line 42, in run_single_task
    result = evaluate_model(task_cfg, outputs)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/run.py", line 147, in evaluate_model
    res_dict = evaluator.eval()
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/evaluator/evaluator.py", line 96, in eval
    dataset_dict = self.benchmark.load_dataset()
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 62, in load_dataset
    self.test_dataset, self.fewshot_dataset = self.load()
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 81, in load
    return self.load_from_remote()
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 89, in load_from_remote
    test_dataset = self.load_subsets(test_load_func)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 227, in load_subsets
    subset_data = load_func(subset)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/api/benchmark/adapters/default_data_adapter.py", line 262, in load_subset
    dataset = loader.load()
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/api/dataset/loader.py", line 162, in load
    samples=data_to_samples(data=dataset, data_to_sample=data_to_sample),
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/api/dataset/utils.py", line 53, in data_to_samples
    record_samples = as_sample_list(data_to_sample(record=record))
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/site-packages/evalscope/benchmarks/omnidoc_bench/omnidoc_bench_adapter.py", line 101, in record_to_sample
    input=[ChatMessageUser(content=content_list)], target='', metadata=ast.literal_eval(record['answer'])
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 110, in literal_eval
    return _convert(node_or_string)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 99, in _convert
    return dict(zip(map(_convert, node.keys),
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 90, in _convert
    return list(map(_convert, node.elts))
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 99, in _convert
    return dict(zip(map(_convert, node.keys),
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 109, in _convert
    return _convert_signed_num(node)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 83, in _convert_signed_num
    return _convert_num(node)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 74, in _convert_num
    _raise_malformed_node(node)
  File "/cx8k/fs101/wzp/miniforge3/envs/evalscope/lib/python3.10/ast.py", line 71, in _raise_malformed_node
    raise ValueError(msg + f': {node!r}')
ValueError: malformed node or string on line 1: <ast.Name object at 0x7f13923f7ac0>
```

## 运行环境

- 操作系统：ubuntu-22.04
- Python版本：python-3.10

## 其他信息
```
Package                           Version
--------------------------------- -------------
absl-py                           2.3.1
accelerate                        1.12.0
addict                            2.4.0
aiofiles                          23.2.1
aiohappyeyeballs                  2.6.1
aiohttp                           3.13.2
aiosignal                         1.4.0
annotated-doc                     0.0.4
annotated-types                   0.7.0
anthropic                         0.71.0
antlr4-python3-runtime            4.9.3
anyio                             4.12.0
apache-tvm-ffi                    0.1.5
apted                             1.0.3
astor                             0.8.1
asttokens                         3.0.1
async-timeout                     5.0.1
attrs                             25.4.0
av                                16.0.1
beautifulsoup4                    4.14.3
blake3                            1.0.8
bleu                              0.3
bs4                               0.0.2
cachetools                        6.2.2
cbor2                             5.7.1
certifi                           2025.11.12
cffi                              2.0.0
charset-normalizer                3.4.4
click                             8.2.1
cloudpickle                       3.1.2
colorama                          0.4.6
colorlog                          6.10.1
compressed-tensors                0.12.2
contourpy                         1.3.2
cuda-bindings                     13.1.1
cuda-pathfinder                   1.3.3
cuda-python                       13.1.1
cupy-cuda12x                      13.6.0
cycler                            0.12.1
datasets                          3.5.0
decorator                         5.2.1
depyf                             0.20.0
dill                              0.3.8
diskcache                         5.6.3
Distance                          0.1.3
distro                            1.9.0
dnspython                         2.8.0
docstring_parser                  0.17.0
dotenv                            0.9.9
editdistance                      0.8.1
efficiency                        2.0
einops                            0.8.1
email-validator                   2.3.0
et_xmlfile                        2.0.0
evalscope                         0.0.0.dev0
evaluate                          0.4.6
exceptiongroup                    1.3.1
executing                         2.2.1
fastapi                           0.124.0
fastapi-cli                       0.0.16
fastapi-cloud-cli                 0.6.0
fastar                            0.8.0
fastrlock                         0.8.3
ffmpy                             1.0.0
filelock                          3.20.0
flashinfer-python                 0.5.2
fonttools                         4.61.0
frozendict                        2.4.7
frozenlist                        1.8.0
fsspec                            2024.12.0
func_timeout                      4.3.5
gguf                              0.17.1
google-auth                       2.43.0
google-genai                      1.54.0
gradio                            5.4.0
gradio_client                     1.4.2
h11                               0.16.0
hf-xet                            1.2.0
httpcore                          1.0.9
httptools                         0.7.1
httpx                             0.28.1
huggingface-hub                   0.36.0
idna                              3.11
ImageIO                           2.37.2
interegular                       0.3.3
ipdb                              0.13.13
ipython                           8.37.0
jedi                              0.19.2
jieba                             0.42.1
Jinja2                            3.1.6
jiter                             0.12.0
jmespath                          1.0.1
joblib                            1.5.2
json_repair                       0.54.2
jsonlines                         4.0.0
jsonschema                        4.25.1
jsonschema-specifications         2025.9.1
kiwisolver                        1.4.9
langdetect                        1.0.9
lark                              1.2.2
latex2sympy2                      1.9.1
latex2sympy2_extended             1.10.2
Levenshtein                       0.27.3
llguidance                        1.3.0
llvmlite                          0.44.0
lm-format-enforcer                0.11.3
loguru                            0.7.3
lxml                              6.0.2
markdown-it-py                    4.0.0
MarkupSafe                        2.1.5
math-verify                       0.8.0
matplotlib                        3.10.7
matplotlib-inline                 0.2.1
mdurl                             0.1.2
mistral_common                    1.8.6
model-hosting-container-standards 0.1.11
modelscope                        1.32.0
mpmath                            1.3.0
ms-vlmeval                        0.0.19
msgpack                           1.1.2
msgspec                           0.20.0
multidict                         6.7.0
multiprocess                      0.70.16
networkx                          3.4.2
ninja                             1.13.0
nltk                              3.9.2
numba                             0.61.2
numpy                             2.2.6
nvidia-cublas-cu12                12.8.4.1
nvidia-cuda-cupti-cu12            12.8.90
nvidia-cuda-nvrtc-cu12            12.8.93
nvidia-cuda-runtime-cu12          12.8.90
nvidia-cudnn-cu12                 9.10.2.21
nvidia-cudnn-frontend             1.16.0
nvidia-cufft-cu12                 11.3.3.83
nvidia-cufile-cu12                1.13.1.3
nvidia-curand-cu12                10.3.9.90
nvidia-cusolver-cu12              11.7.3.90
nvidia-cusparse-cu12              12.5.8.93
nvidia-cusparselt-cu12            0.7.1
nvidia-cutlass-dsl                4.3.3
nvidia-ml-py                      13.590.44
nvidia-nccl-cu12                  2.27.5
nvidia-nvjitlink-cu12             12.8.93
nvidia-nvshmem-cu12               3.3.20
nvidia-nvtx-cu12                  12.8.90
openai                            2.9.0
openai-harmony                    0.0.8
opencv-python                     4.12.0.88
opencv-python-headless            4.12.0.88
openpyxl                          3.1.5
orjson                            3.11.5
outlines_core                     0.2.11
overrides                         7.7.0
packaging                         25.0
pandas                            2.3.3
parso                             0.8.5
partial-json-parser               0.2.1.1.post7
pexpect                           4.9.0
pillow                            11.3.0
pip                               25.3
plotly                            5.24.1
Polygon3                          3.0.9.1
portalocker                       3.2.0
prometheus_client                 0.23.1
prometheus-fastapi-instrumentator 7.1.0
prompt_toolkit                    3.0.52
propcache                         0.4.1
protobuf                          6.33.2
psutil                            7.1.3
ptyprocess                        0.7.0
pure_eval                         0.2.3
py-cpuinfo                        9.0.0
pyarrow                           22.0.0
pyasn1                            0.6.1
pyasn1_modules                    0.4.2
pybase64                          1.4.3
pycountry                         24.6.1
pycparser                         2.23
pydantic                          2.12.5
pydantic_core                     2.41.5
pydantic-extra-types              2.10.6
pydub                             0.25.1
Pygments                          2.19.2
pylatexenc                        2.10
pyparsing                         3.2.5
python-dateutil                   2.9.0.post0
python-dotenv                     1.2.1
python-json-logger                4.0.0
python-multipart                  0.0.20
pytz                              2025.2
PyYAML                            6.0.3
pyzmq                             27.1.0
qwen-vl-utils                     0.0.14
RapidFuzz                         3.14.3
ray                               2.52.1
referencing                       0.37.0
regex                             2025.11.3
requests                          2.32.5
rich                              14.2.0
rich-toolkit                      0.17.0
rignore                           0.7.6
rouge-chinese                     1.0.3
rouge_score                       0.1.2
rpds-py                           0.30.0
rsa                               4.9.1
ruff                              0.14.8
sacrebleu                         2.5.1
safehttpx                         0.1.7
safetensors                       0.7.0
scikit-learn                      1.7.2
scipy                             1.15.3
seaborn                           0.13.2
semantic-version                  2.10.0
sentencepiece                     0.2.1
sentry-sdk                        2.47.0
setproctitle                      1.3.7
setuptools                        80.9.0
shellingham                       1.5.4
simplejson                        3.20.2
six                               1.17.0
sniffio                           1.3.1
sortedcontainers                  2.4.0
soundfile                         0.13.1
soupsieve                         2.8
soxr                              1.0.0
sse-starlette                     3.0.3
stack-data                        0.6.3
starlette                         0.50.0
sty                               1.0.6
supervisor                        4.3.0
sympy                             1.14.0
tabulate                          0.9.0
tenacity                          9.1.2
threadpoolctl                     3.6.0
tiktoken                          0.12.0
timeout-decorator                 0.5.0
timm                              1.0.22
tokenizers                        0.22.1
tomli                             2.3.0
tomlkit                           0.12.0
torch                             2.9.0
torchaudio                        2.9.0
torchvision                       0.24.0
tqdm                              4.67.1
traitlets                         5.14.3
transformers                      4.57.3
triton                            3.5.0
typer                             0.20.0
typing_extensions                 4.15.0
typing-inspection                 0.4.2
tzdata                            2025.2
urllib3                           2.6.1
uvicorn                           0.38.0
uvloop                            0.22.1
validators                        0.35.0
vllm                              0.11.2
watchfiles                        1.1.1
wcwidth                           0.2.14
websockets                        15.0.1
wheel                             0.45.1
word2number                       1.1
xformers                          0.0.33.post1
xgrammar                          0.1.25
xlsxwriter                        3.2.9
xxhash                            3.6.0
yarl                              1.22.0
zss                               1.2.0
```

如果有其他相关信息，请在此处提供。


- `ast.literal_eval(record['answer']` 在处理true、false、null等数据时有问题
- `json.loads(record['answer'])`，是正常的