zhiminy/EvalEng huggingface dataset:
Repository: open-compass/VLMEvalKit
Issue: Which JudgeModel is it?
Root cause: Documentation Deficiency

Which evaluation sets are scored based on rules?
Which evaluation sets are scored based on GPT, and which specific version of GPT is it?
<img width="3592" height="994" alt="Image" src="https://github.com/user-attachments/assets/d4c8db9c-36be-4b9b-8464-edec8795ea8c" />

@kennymckormick 

> [@kennymckormick](https://github.com/kennymckormick)

Hi, @bobo0810 , 

For MathVista, we adopt gpt-4o-mini as the **judge model**;
For MMVet, we adopt gpt-4-turbo-1106 as the **judge model**;
For other MCQ benchmarks, we adopt gpt-3.5-turbo-0125 as the **choice extractor** when heuristic matching fails. 

For more information, you can resort to `run.py` and find related information, starting from: https://github.com/open-compass/VLMEvalKit/blob/88047e87019ae78bc39a0b5aba18d8b66e1031f3/run.py#L398

@kennymckormick Thank you  for your reply. However, OpenAI has already removed the old models. How can we solve this problem?

> [@kennymckormick](https://github.com/kennymckormick) Thank you for your reply. However, OpenAI has already removed the old models. How can we solve this problem?


Currently, you can replace the old models with newer ones, like gpt-4o-mini or gpt-4.1. However, you should note that those evaluation results may not be comparable with ones reported by VLMEvalKit. We consider to replace the legacy evaluation results by ones produced by newer judges (maybe opensource ones), however, that work is cumbersome and we currently have no time for it.

Since old users can still call the legacy APIs, currently the problem has not affected us. If you consider to switch to a different judge, we have released most of our evaluation records here, and you only need to re-run evaluation with a newer judge: https://huggingface.co/datasets/VLMEval/OpenVLMRecords

请问如何使用Qwen2.5-vl-7b-instruct 作为judge model? 应该在哪里配置？？
Repository: open-compass/VLMEvalKit
Issue: Gemma 3 models should be loaded in bf16
Root cause: Configuration Error

Trying to launch any benchmark with Gemma 3 will give an incompatibility error between bf32 and flash attention.
```
ValueError: Flash Attention 2.0 only supports torch.float16 and torch.bfloat16 dtypes. You passed torch.float32, this might lead to unexpected behaviour.
```
This is fixed by adding `torch_dtype=torch.bfloat16` to the model loading line of `gemma.py`. 
I'm considering making a PR about this, but  scores were visibly computed at least once using default (bf32) settings. How did that first execution run ?
Repository: open-compass/VLMEvalKit
Issue: 为什么出现这个错误:assert hasattr(model, 'chat_inner')
Root cause: Validation Gap

<img width="1003" height="524" alt="Image" src="https://github.com/user-attachments/assets/1cbccbbf-58e8-4401-99f3-29bc13ea560a" />
我在推理其他数据集的时候都正常，但是MMDU这个数据集会出现这个错误，请问一下为什么出现这个错误assert hasattr(model, 'chat_inner')呢

我也遇到了相同的问题 请问您解决了吗
Repository: open-compass/VLMEvalKit
Issue: Unable to reproduce any of the Gemma 3 results
Root cause: Configuration Error

Hi,

I'm working on re creating some gemma 3 benchmark results. But the scores I get are always **very** far from the papers or the VLMEvalKit website. It's almost as if the chat template is wrong or something. Example : Running the ChartQA_TEST dataset on Gemma3-4B gives an overall score of 7.2 instead of 30-ish (Also, the paper announced around 70 for this test so this is another mystery). Same for the other datasets I've tried.

Can anyone re-create any gemma 3 results ?
Repository: open-compass/VLMEvalKit
Issue: Wrong download link for R-Bench
Root cause: Configuration Error

The download inks in vlmeval/dataset/image_mcq.py for R-Bench are wrong, they download an html page instead of tsv file:
```      
# R-Bench
'R-Bench-Dis': 'https://huggingface.co/datasets/lcysyzxdxc/R-Bench/blob/main/R-bench-dis.tsv',
'R-Bench-Ref': 'https://huggingface.co/datasets/lcysyzxdxc/R-Bench/blob/main/R-bench-ref.tsv',
```   
The correct link should be 
```   
# Correct R-Bench
'R-Bench-Dis': 'https://huggingface.co/datasets/lcysyzxdxc/R-Bench/resolve/main/R-bench-dis.tsv',
'R-Bench-Ref': 'https://huggingface.co/datasets/lcysyzxdxc/R-Bench/resolve/main/R-bench-ref.tsv',
```
Repository: open-compass/VLMEvalKit
Issue: NameError: name 'init_empty_weights' is not defined
Root cause: External Dependency Breakage

Hi, I tried to run `python run.py --data OCRBench --model Qwen2.5-VL-7B-Instruct --verbose`.

I meet the error below. How should I fix it? 
```
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
[2025-07-10 19:16:41] ERROR - RUN - run.py: main - 505: Model Qwen2.5-VL-7B-Instruct x Dataset OCRBench combination failed: name 'init_empty_weights' is not defined, skipping this combination.
Traceback (most recent call last):
  File "/data0/user/ycliu/work3/VLMEvalKit/run.py", line 373, in main
    model = infer_data_job(
            ^^^^^^^^^^^^^^^
  File "/data0/user/ycliu/work3/VLMEvalKit/vlmeval/inference.py", line 203, in infer_data_job
    model = infer_data(
            ^^^^^^^^^^^
  File "/data0/user/ycliu/work3/VLMEvalKit/vlmeval/inference.py", line 123, in infer_data
    model = supported_VLM[model_name](**kwargs) if isinstance(model, str) else model
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data0/user/ycliu/work3/VLMEvalKit/vlmeval/vlm/qwen2_vl/model.py", line 290, in __init__
    self.model = MODEL_CLS.from_pretrained(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data0/user/ycliu/.conda/envs/vlmkit/lib/python3.12/site-packages/transformers/modeling_utils.py", line 279, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data0/user/ycliu/.conda/envs/vlmkit/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4333, in from_pretrained
    model_init_context = cls.get_init_context(is_quantized, _is_ds_init_called)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data0/user/ycliu/.conda/envs/vlmkit/lib/python3.12/site-packages/transformers/modeling_utils.py", line 3736, in get_init_context
    init_contexts = [no_init_weights(), init_empty_weights()]
                                        ^^^^^^^^^^^^^^^^^^
NameError: name 'init_empty_weights' is not defined
```

It's a problem about transfromers, I upated it to 4.51.1. It works.
Repository: open-compass/VLMEvalKit
Issue: 救! help! AssertionError: Current model not supported pack mode!
Root cause: Documentation Deficiency

I run `python run.py --data MMBench_Video_1fps_pack --model Qwen2-VL-2B-Instruct`,  but raised:
```
Traceback (most recent call last):
  File "/workspace/codes/VLMEvalKit/run.py", line 353, in main
    model = infer_data_job_video(
  File "/workspace/codes/VLMEvalKit/vlmeval/inference_video.py", line 213, in infer_data_job_video
    model = infer_data(
  File "/workspace/codes/VLMEvalKit/vlmeval/inference_video.py", line 116, in infer_data
    assert not getattr(dataset, 'pack', False), 'Current model not supported pack mode!'
AssertionError: Current model not supported pack mode!
```
thanks

so sorry to bother, I change the `MMBench_Video_1fps_pack` to `MMBench_Video_8fps_nopack`, then solved.
我以后会仔细看文档的, 抱歉
Repository: open-compass/VLMEvalKit
Issue: 是否支持部署模型，再通过接入api服务的形式测评？或者是否支持NPU环境测评？
Root cause: Documentation Deficiency


支持的，可以用vllm-ascend部署好，然后走api的形式评测

> 支持的，可以用vllm-ascend部署好，然后走api的形式足球

大佬，有没有代码示例

> > 支持的，可以用vllm-ascend部署好，然后走api的形式足球
> 
> 大佬，有没有代码示例

vllm_ascend：https://github.com/vllm-project/vllm-ascend.git （it just like normal vllm serve）
看这里 https://github.com/open-compass/VLMEvalKit/blob/main/docs/zh-CN/Quickstart.md 配置好你的base url 和 api_key 跑测试就好了


npu环境跑视频的精度测试时，会报错找不到libGl.so.1，请问有遇到吗，怎么解决的呢


> libGl.so.1

`apt install libgl1-mesa-glx`试试
Repository: open-compass/VLMEvalKit
Issue: 请问代码对哪些安装包的版本是强依赖的？
Root cause: Environment Incompatibility

保持相同的代码，在不同的环境中，保持transformer和tokenizers版本相同，评测在mmbench数据集上的性能，发现结果差异较大。
env1: cn39.3；env2:cn26.9；
还请解答一下谢谢！

主要包括 torch, transformers 的版本，在你的 case 中，两个环境差异在哪些库和版本呢？

> 主要包括 torch, transformers 的版本，在你的 case 中，两个环境差异在哪些库和版本呢？

torch版本是2.1.2，cuda版本分别是12.1和11.8，感觉主要是torch+cuda版本差异的问题导致了输出差异
Repository: open-compass/VLMEvalKit
Issue: Bug Regarding CRPE Evaluation Implementation in VLMEvalKit
Root cause: Algorithmic Error

Hi, @kennymckormick, @Quinn777, @FangXinyu-0913, @ttguoguo3 and @PhoenixZ810.

Thank you for your great work on `VLMEvalKit`!  

However, I find a bug about the `CRPE` dataset evaluation implementation in the repository. It seems [the current code](https://github.com/open-compass/VLMEvalKit/blob/6ffc898fdd267804d2ad7b7642e2aa24dd024d95/vlmeval/dataset/image_vqa.py#L1687
) does not align with the methodology described in the `CRPE` paper. Specifically, the **CircularEval** procedure, which is a key feature of `CRPE`, appears to be missing. According to the `CRPE` protocol, for a question to be considered correctly answered, the model must choose the correct option across **four different permutations of the answer choices**. However, this characteristic is not implemented in the current evaluation code. For reference, you can look at [the official CRPE evaluation implementation](https://github.com/OpenGVLab/all-seeing/blob/main/all-seeing-v2/llava/eval/eval_crpe.py).

Additionally, since `InternVL3` utilized the evaluation code from this repository, the reported `CRPE` metrics in its paper might be overestimated. If possible, it would be ideal to update the evaluation implementation in `VLMEvalKit` and revise the reported results in the `InternVL3` paper to provide accurate benchmarks, so that future works do not reference incorrect data.

Looking forward to your updates on this!

Thank you!

Thank you for your attention to this project!
In fact, the operation of CircularEval is given in the question file, not in the implementation code. If you look at the TSV file, you will find that.

Thank you for your reply. 

But what I mean is not that the question was not asked four times, but that when calculating the correctness of the question, the four responses were not combined to be considered. Current codes only calculate the correctness of each individual response. Please pay attention to the `merged_v` in the official evaluation code, which implements the rule that only when all four responses are correct can it be considered correct.

Yes, you are right. I’ll fix it soon.

Thanks!
Repository: open-compass/VLMEvalKit
Issue: Dataset  GMAI-MMBench_VAL is not working
Root cause: Configuration Error

!python run.py --data GMAI-MMBench_VAL --model idefics_9b_instruct --verbose. The result is [2025-07-21 11:59:53] ERROR - misc.py: load_env - 215: Did not detect the .env file at /content/VLMEvalKit/.env, failed to load. 
2025-07-21 11:59:57.664789: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1753099197.685422   19582 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1753099197.691648   19582 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
[2025-07-21 12:00:01] ERROR - misc.py: load_env - 215: Did not detect the .env file at /content/VLMEvalKit/.env, failed to load. 
[2025-07-21 12:00:01] WARNING - RUN - run.py: main - 216: --reuse is not set, will not reuse previous (before one day) temporary files
[2025-07-21 12:00:01] ERROR - RUN - run.py: main - 472: Model idefics_9b_instruct x Dataset GMAI-MMBench_VAL combination failed: [Errno 2] No such file or directory: '/root/LMUData/GMAI-MMBench_VAL.tsv', skipping this combination.
Traceback (most recent call last):
  File "/content/VLMEvalKit/run.py", line 295, in main
    dataset = build_dataset(dataset_name, **dataset_kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/content/VLMEvalKit/vlmeval/dataset/__init__.py", line 279, in build_dataset
    return cls(dataset=dataset_name, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/content/VLMEvalKit/vlmeval/dataset/image_base.py", line 45, in __init__
    data = self.load_data(dataset)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/content/VLMEvalKit/vlmeval/dataset/image_mcq.py", line 630, in load_data
    if file_size(data_path, 'GB') > 1:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/content/VLMEvalKit/vlmeval/smp/file.py", line 346, in file_size
    stats = os.stat(f)
            ^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/root/LMUData/GMAI-MMBench_VAL.tsv'
Can any one help?
Repository: open-compass/VLMEvalKit
Issue: 评测InternVL3-8B报错, AttributeError: 'InternVLModel' object has no attribute 'chat'
Root cause: Environment Incompatibility

报错命令如下
```
Traceback (most recent call last):
  File "/nas/user/projects/VLMEvalKit/run.py", line 376, in main
    model = infer_data_job(
            ^^^^^^^^^^^^^^^
  File "/nas/user/projects/VLMEvalKit/vlmeval/inference.py", line 203, in infer_data_job
    model = infer_data(
            ^^^^^^^^^^^
  File "/nas/user/projects/VLMEvalKit/vlmeval/inference.py", line 166, in infer_data
    response = model.generate(message=struct, dataset=dataset_name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nas/user/projects/VLMEvalKit/vlmeval/vlm/base.py", line 116, in generate
    return self.generate_inner(message, dataset)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nas/user/projects/VLMEvalKit/vlmeval/vlm/internvl/internvl_chat.py", line 467, in generate_inner
    return self.generate_v2(message, dataset)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/nas/user/projects/VLMEvalKit/vlmeval/vlm/internvl/internvl_chat.py", line 422, in generate_v2
    response = self.model.chat(
               ^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1931, in __getattr__
    raise AttributeError(
AttributeError: 'InternVLModel' object has no attribute 'chat'
```
执行的命令为
```
python run.py --data ChartQA_TEST --model InternVL3-8B --work-dir results/ --judge gpt-4o --mode all  --reuse
```

Hi, please check your transformers version and make sure the  version number is at least 4.46 for internvl3

hi, have you solved this problem?
Repository: open-compass/VLMEvalKit
Issue: Provide faster evaluation for vlmevalkit
Root cause: Unimplemented Feature Gap

The current model evaluation speed is greatly limited, especially the implementation method of multi-card vllm evaluation. The current VLLM implementation method actually only starts the model in a tensor parallel way, but the evaluation is essentially carried out serially (since it can only be started in python) The multi-process data splitting processing under torchrun was not even effective at all (the data was tested one by one), which completely failed to take advantage of the powerful concurrent performance of the vllm inference server. This is not conducive to use in the context of an increasing number of R1 paradigm inference models

I think we should refer to the practice of MLLM RL training, provide a new evaluation mode (or use parameters compatible with the previous evaluation mode), deploy the evaluation model as a separate evaluation server, and conduct the evaluation entirely through API calls. At the same time, the concurrency of the API can be controlled through the api_nproc parameter. This enables rapid testing, and the solution can be easily compatible with inference servers such as sglang

In fact, I have already completed a simple version of this idea by myself, supporting qwenvl2.5-7B and an inference model that we have not yet disclosed. If the author is interested, I can propose a PR to try to merge it into the repository. However, the workload of fully supporting all current models is quite large, which requires refactoring an interface for all models. Process the prompt and package it as a standard openai API request

I also welcome someone to work together on the refactoring. The specific implementation plan may need further discussion

Hi! I’m also trying to use vLLM with an API deployment to speed up evaluation. Would you share the code modifications you have made? That would be extremely helpful！Thank you!
Repository: open-compass/VLMEvalKit
Issue: OutOfMemory issue with Qwen2.5-VL (3B, 7B)
Root cause: Resource Mishandling

When running:

```
python3 run.py --data CCOCR_MultiSceneOcr_Cord --model ${MODEL_NAME} --work-dir ${SUB_OUTPUT_DIR} --verbose
```

It **fails with the OutOfMemory issue**:

```
"File "/opt/conda/envs/cc-ocr-py310/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 185, in eager_attention_forward
    attn_weights = torch.matmul(query, key_states.transpose(2, 3)) * scaling
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 28.43 GiB. GPU 0 has a total capacity of 15.77 GiB of which 13.38 GiB is free. Including non-PyTorch memory, this process has 2.38 GiB memory in use. Of the allocated memory 2.00 GiB is allocated by PyTorch, and 9.25 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)"
```

In model.py inside the class Qwen2VLChat I have:

```
self.model = MODEL_CLS.from_pretrained(
                model_path, torch_dtype='auto', device_map="auto", attn_implementation='eager', load_in_8bit=True, low_cpu_mem_usage=True, max_memory={0: "15GiB", 1: "15GiB", 2: "15GiB", 3: "15GiB", "cpu": "40GiB"}
            )
```
Using eager attention, as flash attention could not be installed due to a system-level incompatibility with GLIBC (current version = GLIBC 2.31, required: 2.32)

Tried:
```
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
```

Machine config: 4 x NVIDIA V100.

Any suggestions are highly appreciated!!

```python
self.model = MODEL_CLS.from_pretrained(
                model_path, torch_dtype='auto', , attn_implementation='eager', load_in_8bit=True, low_cpu_mem_usage=True, max_memory={0: "15GiB", 1: "15GiB", 2: "15GiB", 3: "15GiB", "cpu": "40GiB"}
            )
```

from the error message, maybe you should adjust `torch_dtype='auto'` to `torch_dtype=torch.float16`+`device_map="auto"` to `device_map="balanced"`.

if it is still not work, try to use 4bit?
Repository: open-compass/VLMEvalKit
Issue: [BUG] dataset parameter not passed when using config.json with MMMUProDataset
Root cause: Configuration Error

## 🐛 Bug Report: `dataset` parameter not passed when using `config.json` with `MMMUProDataset`

### What's happening?

When trying to load any benchmark from `MMMUProDataset` via `config.json`, the dataset fails to build correctly because the `dataset` parameter is not passed to the class constructor.

This happens due to how `build_dataset_from_config()` filters parameters based on the class constructor signature using Python's `inspect` module.

---

### ✅ Works (using command-line args)

```bash
python3 run.py --data MMMU_Pro_10c --model Qwen2-VL-2B-Instruct --verbose
```

---

### ❌ Fails (using config file)

```bash
python3 run.py --config config.json
```

#### Example `config.json`:

```json
{
  "model": {
    "Qwen2-VL-2B-Instruct": {
      "class": "Qwen2VLChat",
      "model_path": "Qwen/Qwen2-VL-2B-Instruct",
      "temperature": 0.1
    }
  },
  "data": {
    "MMMU": {
      "class": "MMMUProDataset",
      "dataset": "MMMU_Pro_10c"
    }
  }
}
```

---

### 📌 Root Cause

- The function `build_dataset_from_config()` uses this logic:

  ```python
  valid_params = {k: v for k, v in config.items() if k in sig.parameters}
  ```

- `MMMUProDataset.__init__()` is defined with `**kwargs`:

  ```python
  def __init__(self, **kwargs):
      super().__init__(**kwargs)
  ```

- Because of this, `inspect.signature(cls.__init__)` returns:

  ```python
  Signature(self, **kwargs)
  ```

- It does **not** include `dataset`, even though the superclass (`ImageBaseDataset`) accepts `dataset` explicitly:

  ```python
  def __init__(self, dataset='MMBench', skip_noimg=True):
      self.dataset_name = dataset
  ```

- So the filtering logic incorrectly drops the `dataset` key from `config`.

---

### ✅ Why `build_dataset()` works

In contrast, `build_dataset()` passes arguments directly without filtering:

```python
return cls(dataset=dataset_name, **kwargs)
```

It doesn't rely on `inspect.signature`, so `dataset` is passed correctly and everything works.

---

### 🔁 Suggested Fix

Update the logic in `build_dataset_from_config()` to detect when a class uses `**kwargs`, and skip filtering in that case:

```python
if any(p.kind == p.VAR_KEYWORD for p in sig.parameters.values()):
    valid_params = config
else:
    valid_params = {k: v for k, v in config.items() if k in sig.parameters}
```

Alternatively, traverse the class's MRO to find the first constructor with concrete parameters (not just `**kwargs`).

---

### ✅ Expected Behavior

Running with `--config config.json` should work identically to the command-line approach, even for classes that inherit `dataset` via `super().__init__(**kwargs)`.

---

I also encountered this problem and hope the author fixes it as soon as possible.

Cool!
Repository: open-compass/VLMEvalKit
Issue: mvbench bug
Root cause: Algorithmic Error

https://github.com/open-compass/VLMEvalKit/blob/76d6258c63323db464a0cc20da2e653becb610cc/vlmeval/dataset/mvbench.py#L410

这里应该是 data.loc[data['index'] == idx, 'score'] = int(check_ans_with_model ?

跑mvbench前十个子集的时候发现出现NaN的类别，看着像这行的问题
Repository: open-compass/VLMEvalKit
Issue: ImportError: cannot import name 'auto_split_flag' from 'vlmeval.smp'(/root/autodl-tmp/EVAL_BASE_MODELS/VLMEvalKit/vlmeval/smp/__init__.py)
Root cause: Unimplemented Feature Gap

是代码版本更更新了吗，auto_split_flag()在哪里定义的呀

我们已经不再使用 AUTO_SPLIT 这个选项。目前，利用 torchrun 启动时，可以设置 nproc-per-node 参数，随后，会自动为每 NGPU // nproc-per-node 个 GPU 分配一个模型实例
Repository: open-compass/VLMEvalKit
Issue: Can not download MM-Math and DynaMath  and  Math-V dataset from the default links
Root cause: Configuration Error

DynaMath: https://opencompass.openxlab.space/utils/VLMEval/DynaMath.tsv
Exception:  https://opencompass.openxlab.space/utils/benchmarks/MMBench/MMBench_TEST_EN_V11.tsv

Hello, the links are not working for downloading. Could you please update them?

According to my test, the download link of DynaMath is good. 
Currently we do not provide the access to MMBench_TEST_EN_V11.  

> According to my test, the download link of DynaMath is good. Currently we do not provide the access to MMBench_TEST_EN_V11.

<img width="1595" height="710" alt="Image" src="https://github.com/user-attachments/assets/5498927d-89c2-4dea-83bb-e97e0cc36e19" />
I am unable to access the following link directly via my browser. Could you kindly share the download process with me? Additionally, the issue also affects datasets such as Math-V, HallBench, and MathVerse. I would appreciate any assistance you can provide.