zhiminy/EvalEng huggingface dataset:
Repository: open-compass/opencompass
Issue: [Bug] 使用Slurm运行时报错OpenICLInfer[mymodel_roberta/siqa] failed with code 127
Root cause: Environment Incompatibility

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我修改了代码（配置不视为代码），或者我正在处理我自己的任务/模型/数据集。

### 环境

pip install -e.

### 重现问题 - 代码/配置示例

infer = dict(
    partitioner=dict(type=NaivePartitioner),  # 评测任务只有这一种
    runner=dict(
        type=SlurmRunner,
        task=dict(type=OpenICLInferTask),  # 待运行任务
        max_num_workers=16,  # 最大同时评测任务数
        retry=2,  # 任务失败的重试次数，可以避免意外发生的错误
    )
)

eval = dict(
    partitioner=dict(type=NaivePartitioner),

    runner=dict(
        type=SlurmRunner,
        task=dict(type=OpenICLEvalTask),  # 待运行任务
        max_num_workers=16,  # 最大同时评测任务数
        retry=2,  # 任务失败的重试次数，可以避免意外发生的错误
    )
)

### 重现问题 - 命令或脚本

python run.py configs/mymodel_demo.py -w outputs/mymodel_1213_Slurm

### 重现问题 - 错误信息

12/13 01:45:41 - OpenCompass - ERROR - /workspace/opencompass-main/opencompass/runners/base.py - summarize - 63 - OpenICLEval[mymodel_roberta/siqa] failed with code 127

### 其他信息

_No response_

Please check the output log in your `outputs/mymodel_1213_Slurm` folder, which saves the error message

Feel free to re-open if needed.
Repository: open-compass/opencompass
Issue: [Feature] How to download the entire leaderboard?
Root cause: Unimplemented Feature Gap

### Describe the feature

Including all the evaluation results shown on the webpage?

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!

We have updated our website to show the entire leaderboard, feel free to reopen it if needed

> We have updated our website to show the entire leaderboard, feel free to reopen it if needed

Thanks, @bittersweet1999 But I still cannot find the button to download the leaderboard directly from your website. Can you give me the linkage? 

sorry for misunderstanding your aim, do you means download the whole leaderboard and for example to save as csv file?

> sorry for misunderstanding your aim, do you means download the whole leaderboard and for example to save as csv file?

Yeah, do you provide such functionality?

@tonysy It seems like this is a small requirement that can be implemented relatively quickly on the front end.
Repository: open-compass/opencompass
Issue: [Bug] Why CEval-test dataset test result is 0?
Root cause: Validation Gap

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

Test Model: Llama2-7b-hf model
Baseline OpenCompass v0.2.0

### Reproduces the problem - code/configuration sample

Just use the official cmd:
```
python run.py --datasets ceval_ppl_93e5ce \
              --hf-path /Llama-2-7b-hf \
              --model-kwargs device_map='auto' \
              --max-seq-len 4096 \
              --batch-size 1 \
              --num-gpus 1 \
              --max-partition-size 3000 \
              --max-workers-per-gpu 2
```

Notice: dev package test results are all right but test package get accuracy 0.
![图片](https://github.com/open-compass/opencompass/assets/10757117/45cd899a-54b6-4251-9cfe-1e22c955a123)


### Reproduces the problem - command or script

Just use the official cmd:
python run.py --datasets ceval_ppl_93e5ce \
              --hf-path $MODEL_PATH \
              --model-kwargs device_map='auto' \
              --max-seq-len 4096 \
              --batch-size 1 \
              --num-gpus 1 \
              --max-partition-size 3000 \
              --max-workers-per-gpu 2

### Reproduces the problem - error message

No error message:
20231213_211334_ceval_llama2-7b-hf_1213/logs/eval/opencompass.models.huggingface.HuggingFace_LLM_Llama-2-7b-hf# cat ceval-test-accountant.out
12/13 22:10:48 - OpenCompass - INFO - Task [opencompass.models.huggingface.HuggingFace_LLM_Llama-2-7b-hf/ceval-test-accountant]: {'accuracy': 0.0}
12/13 22:10:48 - OpenCompass - INFO - time elapsed: 1.72s



### Other information

_No response_

Answer of test split is not released. Please submit the prediction on the C-Eval leaderboard.
Repository: open-compass/opencompass
Issue: [Feature] Needle In A Haystack - Pressure Testing LLMs LongContext
Root cause: Unimplemented Feature Gap

### 描述该功能

在长文本评测中，“大海捞针” 压力测试是一个非常不错的评测方式：
[视频](https://www.youtube.com/watch?v=KwRRuiCCdmc)
[Github](https://github.com/gkamradt/LLMTest_NeedleInAHaystack/tree/main)

不过看上去，评测的表现形式不太容易在 opencompass 中实现。

### 是否希望自己实现该功能？

- [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！

Thanks for the insightful suggestions, we are working on this feature and it will be supported within this week.

#714
Repository: open-compass/opencompass
Issue: [Bug] Why different models using different test method some PPL some Gen?
Root cause: Documentation Deficiency

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

The same environment.
This is just a question about the opencompass leader board.

### Reproduces the problem - code/configuration sample

Visit https://opencompass.org.cn/leaderboard-llm
We can see many models test results there.
Whe different models using different test method some PPL some Gen?

Example 1:
For chatglm3-6b-base uses "generate" method to evalute ceval, mmlu, cmmlu, WiC, OCNLI, but [LLaMA-2-7B](https://opencompass.org.cn/model-detail/LLaMA-2-7B) uses "PPL" method.

Example 2:
For CEval datasets, different models uses different test method:
[Qwen-14B](https://opencompass.org.cn/model-detail/Qwen-14B) uses PPL method
[Yi-6B](https://opencompass.org.cn/model-detail/Yi-6B) uses gen method
[InternLM-20B](https://opencompass.org.cn/model-detail/InternLM-20B) uses PPL method

Will this lead to unfair evaluation? Why?

Thanks











### Reproduces the problem - command or script

Just visit the OpenCompass official website https://opencompass.org.cn/leaderboard-llm

### Reproduces the problem - error message

There is no message about this.

### Other information

_No response_

https://github.com/open-compass/opencompass/discussions/646

@tonysy  Thanks for your feedback.
I've read that link and got that for multiple-choice case, PPL can represent the model most ability but can't represent the instruction-following ability.

But for the opencompass leaderbord, what are the criteria for selecting evaluation methods PPL or Gen? Or just test two methods and got two scores just upload the higher one as the official public one?

Thanks


Typically,  Base with PPL, Chat with Gen

Feel free to re-open if needed.

> Typically, Base with PPL, Chat with Gen

[LLaMA-2-70B](https://rank.opencompass.org.cn/model-detail/LLaMA-2-70B)
[Qwen-72B](https://rank.opencompass.org.cn/model-detail/Qwen-72B)
May I ask why these models use gen on the C3 and OCNLI datasets.
Repository: open-compass/opencompass
Issue: [Feature] Some suggestion about design

### 描述该功能

https://github.com/open-compass/opencompass/blob/637628a70fc708057cfd6dfe8717ca9035553bc8/opencompass/tasks/openicl_eval.py#L127-L149

这一段的逻辑是不是可以放在 https://github.com/open-compass/opencompass/blob/97c2068bd9b21ac2b30177db6531554f4695bc51/opencompass/models/base.py#L132 里？

`_extract_role_pred` 看上去将Chat模型的回答中提取出 begin_token 与 end_token 中间的部分，放在**模型中似乎更合理**。

'pred_role' 看上去只是指示使用 meta_tmplate中的哪一个角色的begin_token\end_token，本质上还是使用最后一段话，我认为不如直接约定为 'BOT' 或者 ‘assistant’的begin_token\end_token。


考虑到的点：
1. openicl_eval 里的这段逻辑有些奇怪，放在model里面合理很多。
2. predction中不会存在特殊的toekn
3. chatinferencer 使用时不用在infer中间去除这些特殊的token


### 是否希望自己实现该功能？

- [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！
Repository: open-compass/opencompass
Issue: [Feature] Evaluation detail results of humaneval and evalplus
Root cause: Unimplemented Feature Gap

### Describe the feature

- https://github.com/open-compass/opencompass/pull/720#issuecomment-1863958692
@jingmingzhuo 

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!
Repository: open-compass/opencompass
Issue: [Feature] ModuleNotFoundError: No module named 'opencompass.datasets.lawbench.evaluation_functions'
Root cause: Unimplemented Feature Gap

### Describe the feature


我执行命令python run.py --datasets ceval_ppl mmlu_ppl --hf-path /T106/LLM_model/llama-7b --model-kwargs device_map='auto' --tokenizer-kwargs padding_side='left' truncation='left' use_fast=False --max-out-len 100 --max-seq-len 2048 --batch-size 8 --no-batch-padding --num-gpus 1
...................................
 98%|████████████████████████████████████████████████████████████████████████████████████████████▎ | 107/109 [04:13<00:01,  1.99it/s]12/25 12:59:23 - OpenCompass - WARNING - task OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_conceptual_physics] fail, see
./outputs/default/20231225_125408/logs/eval/opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_conceptual_physics.out
12/25 12:59:23 - OpenCompass - WARNING - task OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_us_foreign_policy] fail, see
./outputs/default/20231225_125408/logs/eval/opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_us_foreign_policy.out
100%|██████████████████████████████████████████████████████████████████████████████████████████████| 109/109 [04:13<00:00,  2.33s/it]
12/25 12:59:23 - OpenCompass - ERROR - /T106/model_main/opencompass-main/opencompass-main/opencompass/runners/base.py - summarize - 63 - OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/ceval-computer_network] failed with code 1
12/25 12:59:23 - OpenCompass - ERROR - /T106/model_main/opencompass-main/opencompass-main/opencompass/runners/base.py - summarize - 63 - OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/ceval-operating_system] failed with code 1
...................
...................
...................
12/25 12:59:23 - OpenCompass - ERROR - /T106/model_main/opencompass-main/opencompass-main/opencompass/runners/base.py - summarize - 63 - OpenICLEval[opencompass.models.huggingface.HuggingFace_LLM_model_llama-7b/lukaemon_mmlu_conceptual_physics] failed with code 1
dataset      
...............................
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/conda/envs/mixtralkit/lib/python3.10/site-packages/opencompass/datasets/__init__.py", line 51, in <module>
    from .lawbench import *  # noqa: F401, F403
  File "/opt/conda/envs/mixtralkit/lib/python3.10/site-packages/opencompass/datasets/lawbench/__init__.py", line 1, in <module>
    from .lawbench import LawBenchDataset  # noqa: F401
  File "/opt/conda/envs/mixtralkit/lib/python3.10/site-packages/opencompass/datasets/lawbench/lawbench.py", line 10, in <module>
    from .evaluation_functions import (cjft, flzx, ftcs, jdzy, jec_ac, jec_kd,
ModuleNotFoundError: No module named 'opencompass.datasets.lawbench.evaluation_functions'

### Will you implement it?



在跑mixtral-8x7b-v0.1的时候遇到同样的问题，想问下预计什么时候支持呢？

same
Repository: open-compass/opencompass
Issue: [Bug] ValueError: The provided pattern matches 0 or more than one config.
Root cause: Validation Gap

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

官方推荐环境

### 重现问题 - 代码/配置示例

```python
python run.py --datasets mmlu cmmlu ceval AGIEval humaneval GAOKAO-BENCH ARC BBH gsm8k \
--hf-path FarReelAILab/Machine_Mindset_en_ENFP \
--model-kwargs device_map='auto' trust_remote_code=True \
--tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True \
--max-out-len 100 \
--max-seq-len 2048 \
--batch-size 128  \
--num-gpus 4 \
--max-num-workers 32 \
-w outputs/Machine_Mindset_en_ENFP
```

### 重现问题 - 命令或脚本

```python
python run.py --datasets mmlu cmmlu ceval AGIEval humaneval GAOKAO-BENCH ARC BBH gsm8k \
--hf-path FarReelAILab/Machine_Mindset_en_ENFP \
--model-kwargs device_map='auto' trust_remote_code=True \
--tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True \
--max-out-len 100 \
--max-seq-len 2048 \
--batch-size 128  \
--num-gpus 4 \
--max-num-workers 32 \
-w outputs/Machine_Mindset_en_ENFP
```

### 重现问题 - 错误信息

```
Traceback (most recent call last):
  File "/root/eval_mbti/opencompass/run.py", line 342, in <module>
    main()
  File "/root/eval_mbti/opencompass/run.py", line 209, in main
    cfg = get_config_from_arg(args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/eval_mbti/opencompass/opencompass/utils/run.py", line 65, in get_config_from_arg
    for dataset in match_cfg_file(datasets_dir, args.datasets):
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/eval_mbti/opencompass/opencompass/utils/run.py", line 46, in match_cfg_file
    raise ValueError(err_msg)
ValueError: The provided pattern matches 0 or more than one config. Please verify your pattern and try again. You may use tools/list_configs.py to list or locate the configurations.
+------------------------+
| Not matched patterns   |
|------------------------|
| mmlu                   |
| cmmlu                  |
| ceval                  |
| AGIEval                |
| humaneval              |
| GAOKAO-BENCH           |
| ARC                    |
| BBH                    |
| gsm8k                  |
+------------------------+
```

### 其他信息

官网说：
```
对于 HuggingFace 模型，用户可以通过命令行直接设置模型参数，无需额外的配置文件。例如，对于 facebook/opt-125m 模型，您可以使用以下命令进行评估：
```

https://opencompass.readthedocs.io/zh-cn/latest/get_started/quick_start.html#id3
Repository: open-compass/opencompass
Issue: [Bug] kwargs: stopping_criteria']  没有起作用
Root cause: Configuration Error

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I have modified the code (config is not considered code), or I'm working on my own tasks/models/datasets.

### Environment

{'CUDA available': True,
...
 'NVCC': 'Cuda compilation tools, release 11.7, V11.7.99',
 'OpenCV': '4.8.1',
 'PyTorch': '1.13.1+cu117',

### Reproduces the problem - code/configuration sample

evaluation/opencompass/configs/datasets/gsm8k/gsm8k_gen_1dce88.py:
    retriever=dict(type=ZeroRetriever),
    inferencer=dict(type=GenInferencer, max_out_len=512, stopping_criteria=[":", "Question:", "Question"]))

### Reproduces the problem - command or script

stopping_criteria does not work
the prediction of gsm8k inference still include many "prediction"

### Reproduces the problem - error message

stopping_criteria does not work

### Other information

_No response_

soloved, only for single generation (or --no-batch-padding )

btw，微信群好像加不进去了
Repository: open-compass/opencompass
Issue: FileNotFoundError: Couldn't find a module script at /data/opencompass/bleurt/bleurt.py. Module 'bleurt' doesn't exist on the Hugging Face Hub either.[Bug] 
Root cause: External Dependency Breakage

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

1

### Reproduces the problem - code/configuration sample

1

### Reproduces the problem - command or script

1

### Reproduces the problem - error message

Traceback (most recent call last):
  File "/data/opencompass/opencompass/tasks/openicl_eval.py", line 356, in <module>
    inferencer.run()
  File "/data/opencompass/opencompass/tasks/openicl_eval.py", line 76, in run
    self._score()
  File "/data/opencompass/opencompass/tasks/openicl_eval.py", line 185, in _score
    result = icl_evaluator.score(**preds)
  File "/data/opencompass/opencompass/datasets/truthfulqa.py", line 127, in score
    results = self.basic_score(predictions, references)
  File "/data/opencompass/opencompass/datasets/truthfulqa.py", line 134, in basic_score
    metrics = {key: evaluate.load(key) for key in self.metrics}
  File "/data/opencompass/opencompass/datasets/truthfulqa.py", line 134, in <dictcomp>
    metrics = {key: evaluate.load(key) for key in self.metrics}
  File "/data/Anaconda3/envs/opencompass/lib/python3.10/site-packages/evaluate/loading.py", line 748, in load
    evaluation_module = evaluation_module_factory(
  File "/data/Anaconda3/envs/opencompass/lib/python3.10/site-packages/evaluate/loading.py", line 681, in evaluation_module_factory
    raise FileNotFoundError(


### Other information

happen when evaluate truthfulqa

您好，这个问题最后怎么解决的呢？

Thank you for your email !您好，你的邮件我已收到，我会尽快给你回复。
Repository: open-compass/opencompass
Issue: [Feature]  Incorrect config file link on leaderboard.
Root cause: Configuration Error

### Describe the feature

On the [LLM leaderoard](https://opencompass.org.cn/leaderboard-llm), some scores are linked to incorrect config file. Just click the button for "View the configuration file for this score" and confirm this. Specifically, 
1. All model's RTE score is linked to a "SuperGLUE_AX_g_xxx_xxxxxx".
2. Qwen-7B's CMNLI score is linked to "CLUE_ocnli_gen_51e956", not a "CLUE_cmnli_xxx_xxxxxx".

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!

Thanks for the notification. We will fix it recently.

Feel free to reopen it if needed
Repository: open-compass/opencompass
Issue: [Bug] Unexpected version in summary, when using summarizers.leaderboard, for some datasets (chid, race, csl, eprstmt).
Root cause: Configuration Error

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [ ] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

opencompass verison: v0.2.0
data source: OpenCompassData-complete-20231110


### Reproduces the problem - code/configuration sample

Part of my config file (only show the datasets related to this bug):
```
from mmengine.config import read_base

with read_base():
    from .summarizers.leaderboard import summarizer

    from .datasets.FewCLUE_chid.FewCLUE_chid_ppl_8f2872 import chid_datasets
    from .datasets.race.race_ppl_5831a0 import race_datasets
    from .datasets.FewCLUE_csl.FewCLUE_csl_ppl_841b62 import csl_datasets
    from .datasets.FewCLUE_eprstmt.FewCLUE_eprstmt_ppl_f1e631 import eprstmt_datasets

datasets = sum((v for k, v in locals().items() if k.endswith('_datasets')), [])
```

### Reproduces the problem - command or script

python run.py configs/myeval.py

### Reproduces the problem - error message

No error message.

### Other information

Part of the summary_[timestamp.txt] (only show the datasets related to this bug):
```
dataset                                 version    metric            mode  
--------------------------------------  ---------  ----------------  ------
--------- 考试 Exam ---------           -          -                 -      

--------- 语言 Language ---------       -          -                 -      
chid-dev                                90451d     accuracy          ppl    
--------- 知识 Knowledge ---------      -          -                 -      

--------- 理解 Understanding ---------  -          -                 -      
race-middle                             dda65f     accuracy          ppl    
race-high                               dda65f     accuracy          ppl    
csl_dev                                 46f772     accuracy          ppl    
eprstmt-dev                             ca49a2     accuracy          ppl    
--------- 推理 Reasoning ---------      -          -                 -     
```

These datasets's version in summary is not same as their version used in the config file, while other datasets's version is.
And the versions above are not appear in the corresponding configs/datasets/[dataset], for example, there is no "race_ppl_5831a0.py" in "configs/datasets/race".
Repository: open-compass/opencompass
Issue: [documentation] 希望增加一个完整链路级别的文档指导如何测评一个指定模型
Root cause: Documentation Deficiency

### 描述该错误

https://opencompass.readthedocs.io/en/latest/advanced_guides/new_model.html
这个文档中已经有关于如何新增model支持的描述，但是对于如何完成评测一个新增model，这部分文档还比较分散，比如需要：
1. 新增model类的几个方法的实现（以api为例）
2. 新增对应config
希望文档能有一个整体性的描述，对小白用户比较友好

### 环境信息

无

### 其他信息

_No response_
Repository: open-compass/opencompass
Issue: Multi-Model dataset evaluation
Root cause: Validation Gap

### Describe the feature

How can I test the Multi-Model dataset such as okvqa? I saw the config in `opencompass/configs/multimodal/minigpt_4/minigpt_4_7b_ok_vqa.py`, but I do not know how to use it. And I can't see the config_base, either, when I run `tools/list_config.py`


I found the run_file example in the `opencompass/configs/multimodal/instructblip/README.md`.
`test.py` Code:
```python
from mmengine.config import read_base

from .qwen.qwenvl_chat_7b_ok_vqa import (qwen_okvqa_dataloader,
                                             qwen_okvqa_evaluator,
                                             qwen_okvqa_load_from,
                                             qwen_okvqa_model)
models = [qwen_okvqa_model]
datasets = [qwen_okvqa_dataloader]
evaluators = [qwen_okvqa_evaluator]
load_froms = [qwen_okvqa_load_from]

num_gpus = 2
num_procs = 8
launcher = 'pytorch' 
```
### Evaluation
```sh
python run.py configs/multimodal/tasks.py --mm-eval 
```
But I get `TypeError: argument of type 'LazyObject' is not iterable`:
```sh
Traceback (most recent call last):
  File "/code/Prompt-Ada/opencompass/run.py", line 357, in <module>
    main()
  File "/code/Prompt-Ada/opencompass/run.py", line 224, in main
    cfg = get_config_from_arg(args)
  File "/code/Prompt-Ada/opencompass/opencompass/utils/run.py", line 62, in get_config_from_arg
    if 'type' not in dataset:
TypeError: argument of type 'LazyObject' is not iterable
```

I found it useless that 
```python
    parser.add_argument('--mm-eval',
                        help='Whether or not enable multimodal evaluation',
                        action='store_true',
                        default=False)
```
Can you tell me how to edit this?

Please try https://github.com/open-compass/VLMEvalKit