zhiminy/EvalEng huggingface dataset:
Repository: open-compass/opencompass
Issue: Display Estimated Evaluation Time  in Terminal
Root cause: Unimplemented Feature Gap

### 描述该功能

While the predicted jsons are created and saved in the `work dir` folder, there is no estimated time in the terminal.
![pred](https://github.com/open-compass/opencompass/assets/49086900/72e680ca-e6de-4f8c-a7b3-f8e8db11bf15)
![time](https://github.com/open-compass/opencompass/assets/49086900/9a98e766-2dc5-4751-99a8-8450d440447b)



### 是否希望自己实现该功能？

- [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！

Please the check the log in in `logs` folder

It's been 5 months since I last used opencompass, and I need to adapt to changes in the latest version.
Repository: open-compass/opencompass
Issue: [Feature] Add a new evaluation dataset
Root cause: Unimplemented Feature Gap

### Describe the feature

We want to contribute our benchmark to OpenCompass.  Here is the repo:https://github.com/IAAR-Shanghai/UHGEval.
However, there is an issue: in one of the tasks within this benchmark, multiple model calls are required to obtain the final evaluation result for a single evaluation data point. We have observed that most benchmarks typically involve only one model call per data point. Please inform us if OpenCompass supports multiple model calls within a single evaluation.

### Will you implement it?

- [X] I would like to implement this feature and create a PR!

models is a list of dict. You can evalute multiple models with one config

> models is a list of dict. You can evalute multiple models with one config

My expression may not be clear enough. What I mean is that when evaluating a data point, I need to construct multiple prompts and call the model multiple times. For example, in discriminative evaluation, a data point may contain two sentences, one of which contains hallucinated content, and the other does not. I need to call the model twice. The first call is for evaluating whether the model can correctly identify the presence of hallucinated content in the first sentence, i.e., giving the judgment: that the first sentence contains hallucinated content. The second call is for evaluating whether the model can correctly identify the absence of hallucinated content in the second sentence, i.e., giving the judgment: that the second sentence does not contain hallucinated content. The evaluation is considered successful only when both judgments are correct.

The code is as follows:
```
answer_hallu, reason_hallu = self.model.is_continuation_hallucinated(hallu, data_point, with_reason=True)
answer_unhallu, reason_unhallu = self.model.is_continuation_hallucinated(unhallu, data_point, with_reason=True)

```
Each execution of the is_continuation_hallucinated method will call the model once.


请问咱们当前有专一评测大模型翻译能力的bench吗？如何使用？谢谢

@White-Friday Please check Flores. Feel free to re-open if needed.
Repository: open-compass/opencompass
Issue: [Bug] 通过Opencompass跑自己的模型做evaluation，部分Bench没有评测分数计算出现
Root cause: Validation Gap

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我修改了代码（配置不视为代码），或者我正在处理我自己的任务/模型/数据集。

### 环境

官方环境

### 重现问题 - 代码/配置示例

```python
CUDA_VISIBLE_DEVICES=6 python run.py --datasets mmlu_gen cmmlu_gen ceval_gen agieval_gen gsm8k_gen GaokaoBench_gen math_gen \
--hf-path FarReelAILab/Machine_Mindset_en_ISTJ \
--model-kwargs device_map='auto' trust_remote_code=True \
--tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True \
--max-out-len 100 \
--max-seq-len 2048 \
--batch-size 8  \
--num-gpus 1 \
--max-num-workers 32 \
-w outputs/Machine_Mindset_en_ISTJ
```

### 重现问题 - 命令或脚本

```python
CUDA_VISIBLE_DEVICES=6 python run.py --datasets mmlu_gen cmmlu_gen ceval_gen agieval_gen gsm8k_gen GaokaoBench_gen math_gen \
--hf-path FarReelAILab/Machine_Mindset_en_ISTJ \
--model-kwargs device_map='auto' trust_remote_code=True \
--tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True \
--max-out-len 100 \
--max-seq-len 2048 \
--batch-size 8  \
--num-gpus 1 \
--max-num-workers 32 \
-w outputs/Machine_Mindset_en_ISTJ
```

### 重现问题 - 错误信息

![QQ截图20240115165623](https://github.com/open-compass/opencompass/assets/55651568/c25cdcf5-f853-483e-a518-7f799849b162)

以上是一次评测中未出现分数的情况

### 其他信息

有时候跑的所有bench都会有分数

Please check the log in `logs/infer`

thank you, I will check it.
Repository: open-compass/opencompass
Issue: 请问leaderboard中数据集的评测方式是gen还是ppl?
Root cause: Documentation Deficiency

### 描述该功能

部分数据集下有 ‘ppl’ 和 ‘gen’ 两种评估方式，请问在leaderboard中是哪种评估方式？比如mmlu、ceval、gsm8k、hellaswag...

### 是否希望自己实现该功能？

- [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！

Thank you for your question.

Generally,
For base models, we use the `ppl` metric for datasets that can be evaluated with `ppl`, and we use the `gen` metric for other datasets. (The specific list can be found in `configs/datasets/collections/base_medium.py`)
For chat models, we use the `gen` method for all evaluations. (The specific list can be found in `configs/datasets/collections/chat_medium.py`)

However, there are also special cases:
For some chat models, we evaluate using both `ppl` and `gen` methods and take the higher of the two values, the effect of which is equivalent to performing prompt optimization.
For a few other models (such as the llama series), we use separate prompts in some of the datasets.

In the future, we hope to test each model with all evaluation methods/prompt templates, take the highest score, and report which configuration it was achieved with.

非常感谢！

另外，
对于few shot的另外几种方式（random, voteK, topK...）可以增加几个demo实现吗？

That seems like a different issue, we will update the progress in https://github.com/InternLM/opencompass/issues/83

Closing this issue, feel free to reopen if you have more questions.

> 非常感谢！
> 
> 另外， 对于few shot的另外几种方式（random, voteK, topK...）可以增加几个demo实现吗？

Also thanks for the suggestions and welcome to contribute the implementations of these settings.
Repository: open-compass/opencompass
Issue: [Feature] Can we find the config version that is used to produce the score displayed on leaderboard?
Root cause: Documentation Deficiency

### Describe the feature

Hi, thanks for sharing this wonderful project! 
I am wondering is there somewhere we can find the config version that is used to produce the score displayed on leaderboard?
For example, there are config files from different versions under configs/datasets/ARC_c. How should I choose the right version to reproduce the displayed results on leaderboard?

![image](https://github.com/open-compass/opencompass/assets/47023705/5eb319a9-7a04-4bdf-98be-54c74bb12023)

Any method to obtain the right version information would be appreciated!

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!
Repository: open-compass/opencompass
Issue: [Bug] The case in Figure 11 is not in the MMBench?
Root cause: Documentation Deficiency

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

The same as the official

### Reproduces the problem - code/configuration sample

question_file = 'mmbench_dev_20230712.tsv'
question_df = pd.read_csv(question_file, sep='\t', encoding='utf-8')
physical_relation = question_df[question_df['category']=='physical_relation']

### Reproduces the problem - command or script

NO

### Reproduces the problem - error message

I iterate the physical_relation and did not find the data displayed in Figure 11.
<img width="1046" alt="image" src="https://github.com/open-compass/opencompass/assets/55051961/dfec0ae3-5e49-48f3-b561-c018846ff5ec">


### Other information

No

@Richar-Du , the two cases do exist in MMBench, but in the `test` split. That's why you can not find them in the train split.
Repository: open-compass/opencompass
Issue: [Feature] Add few-shot demo by selecting sample in Random / TopK / VoteK
Root cause: Unimplemented Feature Gap

### Describe the feature

As proposed in https://github.com/InternLM/opencompass/issues/80#issuecomment-1644917593

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!
Repository: open-compass/opencompass
Issue: [Details Correction] Shikra is using Vicuna-7/13B but not LLaMA 7B
Root cause: Documentation Deficiency

A minor error: Table 15 in the paper and table in the [benchmark](https://opencompass.org.cn/leaderboard-multimodal) show that Shikra used LLaMA 7B as the language backbone. However, it should be Vicuna-7/13B according to the [paper](https://[arxiv.org/pdf/2306.15195.pdf](https://arxiv.org/pdf/2306.15195.pdf)).


Thank you for your suggestion. We will fix it as soon as possible.
Repository: open-compass/opencompass
Issue: [Bug] FileNotFoundError: Couldn't find a module script at /mnt/workspace/xxx/evaluation/opencompass/accuracy/accuracy.py. Module 'accuracy' doesn't exist on the Hugging Face Hub either.
Root cause: Environment Incompatibility

### 描述该错误

Hello, I encountered the following problems when evaluating a custom model on the Alibaba Cloud server, how to solve it?

FileNotFoundError: Couldn't find a module script at /mnt/workspace/xxx/evaluation/opencompass/accuracy/accuracy.py. Module 'accuracy' doesn't exist on the Hugging Face Hub either.



### 环境信息

python run.py ./config/eval_my_model.py --debug

### 其他信息

_No response_

This is usually due to network issues, which cause HuggingFace's `datasets.load_metric()` to be unable to connect to the Internet and locate the module. The solution is to set up a proxy and direct the `http_proxy` and `https_proxy` in the environment variables to the proxy address.

> This is usually due to network issues, which cause HuggingFace's `datasets.load_metric()` to be unable to connect to the Internet and locate the module. The solution is to set up a proxy and direct the `http_proxy` and `https_proxy` in the environment variables to the proxy address.

Yes, thanks

> This is usually due to network issues, which cause HuggingFace's `datasets.load_metric()` to be unable to connect to the Internet and locate the module. The solution is to set up a proxy and direct the `http_proxy` and `https_proxy` in the environment variables to the proxy address.

如果服务器不能上网，这个问题该如何解决呢？
多谢。。
Repository: open-compass/opencompass
Issue: [doc] Link check failed 

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

-

### 重现问题 - 代码/配置示例

[[-](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606)](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606)

### 重现问题 - 命令或脚本

-

### 重现问题 - 错误信息

昨天的link check结果，感觉有部分适合修一下，列了几个：
```
Name       `OpenCompass Open Platform Service Agreement'

Parent URL https://opencompass.readthedocs.io/en/latest/, line 559, col 7
Real URL   https://opencompass.readthedocs.io/policy

Parent URL https://opencompass.readthedocs.io/zh-cn/latest/advanced_guides/code_eval_service.html, line 451, col 427
[741](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606#step:2:742)Real URL   https://opencompass.readthedocs.io/zh-cn/latest/get_started.html#id7

Parent URL https://opencompass.readthedocs.io/zh-cn/latest/index.html, line 240, col 24
[526](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606#step:2:527)Real URL   https://opencompass.readthedocs.io/zh-cn/latest/advanced_guides/custom_dataset.html

Parent URL https://opencompass.readthedocs.io/zh-cn/latest/index.html, line 239, col 24
[514](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606#step:2:515)Real URL   https://opencompass.readthedocs.io/zh-cn/latest/advanced_guides/new_dataset.html

Parent URL https://opencompass.readthedocs.io/zh-cn/latest/index.html, line 235, col 24
[502](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606#step:2:503)Real URL   https://opencompass.readthedocs.io/zh-cn/latest/prompt/chain_of_thought.html
Parent URL https://opencompass.readthedocs.io/zh-cn/latest/index.html, line 228, col 24
[454](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606#step:2:455)Real URL   https://opencompass.readthedocs.io/zh-cn/latest/user_guides/summarizer.html
Parent URL https://opencompass.readthedocs.io/zh-cn/latest/index.html, line 234, col 24
[490](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606#step:2:491)Real URL   https://opencompass.readthedocs.io/zh-cn/latest/prompt/meta_template.html
```

### 其他信息

_No response_

Feel free to reopen it if needed
Repository: open-compass/opencompass
Issue: [Feature] 历史 repo 和版本太多，希望增加对 lmdeploy+puyu+gRPC 接口支持
Root cause: External Dependency Breakage

### Describe the feature

之前给的 pjeval/add_grpc_support 现在没法跑了。需要
* llama_service 切 v0.4.3
* unpack 也报错还得修

```bash
  File "/workspace/GitProjects/pjeval/opencompass/models/grpc_api.py", line 108, in _generate
    for status, result, tokens in self.grpcapi.generate(
TypeError: cannot unpack non-iterable StatusCode object
```
* 然后发现是 llama_service v0.4.3 版本似乎不对...彻底没法维护了。

现在 lmdeploy 里面就有 llama_service，干脆适配一次 lmdeploy，废弃掉内部的吧。
需要用这个来测非对称量化版本精度。

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!
Repository: open-compass/opencompass
Issue: [Bug] 
Root cause: Unimplemented Feature Gap

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

*

### 重现问题 - 代码/配置示例

from mmengine.config import read_base

with read_base():
    from .datasets.gsm8k_contamination.gsm8k_contamination_ppl_ecdd22 import gsm8k_datasets  # includes training, test, and reference sets

datasets = sum((v for k, v in locals().items() if k.endswith('_datasets')), [])

### 重现问题 - 命令或脚本

*

### 重现问题 - 错误信息

KeyError: 'opencompass.datasets.GSM8KReferenceSkywork is not in the opencompass::load_dataset registry. Please check whether the value of `opencompass.datasets.GSM8KReferenceSkywork` is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'

### 其他信息

参考文档https://opencompass.readthedocs.io/en/latest/advanced_guides/contamination_eval.html跑[mock_gsm8k_test](https://huggingface.co/datasets/Skywork/mock_gsm8k_test)数据集报错

Kindly verify that opencompass has been installed correctly on your system.

> Kindly verify that opencompass has been installed correctly on your system.

正确安装了，但是并没有找到opencompass.datasets.GSM8KReferenceSkywork文件

是需要自己实现GSM8KReferenceSkywork这个类吗

GSM8KReferenceSkywork这个文件是不是没有传上来？
Repository: open-compass/opencompass
Issue: [Bug] 2.0 Rankings, Hover score cannot view configuration items.
Root cause: Documentation Deficiency

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

```no run```

### Reproduces the problem - code/configuration sample

```no run```

### Reproduces the problem - command or script

```no run```

### Reproduces the problem - error message

```no run```

### Other information

no run

![image](https://github.com/open-compass/opencompass/assets/46083912/73fd6c5d-5e00-4664-933a-1d3e222ee0c9)
Repository: open-compass/opencompass
Issue: [Feature] 如何在此项目上测试openai接口部署的模型在数据集上的表现 
Root cause: Documentation Deficiency

### Describe the feature

在configs/api_examples上没找到openai接口的py文件，请问有类似的py文件吗？

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!

https://github.com/open-compass/opencompass/blob/main/configs/eval_gpt4.py
Repository: open-compass/opencompass
Issue: [Feature] Support for Multi-Turn Inference Dataset
Root cause: Documentation Deficiency

### Describe the feature

I am currently working with a dataset designed for multi-turn model inference, and I need to concatenate the inference result of the first model as the prompt for the subsequent round of inference. For instance, the prompt for the first round of dialogue is: A=2, B=3, what is A+B?
The model's inference result is: 5.
For the second round of dialogue, I append the following query: If B=5, what would be the result of the above?

Now, the complete prompt appears as:
Human: A=2, B=3, what is A+B?
Assistant: 5
Human: If B=5, what would be the result of the above?

Does the current system support this type of inference mode?

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!

We have a plan for multi-turn and interactive evaluation. Can you formulate this idea into the multi-round ChatML format?

> We have a plan for multi-turn and interactive evaluation. Can you formulate this idea into the multi-round ChatML format?

Thank you for your prompt response! I am not entirely sure what you mean by the multi-round ChatML format that you mentioned. To clarify, I am using an example code from the documentation:

Suppose I have a multi-turn dialogue dataset, where one sample of the dataset is as follows:
```python
{
    "question1": "A=2, B=3, what is A+B?",
    "answer1": "5",
    "question2": "If B=5, what would be the result of the above?",
    "answer2": "7"
}
``` 

Additionally, there is a `prompt_template` defined as:
```python
prompt_template=dict(
    type=PromptTemplate,
    template=dict(
        round=[
            dict(role="HUMAN", prompt="Question: {question1}"),
            dict(role="BOT", prompt="Answer: {answer1}"),
            dict(role="HUMAN", prompt="Question: {question2}"),
            dict(role="BOT", prompt="Answer: {answer2}"),
        ]
    )
)
```

In this case, the model's first received prompt would be: `"Question: A=2, B=3, what is A+B?\nAnswer:"`. At this point, the model generates an inference, which we'll assume to be `"A+B=5"`. This allows us to calculate the accuracy of the first round of dialogue.
In the second round of dialogue, the model receives the prompt: `"Question: A=2, B=3, what is A+B?\nAnswer: A+B=5\nQuestion: If B=5, what would be the result of the above?\nAnswer:"`. Subsequently, the model's output can be used to calculate the accuracy of the second round of results.

I hope this aligns with what you're looking for in the multi-round ChatML format and that it helps you.

We have already incorporated the evaluation for multi-turn dialogue datasets right now, and you can refer to the specific configuration in mtbench https://github.com/open-compass/opencompass/blob/main/configs/eval_subjective_mtbench.py

feel free to reopen it if needed
Repository: open-compass/opencompass
Issue: [Bug] Official website ranking page, unable to view configuration items when hovering over scores with mouse
Root cause: Documentation Deficiency

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

no run

### Reproduces the problem - code/configuration sample

no run

### Reproduces the problem - command or script

no run

### Reproduces the problem - error message

no run

### Other information

1. page: https://rank.opencompass.org.cn/leaderboard-llm-v2
2. hover
![image](https://github.com/open-compass/opencompass/assets/46083912/769ac4b8-2488-4f00-ac09-92f4d1d6898c)

Expected: Display and view configuration, can click to jump to the configuration page.
Actual: The mouse hovers over the score without any response



Thank you for bringing this to our attention. We recently underwent a comprehensive website upgrade, and we are aware of the hover issue. Our team is actively working on resolving this matter, and we anticipate completion of the fix within the next 2-3 weeks.


[送你小红花]




> Thank you for bringing this to our attention. We recently underwent a comprehensive website upgrade, and we are aware of the hover issue. Our team is actively working on resolving this matter, and we anticipate completion of the fix within the next 2-3 weeks.

Hi, I was wondering when the fix will be completed ?
Repository: open-compass/opencompass
Issue: huggingface数据集离线加载
Root cause: Documentation Deficiency

### 描述该功能

在加载huggingface上的数据集时，需要服务器联网去huggingface上下载；
而我的服务器不能上网，所以在执行：
from datasets import load_dataset
dataset = load_dataset(**kwargs)
会发生ConnectTimeout错误；

能否提供一种离线的数据加载方式？
或者如何修改代码更加容易从本地disk加载数据集？

请给点建议，非常感谢。

### 是否希望自己实现该功能？

- [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！

First, manually download all the huggingface dependency to where it should be (e.g. `~/.cache/huggingface`). Then you may set the following env variables and start OpenCompass scripts.

```shell
export HF_DATASETS_OFFLINE=1
export TRANSFORMERS_OFFLINE=1
export HF_EVALUATE_OFFLINE=1
```

> First, manually download all the huggingface dependency to where it should be (e.g. `~/.cache/huggingface`). Then you may set the following env variables and start OpenCompass scripts.
> 
> ```shell
> export HF_DATASETS_OFFLINE=1
> export TRANSFORMERS_OFFLINE=1
> export HF_EVALUATE_OFFLINE=1
> ```
手工下载huggingface的数据集的方式有哪些，目前我知道的一种方法是：采用能联网的电脑，使用load_dataset 函数，从网络上load，然后再save_to_disk(my_dataset_dir)，磁盘上应该就会有了；
还有其他方式吗？感谢~

To my knowledge, once `load_dataset` is executed, the dataset has already been cached to the disk. There is no need to `save_to_disk` again. Still, you can download the files in one machine and copy them to another machine in any way possible (e.g. in-cluster networks / USB flask disk / mount disk). And that is all I know.
I am not sure whether this [post](https://mp.weixin.qq.com/s/e_Krti4U7TxPuFWmUep80Q) would be helpful to you. 

> To my knowledge, once `load_dataset` is executed, the dataset has already been cached to the disk. There is no need to `save_to_disk` again. Still, you can download the files in one machine and copy them to another machine in any way possible (e.g. in-cluster networks / USB flask disk / mount disk). And that is all I know. I am not sure whether this [post](https://mp.weixin.qq.com/s/e_Krti4U7TxPuFWmUep80Q) would be helpful to you.

赞，多谢
Repository: open-compass/opencompass
Issue: [Bug] 运行成功后在summary文件夹中的结果都是为空的
Root cause: Configuration Error

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

python run.py configs/eval_gpt3.5.py -w outputs/demo

### 重现问题 - 代码/配置示例

python run.py configs/eval_gpt3.5.py -w outputs/demo

### 重现问题 - 命令或脚本

python run.py configs/eval_gpt3.5.py -w outputs/demo

### 重现问题 - 错误信息

没有收到错误信息，运行ceval数据集测评openai接口的模型，所有infer都成功了，在results也能看到accuracy，但是summary里的结果为空。

如何调整summary的结果展示，比如c-eval的所有目录准确度合并在一起，同时并增加定制数据集的结果？

### 其他信息

_No response_

我遇到了相同的问题请问您是否已将问题解决

可以把summary的调用注释掉就能解决

您好您可以具体说一下是哪个summary吗，能否发一个代码片段