Repository: open-compass/opencompass
Issue: [Bug] 运行ceval无法结果可视化
Root cause: Environment Incompatibility

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

```python
无其他描述
```


### 重现问题 - 代码/配置示例

```
from mmengine.config import read_base

with read_base():
    from .datasets.ceval.ceval_gen import ceval_datasets
    # from .datasets.winograd.winograd_ppl import winograd_datasets
    from .models.baichuan.hf_baichuan2_13b_chat import bc_models
    # from .models.opt.hf_opt_350m import opt350m

datasets = [*ceval_datasets]
models = [bc_models]
```

### 重现问题 - 命令或脚本

python run.py configs/eval_baichuan2.py
执行完上述命令生成了json文件，随后一直卡着，没有任何动静。
100%|██████████| 3/3 [00:04<00:00,  1.45s/it]
09/11 16:25:47 - OpenCompass - INFO - Start inferencing [baichuan2-13b-chat-hf/ceval-discrete_mathematics]
100%|██████████| 16/16 [00:00<00:00, 272800.26it/s]
[2023-09-11 16:25:47,902] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
100%|██████████| 2/2 [00:02<00:00,  1.23s/it]
09/11 16:25:53 - OpenCompass - INFO - Start inferencing [baichuan2-13b-chat-hf/ceval-middle_school_geography]
100%|██████████| 12/12 [00:00<00:00, 255490.60it/s]
[2023-09-11 16:25:53,163] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
100%|██████████| 2/2 [00:01<00:00,  1.09it/s]
09/11 16:25:55 - OpenCompass - INFO - time elapsed: 557.28s

Ctrl+C退出命令后再执行下列命令，全是空。
python run.py configs/eval_baichuan2.py -r --mode viz

09/11 16:28:22 - OpenCompass - INFO - Reusing experiements from 20230911_161623
dataset                                         version    metric    mode    baichuan2-13b-chat-hf
----------------------------------------------  ---------  --------  ------  -----------------------
ceval-computer_network                          -          -         -       -
ceval-operating_system                          -          -         -       -
ceval-computer_architecture                     -          -         -       -
ceval-college_programming                       -          -         -       -
ceval-college_physics                           -          -         -       -
ceval-college_chemistry                         -          -         -       -
ceval-advanced_mathematics                      -          -         -       -
....

### 重现问题 - 错误信息

执行第一行命令推理完为什么会卡住？这是主要原因，为什么不会生成可视化结果

### 其他信息

无

Try to run

```bash
python run.py configs/eval_baichuan2.py -r --mode eval
```

before you run

```bash
python run.py configs/eval_baichuan2.py -r --mode viz
```

As for the stuck after the inference process, are all 52 subtask jsons all gererated, or only part of the jsons are generated?

```python
python run.py configs/eval_baichuan2.py -r --mode eval
```
执行上述命令出现以下结果
launch OpenICLEval[baichuan2-13b-chat-hf/ceval-teacher_qualification] on CPU                                                                                                                                                 
launch OpenICLEval[baichuan2-13b-chat-hf/ceval-high_school_politics] on CPU                                                                                                                                                  
launch OpenICLEval[baichuan2-13b-chat-hf/ceval-high_school_geography] on CPU                                                                                                                                                 
launch OpenICLEval[baichuan2-13b-chat-hf/ceval-middle_school_politics] on CPU                                                                                                                                                
launch OpenICLEval[baichuan2-13b-chat-hf/ceval-middle_school_geography] on CPU                                                                                                                                               
launch OpenICLEval[baichuan2-13b-chat-hf/ceval-modern_chinese_history] on CPU                                                                                                                                                
launch OpenICLEval[baichuan2-13b-chat-hf/ceval-ideological_and_moral_cultivation] on CPU                                                                                                                                     
  0%|                                                                                                                                                                                                 | 0/52 [00:17<?, ?it/s]
到了这里就一直卡着，没有任何响应，一开始CPU还是100%，后面依然CPU不在运行了依然卡着，且每次重新跑都会卡在此处，ideological文件并无异常，和其他文件相同类型的生成结果。
Ctrl+C后出现下列输出：
```shell
^CTraceback (most recent call last):
  File "/code/opencompass/run.py", line 311, in <module>
    main()
  File "/code/opencompass/run.py", line 302, in main
    runner(tasks)
  File "/code/opencompass/opencompass/runners/base.py", line 38, in __call__
    status = self.launch(tasks)
  File "/code/opencompass/opencompass/runners/local.py", line 125, in launch
    with ThreadPoolExecutor(
  File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 649, in __exit__
    self.shutdown(wait=True)
  File "/opt/conda/lib/python3.10/concurrent/futures/thread.py", line 235, in shutdown
    t.join()
  File "/opt/conda/lib/python3.10/threading.py", line 1096, in join
    self._wait_for_tstate_lock()
  File "/opt/conda/lib/python3.10/threading.py", line 1116, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt
```
跑的是所有52个val任务。
附加信息（生成的json文件）：
![image](https://github.com/open-compass/opencompass/assets/34636084/1b6c15f9-460f-4468-9f43-02fed45762de)
ceval-ideological_and_moral_cultivation.json文件内容：
![image](https://github.com/open-compass/opencompass/assets/34636084/b82273cb-002a-40ca-ba71-860d4e86a325)


You may try to run in debug mode for more information.

```bash
python run.py configs/eval_baichuan2.py -r --mode eval --debug
```

However, I guess the problem lies in the failure to connect to `huggingface.co` and download the `accuracy` metric.

中间出现警告：
```shell
./outputs/default/20230911_161623/logs/eval/baichuan2-13b-chat-hf/ceval-computer_architecture.out
 62%|███████████████████████████████████████████████████████████████████                                          | 32/52 [16:08<00:19,  1.01it/s]09/11 17:44:38 - OpenCompass - WARNING - task OpenICLEval[baichuan2-13b-chat-hf/ceval-logic] fail, see
./outputs/default/20230911_161623/logs/eval/baichuan2-13b-chat-hf/ceval-logic.out
 63%|█████████████████████████████████████████████████████████████████████▏                                       | 33/52 [16:18<01:01,  3.25s/it]^CException ignored in: <module 'threading' from '/opt/conda/lib/python3.10/threading.py'>
......
```
看了一些对应日志，出现下面错误：
```shell
/opt/conda/lib/python3.10/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Traceback (most recent call last):
  File "/code/opencompass/opencompass/tasks/openicl_eval.py", line 218, in <module>
    inferencer.run()
  File "/code/opencompass/opencompass/tasks/openicl_eval.py", line 68, in run
    self._score()
  File "/code/opencompass/opencompass/tasks/openicl_eval.py", line 159, in _score
    result = icl_evaluator.score(
  File "/code/opencompass/opencompass/openicl/icl_evaluator/icl_hf_evaluator.py", line 75, in score
    metric = evaluate.load(self.metric)
  File "/opt/conda/lib/python3.10/site-packages/evaluate/loading.py", line 731, in load
    evaluation_module = evaluation_module_factory(
  File "/opt/conda/lib/python3.10/site-packages/evaluate/loading.py", line 681, in evaluation_module_factory
    raise FileNotFoundError(
FileNotFoundError: Couldn't find a module script at /code/opencompass/accuracy/accuracy.py. Module 'accuracy' doesn't exist on the Hugging Face Hub either.
```
在进行ceval-gen时，这个accuracy.py是必须的吗？如果是必须的，请问该从何处下载？

Yes, the `accuracy` module is a must. Try to run the following codes:

```python
import evaluate
evaluate.load('accuracy')
```

If no error, it is downloaded successfully.

Feel free to re-open if needed.

你好，按照你的eval_baichuan2.py中的代码修改后，运行 python run.py configs/eval_baichuan2.py之后，代码出现如下错误：
![image](https://github.com/open-compass/opencompass/assets/139730426/cfe454fe-5000-429e-b1ab-a81e36c7dbf5)