Repository: open-compass/opencompass
Issue: [Bug] error of aime2024 datasets evaluator import
Root cause: Validation Gap

### 先决条件

- [x] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [x] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

{'CUDA available': False,
 'GCC': 'gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0',
 'MMEngine': '0.10.7',
 'MUSA available': False,
 'OpenCV': '4.11.0',
 'PyTorch': '2.7.1+cu126',
 'PyTorch compiling details': 'PyTorch built with:\n'
                              '  - GCC 11.2\n'
                              '  - C++ Version: 201703\n'
                              '  - Intel(R) oneAPI Math Kernel Library Version '
                              '2024.2-Product Build 20240605 for Intel(R) 64 '
                              'architecture applications\n'
                              '  - Intel(R) MKL-DNN v3.7.1 (Git Hash '
                              '8d263e693366ef8db40acc569cc7d8edf644556d)\n'
                              '  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
                              '  - LAPACK is enabled (usually provided by '
                              'MKL)\n'
                              '  - NNPACK is enabled\n'
                              '  - CPU capability usage: AVX512\n'
                              '  - Build settings: BLAS_INFO=mkl, '
                              'BUILD_TYPE=Release, '
                              'COMMIT_SHA=e2d141dbde55c2a4370fac5165b0561b6af4798b, '
                              'CUDA_VERSION=12.6, CUDNN_VERSION=9.5.1, '
                              'CXX_COMPILER=/opt/rh/gcc-toolset-11/root/usr/bin/c++, '
                              'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 '
                              '-fvisibility-inlines-hidden -DUSE_PTHREADPOOL '
                              '-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER '
                              '-DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM '
                              '-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK '
                              '-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC '
                              '-Wall -Wextra -Werror=return-type '
                              '-Werror=non-virtual-dtor '
                              '-Werror=range-loop-construct '
                              '-Werror=bool-operation -Wnarrowing '
                              '-Wno-missing-field-initializers '
                              '-Wno-unknown-pragmas -Wno-unused-parameter '
                              '-Wno-strict-overflow -Wno-strict-aliasing '
                              '-Wno-stringop-overflow -Wsuggest-override '
                              '-Wno-psabi -Wno-error=old-style-cast '
                              '-fdiagnostics-color=always -faligned-new '
                              '-Wno-maybe-uninitialized -fno-math-errno '
                              '-fno-trapping-math -Werror=format '
                              '-Wno-stringop-overflow, LAPACK_INFO=mkl, '
                              'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
                              'TORCH_VERSION=2.7.1, USE_CUDA=ON, USE_CUDNN=ON, '
                              'USE_CUSPARSELT=1, USE_GFLAGS=OFF, USE_GLOG=OFF, '
                              'USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, '
                              'USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, '
                              'USE_OPENMP=ON, USE_ROCM=OFF, '
                              'USE_ROCM_KERNEL_ASSERT=OFF, \n',
 'Python': '3.10.18 (main, Jun  5 2025, 13:14:17) [GCC 11.2.0]',
 'lmdeploy': "not installed:No module named 'lmdeploy'",
 'numpy_random_seed': 2147483648,
 'opencompass': '0.4.2+',
 'sys.platform': 'linux',
 'transformers': '4.53.2'}

### 重现问题 - 代码/配置示例

from mmengine.config import read_base


with read_base():
    # 数据集和模型总配置文件，所有配置写在这里之后只需要执行run.py时传入这个配置文件就行。后续其他数据集也在这里导入，只需要将需要跑的数据集取消注释，其他数据集注释掉即可
    # 1. 导入数据集配置（使用快速评测版本）

    # AIME2024数据集
    from ..datasets.aime2024.aime2024_gen import aime2024_datasets


    # 2. 导入模型配置
    from ..models.my_api_config.my_api_config import my_think_model_api



# AIME2024数据集
current_dataset_name="aime2024"
temp_datasets=aime2024_datasets

temp_models = my_think_model_api

# 也可以在这里对数据集和模型的配置做一些修改
for dataset in temp_datasets:
    dataset['reader_cfg']['test_range'] = '[0:3]'  # 只评测第一条样本，如果要全测把这个删掉就好。

# 将当前数据集的abbr属性传递给模型，用来在R1正常思考时，储存其思考内容到对应的数据集的文件中。
for temp_model in temp_models:
    temp_model["dataset_name"] = current_dataset_name

print("当前数据集名字为：",current_dataset_name)
print("当前模型名字为：",temp_models[0]["abbr"])
# 3. 组合配置
datasets = temp_datasets
models = temp_models


### 重现问题 - 命令或脚本

python run.py opencompass/configs/my_eval_config/my_eval_config.py

### 重现问题 - 错误信息

Traceback (most recent call last):
  File "/root/autodl-tmp/think_experiment/opencompass/opencompass/tasks/openicl_eval.py", line 466, in <module>
    inferencer.run()
  File "/root/autodl-tmp/think_experiment/opencompass/opencompass/tasks/openicl_eval.py", line 88, in run
    self._score()
  File "/root/autodl-tmp/think_experiment/opencompass/opencompass/tasks/openicl_eval.py", line 100, in _score
    result = self._evaluate_predictions(
  File "/root/autodl-tmp/think_experiment/opencompass/opencompass/tasks/openicl_eval.py", line 249, in _evaluate_predictions
    sig = signature(evaluator_type)
  File "/root/miniconda3/envs/opencompass/lib/python3.10/inspect.py", line 3254, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
  File "/root/miniconda3/envs/opencompass/lib/python3.10/inspect.py", line 3002, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
  File "/root/miniconda3/envs/opencompass/lib/python3.10/inspect.py", line 2396, in _signature_from_callable
    raise TypeError('{!r} is not a callable object'.format(obj))
TypeError: None is not a callable object

### 其他信息

opencompass/configs/aime2024/aime2024_gen.py文件中引用的aime2024_gen_17d799配置文件中MATHEvaluator评估器导入路径错误

This issue has been resolved by [PR#2196](https://github.com/open-compass/opencompass/pull/2196) .