Repository: open-compass/opencompass
Issue: [Bug] ModuleNotFoundError: No module named 'mmengine'
Root cause: Environment Incompatibility

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

{'CUDA available': True,
 'CUDA_HOME': '/usr/local/cuda-12.4',
 'GCC': 'gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0',
 'GPU 0,1,2,3,4,5,6,7': 'NVIDIA H800 PCIe',
 'MMEngine': '0.10.4',
 'MUSA available': False,
 'NVCC': 'Cuda compilation tools, release 12.4, V12.4.131',
 'OpenCV': '4.9.0',
 'PyTorch': '2.3.0',
 'PyTorch compiling details': 'PyTorch built with:\n'
                              '  - GCC 9.3\n'
                              '  - C++ Version: 201703\n'
                              '  - Intel(R) oneAPI Math Kernel Library Version '
                              '2023.1-Product Build 20230303 for Intel(R) 64 '
                              'architecture applications\n'
                              '  - Intel(R) MKL-DNN v3.3.6 (Git Hash '
                              '86e6af5974177e513fd3fee58425e1063e7f1361)\n'
                              '  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
                              '  - LAPACK is enabled (usually provided by '
                              'MKL)\n'
                              '  - NNPACK is enabled\n'
                              '  - CPU capability usage: AVX512\n'
                              '  - CUDA Runtime 12.1\n'
                              '  - NVCC architecture flags: '
                              '-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n'
                              '  - CuDNN 8.9.2\n'
                              '  - Magma 2.6.1\n'
                              '  - Build settings: BLAS_INFO=mkl, '
                              'BUILD_TYPE=Release, CUDA_VERSION=12.1, '
                              'CUDNN_VERSION=8.9.2, '
                              'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
                              'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 '
                              '-fabi-version=11 -fvisibility-inlines-hidden '
                              '-DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO '
                              '-DLIBKINETO_NOROCTRACER -DUSE_FBGEMM '
                              '-DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK '
                              '-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE '
                              '-O2 -fPIC -Wall -Wextra -Werror=return-type '
                              '-Werror=non-virtual-dtor -Werror=bool-operation '
                              '-Wnarrowing -Wno-missing-field-initializers '
                              '-Wno-type-limits -Wno-array-bounds '
                              '-Wno-unknown-pragmas -Wno-unused-parameter '
                              '-Wno-unused-function -Wno-unused-result '
                              '-Wno-strict-overflow -Wno-strict-aliasing '
                              '-Wno-stringop-overflow -Wsuggest-override '
                              '-Wno-psabi -Wno-error=pedantic '
                              '-Wno-error=old-style-cast -Wno-missing-braces '
                              '-fdiagnostics-color=always -faligned-new '
                              '-Wno-unused-but-set-variable '
                              '-Wno-maybe-uninitialized -fno-math-errno '
                              '-fno-trapping-math -Werror=format '
                              '-Wno-stringop-overflow, LAPACK_INFO=mkl, '
                              'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
                              'PERF_WITH_AVX512=1, TORCH_VERSION=2.3.0, '
                              'USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, '
                              'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, '
                              'USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, '
                              'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, '
                              'USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, '
                              'USE_ROCM_KERNEL_ASSERT=OFF, \n',
 'Python': '3.10.14 (main, Mar 21 2024, 16:24:04) [GCC 11.2.0]',
 'TorchVision': '0.18.0+cu121',
 'numpy_random_seed': 2147483648,
 'opencompass': '0.2.4+cce5b6f',
 'sys.platform': 'linux'}

### Reproduces the problem - code/configuration sample

openical_eval.py

import argparse
import copy
import fnmatch
import math
import os
import os.path as osp
import re
import statistics
import sys
import time
from collections import Counter
from inspect import signature
from typing import List, Optional

from mmengine.config import Config, ConfigDict
from mmengine.utils import mkdir_or_exist

### Reproduces the problem - command or script

CUDA_VISIBLE_DEVICES=2 python run.py configs/eval_llama3.py -w outputs/demo --debug

### Reproduces the problem - error message

Traceback (most recent call last):
  File "/mnt/data131/junyi/code/opencompass/opencompass/tasks/openicl_infer.py", line 9, in <module>
    from mmengine.config import Config, ConfigDict
ModuleNotFoundError: No module named 'mmengine'

### Other information

The `mmengine` package is already in the environment; however, the error "No module named 'mmengine'" still occurs when running `run.py`.

You might choose the wrong python interpreter to run start the job

I performed a test using the following code.
import argparse
import os
import os.path as osp
import random
import sys
import time
from typing import Any
import importlib.util

def is_package_installed(package_name):
    package_spec = importlib.util.find_spec(package_name)
    return package_spec is not None

 测试 mmengine 包是否安装
package_name = "mmengine"
if is_package_installed(package_name):
    print(f"{package_name} is installed.")
else:
    print(f"{package_name} is not installed.")

from mmengine.config import Config, ConfigDict
from mmengine.utils import mkdir_or_exist


The results are as following

/mnt/data131/junyi/code/opencompass$ CUDA_VISIBLE_DEVICES=2 python run.py configs/eval_llama3.py -w outputs/demo --debug
**mmengine is installed.**
07/17 10:49:43 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's partitioner registry have been automatically imported from opencompass.partitioners
07/17 10:49:43 - OpenCompass - DEBUG - Get class `SizePartitioner` from "partitioner" registry in "opencompass"
07/17 10:49:43 - OpenCompass - DEBUG - An `SizePartitioner` instance is built from registry, and its implementation can be found in opencompass.partitioners.size
07/17 10:49:43 - OpenCompass - DEBUG - Key eval.runner.task.judge_cfg not found in config, ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Key eval.runner.task.dump_details not found in config, ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Key eval.given_pred not found in config, ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Additional config: {}
07/17 10:49:43 - OpenCompass - INFO - Partitioned into 15 tasks.
07/17 10:49:43 - OpenCompass - DEBUG - Task 0: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_0]
07/17 10:49:43 - OpenCompass - DEBUG - Task 1: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_1]
07/17 10:49:43 - OpenCompass - DEBUG - Task 2: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_2]
07/17 10:49:43 - OpenCompass - DEBUG - Task 3: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_3]
07/17 10:49:43 - OpenCompass - DEBUG - Task 4: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_4]
07/17 10:49:43 - OpenCompass - DEBUG - Task 5: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_5]
07/17 10:49:43 - OpenCompass - DEBUG - Task 6: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_6]
07/17 10:49:43 - OpenCompass - DEBUG - Task 7: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_7]
07/17 10:49:43 - OpenCompass - DEBUG - Task 8: [llama-3-8b-instruct-hf/lukaemon_mmlu_moral_scenarios,llama-3-8b-instruct-hf/lukaemon_mmlu_miscellaneous]
07/17 10:49:43 - OpenCompass - DEBUG - Task 9: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_psychology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_psychology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_macroeconomics,llama-3-8b-instruct-hf/lukaemon_mmlu_elementary_mathematics]
07/17 10:49:43 - OpenCompass - DEBUG - Task 10: [llama-3-8b-instruct-hf/lukaemon_mmlu_moral_disputes,llama-3-8b-instruct-hf/lukaemon_mmlu_prehistory,llama-3-8b-instruct-hf/lukaemon_mmlu_philosophy,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_biology,llama-3-8b-instruct-hf/lukaemon_mmlu_nutrition,llama-3-8b-instruct-hf/lukaemon_mmlu_professional_accounting]
07/17 10:49:43 - OpenCompass - DEBUG - Task 11: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_medicine,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_mathematics,llama-3-8b-instruct-hf/lukaemon_mmlu_clinical_knowledge,llama-3-8b-instruct-hf/lukaemon_mmlu_security_studies,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_microeconomics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_world_history,llama-3-8b-instruct-hf/lukaemon_mmlu_conceptual_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_marketing]
07/17 10:49:43 - OpenCompass - DEBUG - Task 12: [llama-3-8b-instruct-hf/lukaemon_mmlu_human_aging,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_statistics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_us_history,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_chemistry,llama-3-8b-instruct-hf/lukaemon_mmlu_sociology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_geography,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_government_and_politics,llama-3-8b-instruct-hf/lukaemon_mmlu_college_medicine,llama-3-8b-instruct-hf/lukaemon_mmlu_world_religions,llama-3-8b-instruct-hf/lukaemon_mmlu_virology]
07/17 10:49:43 - OpenCompass - DEBUG - Task 13: [llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_european_history,llama-3-8b-instruct-hf/lukaemon_mmlu_logical_fallacies,llama-3-8b-instruct-hf/lukaemon_mmlu_astronomy,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_electrical_engineering,llama-3-8b-instruct-hf/lukaemon_mmlu_college_biology,llama-3-8b-instruct-hf/lukaemon_mmlu_anatomy,llama-3-8b-instruct-hf/lukaemon_mmlu_human_sexuality,llama-3-8b-instruct-hf/lukaemon_mmlu_formal_logic,llama-3-8b-instruct-hf/lukaemon_mmlu_international_law,llama-3-8b-instruct-hf/lukaemon_mmlu_econometrics,llama-3-8b-instruct-hf/lukaemon_mmlu_machine_learning,llama-3-8b-instruct-hf/lukaemon_mmlu_public_relations,llama-3-8b-instruct-hf/lukaemon_mmlu_jurisprudence,llama-3-8b-instruct-hf/lukaemon_mmlu_management]
07/17 10:49:43 - OpenCompass - DEBUG - Task 14: [llama-3-8b-instruct-hf/lukaemon_mmlu_college_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_college_chemistry,llama-3-8b-instruct-hf/lukaemon_mmlu_college_computer_science,llama-3-8b-instruct-hf/lukaemon_mmlu_college_mathematics,llama-3-8b-instruct-hf/lukaemon_mmlu_abstract_algebra,llama-3-8b-instruct-hf/lukaemon_mmlu_global_facts,llama-3-8b-instruct-hf/lukaemon_mmlu_computer_security,llama-3-8b-instruct-hf/lukaemon_mmlu_medical_genetics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_computer_science,llama-3-8b-instruct-hf/lukaemon_mmlu_business_ethics,llama-3-8b-instruct-hf/lukaemon_mmlu_us_foreign_policy]
07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's runner registry have been automatically imported from opencompass.runners
07/17 10:49:43 - OpenCompass - DEBUG - Get class `LocalRunner` from "runner" registry in "opencompass"
07/17 10:49:43 - OpenCompass - DEBUG - An `LocalRunner` instance is built from registry, and its implementation can be found in opencompass.runners.local
07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's task registry have been automatically imported from opencompass.tasks
07/17 10:49:43 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
07/17 10:49:43 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
**mmengine is not installed.**
Traceback (most recent call last):
  File "/mnt/data131/junyi/code/opencompass/opencompass/tasks/openicl_infer.py", line 21, in <module>
    from mmengine.config import Config, ConfigDict
ModuleNotFoundError: No module named 'mmengine'

For the first time, the mmengine  package in installed, but for the second time , the mmengine package is not installed

@wyzh0912 Were you using a virtual environment? Can you perhaps check the environment used to run the code? Something like:

```bash
which python
pip list | grep mmengine
```

May I ask if you have solved it? I have also encountered a similar problem