Repository: open-compass/opencompass
Issue: [Bug] Deepseek 32b量化模型评测异常
Root cause: Resource Mishandling

### Prerequisite

- [x] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [x] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I have modified the code (config is not considered code), or I'm working on my own tasks/models/datasets.

### Environment

{'CUDA available': True,
 'CUDA_HOME': '/usr/local/cuda',
 'GCC': 'gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0',
 'GPU 0,1,2,3': 'NVIDIA GeForce RTX 4090',
 'MMEngine': '0.10.6',
 'MUSA available': False,
 'NVCC': 'Cuda compilation tools, release 11.8, V11.8.89',
 'OpenCV': '4.11.0',
 'PyTorch': '2.5.1+cu124',
 'PyTorch compiling details': 'PyTorch built with:\n'
                              '  - GCC 9.3\n'
                              '  - C++ Version: 201703\n'
                              '  - Intel(R) oneAPI Math Kernel Library Version '
                              '2024.2-Product Build 20240605 for Intel(R) 64 '
                              'architecture applications\n'
                              '  - Intel(R) MKL-DNN v3.5.3 (Git Hash '
                              '66f0cb9eb66affd2da3bf5f8d897376f04aae6af)\n'
                              '  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
                              '  - LAPACK is enabled (usually provided by '
                              'MKL)\n'
                              '  - NNPACK is enabled\n'
                              '  - CPU capability usage: AVX512\n'
                              '  - CUDA Runtime 12.4\n'
                              '  - NVCC architecture flags: '
                              '-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n'
                              '  - CuDNN 90.1\n'
                              '  - Magma 2.6.1\n'
                              '  - Build settings: BLAS_INFO=mkl, '
                              'BUILD_TYPE=Release, CUDA_VERSION=12.4, '
                              'CUDNN_VERSION=9.1.0, '
                              'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
                              'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 '
                              '-fabi-version=11 -fvisibility-inlines-hidden '
                              '-DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO '
                              '-DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON '
                              '-DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK '
                              '-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE '
                              '-O2 -fPIC -Wall -Wextra -Werror=return-type '
                              '-Werror=non-virtual-dtor -Werror=bool-operation '
                              '-Wnarrowing -Wno-missing-field-initializers '
                              '-Wno-type-limits -Wno-array-bounds '
                              '-Wno-unknown-pragmas -Wno-unused-parameter '
                              '-Wno-strict-overflow -Wno-strict-aliasing '
                              '-Wno-stringop-overflow -Wsuggest-override '
                              '-Wno-psabi -Wno-error=old-style-cast '
                              '-Wno-missing-braces -fdiagnostics-color=always '
                              '-faligned-new -Wno-unused-but-set-variable '
                              '-Wno-maybe-uninitialized -fno-math-errno '
                              '-fno-trapping-math -Werror=format '
                              '-Wno-stringop-overflow, LAPACK_INFO=mkl, '
                              'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
                              'TORCH_VERSION=2.5.1, USE_CUDA=ON, USE_CUDNN=ON, '
                              'USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, '
                              'USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, '
                              'USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, '
                              'USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, '
                              'USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, \n',
 'Python': '3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0]',
 'TorchVision': '0.20.1+cu124',
 'lmdeploy': '0.2.2',
 'numpy_random_seed': 2147483648,
 'opencompass': '0.4.0+73c8095',
 'sys.platform': 'linux',
 'transformers': '4.48.3'}

### Reproduces the problem - code/configuration sample

/opencompass/opencompass/configs/models/deepseek/hf_deepseek_r1_distill_qwen_32b_int4.py

```
from opencompass.models import HuggingFacewithChatTemplate
from opencompass.utils.text_postprocessors import extract_non_reasoning_content

models = [
    dict(
        type=HuggingFacewithChatTemplate,
        abbr='deepseek-r1-distill-qwen-32b-int4-hf',
        # path='dwetzel/DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4',
        path='/root/autodl-tmp/models/DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4',
        max_out_len=16384,
        batch_size=8,
        run_cfg=dict(num_gpus=4),
        pred_postprocessor=dict(type=extract_non_reasoning_content)
    )
]
```


### Reproduces the problem - command or script

python run.py --hf-path /root/autodl-tmp/models/DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4 --datasets ceval_gen --hf-num-gpus 4 --max-out-len 16384 --debug
```
02/28 14:40:48 - OpenCompass - INFO - Loading ceval_gen: /root/autodl-tmp/opencompass/opencompass/configs/./datasets/ceval/ceval_gen.py
02/28 14:40:48 - OpenCompass - INFO - Loading example: /root/autodl-tmp/opencompass/opencompass/configs/./summarizers/example.py
02/28 14:40:48 - OpenCompass - INFO - Current exp folder: outputs/default/20250228_144048
02/28 14:40:48 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
02/28 14:40:48 - OpenCompass - INFO - Partitioned into 1 tasks.
02/28 14:40:54 - OpenCompass - WARNING - Debug mode, log will be saved to tmp/1269_debug.log
```

### Reproduces the problem - error message

02/27 19:45:06 - OpenCompass - INFO - Task [DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-computer_network,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-operating_system,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-computer_architecture,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-college_programming,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-college_physics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-college_chemistry,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-advanced_mathematics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-probability_and_statistics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-discrete_mathematics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-electrical_engineer,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-metrology_engineer,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_mathematics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_physics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_chemistry,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_biology,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-middle_school_mathematics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-middle_school_biology,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-middle_school_physics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-middle_school_chemistry,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-veterinary_medicine,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-college_economics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-business_administration,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-marxism,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-mao_zedong_thought,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-education_science,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-teacher_qualification,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_politics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_geography,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-middle_school_politics,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-middle_school_geography,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-modern_chinese_history,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-ideological_and_moral_cultivation,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-logic,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-law,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-chinese_language_and_literature,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-art_studies,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-professional_tour_guide,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-legal_professional,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_chinese,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-high_school_history,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-middle_school_history,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-civil_servant,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-sports_science,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-plant_protection,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-basic_medicine,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-clinical_medicine,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-urban_and_rural_planner,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-accountant,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-fire_engineer,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-environmental_impact_assessment_engineer,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-tax_accountant,DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-physician]

Loading checkpoint shards:   0%|          | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards:  20%|██        | 1/5 [00:01<00:04,  1.23s/it]
Loading checkpoint shards:  40%|████      | 2/5 [00:02<00:04,  1.40s/it]
Loading checkpoint shards:  60%|██████    | 3/5 [00:04<00:02,  1.49s/it]
Loading checkpoint shards:  80%|████████  | 4/5 [00:05<00:01,  1.53s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:07<00:00,  1.44s/it]
Loading checkpoint shards: 100%|██████████| 5/5 [00:07<00:00,  1.45s/it]
02/27 19:45:18 - OpenCompass - INFO - using stop words: ['<｜end▁of▁sentence｜>']
ceval-computer_network dev 5
ceval-computer_network val 19
ceval-computer_network test 171
02/27 19:45:20 - OpenCompass - INFO - Start inferencing [DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4_hf/ceval-computer_network]

  0%|          | 0/19 [00:00<?, ?it/s]
100%|██████████| 19/19 [00:00<00:00, 498073.60it/s]
[2025-02-27 19:45:20,499] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting build dataloader
[2025-02-27 19:45:20,499] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...

  0%|          | 0/3 [00:00<?, ?it/s]02/27 19:45:20 - OpenCompass - INFO - Generation Args of Huggingface: 
02/27 19:45:20 - OpenCompass - INFO - {'stopping_criteria': [<opencompass.models.huggingface_above_v4_33._get_stopping_criteria.<locals>.MultiTokenEOSCriteria object at 0x7f9d1db2ffa0>], 'max_new_tokens': 256, 'pad_token_id': 151643}
/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:628: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:633: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.95` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.
  warnings.warn(
W0227 19:52:13.382000 22474 site-packages/torch/distributed/elastic/agent/server/api.py:704] Received Signals.SIGHUP death signal, shutting down workers
W0227 19:52:13.383000 22474 site-packages/torch/distributed/elastic/multiprocessing/api.py:897] Sending process 22546 closing signal SIGHUP
Traceback (most recent call last):
  File "/root/autodl-tmp/.conda/envs/opencompass/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 355, in wrapper
    return f(*args, **kwargs)
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 919, in main
    run(args)
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 910, in run
    elastic_launch(
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 138, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 260, in launch_agent
    result = agent.run()
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/metrics/api.py", line 137, in wrapper
    result = f(*args, **kwargs)
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/agent/server/api.py", line 696, in run
    result = self._invoke_run(role)
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/agent/server/api.py", line 855, in _invoke_run
    time.sleep(monitor_interval)
  File "/root/autodl-tmp/.conda/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/api.py", line 84, in _terminate_process_handler
    raise SignalException(f"Process {os.getpid()} got signal: {sigval}", sigval=sigval)
torch.distributed.elastic.multiprocessing.api.SignalException: Process 22474 got signal: 1


### Other information

_No response_

模型下载地址：git clone https://huggingface.co/dwetzel/DeepSeek-R1-Distill-Qwen-32B-GPTQ-INT4


Please make sure the model can be loaded successfully with pure Transformer rather than the opencompass.