Repository: open-compass/opencompass
Issue: [Bug] RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'
Root cause: Environment Incompatibility

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/InternLM/opencompass/issues/) 和 [讨论](https://github.com/InternLM/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/InternLM/opencompass) 中尚未被修复。

### 问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

### 环境

/opt/miniconda/lib/python3.10/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
{'CUDA available': False,
 'GCC': 'gcc (GCC) 10.2.0',
 'MMEngine': '0.8.4',
 'OpenCV': '4.8.0',
 'PyTorch': '1.13.1+cu117',
 'PyTorch compiling details': 'PyTorch built with:\n'
                              '  - GCC 9.3\n'
                              '  - C++ Version: 201402\n'
                              '  - Intel(R) Math Kernel Library Version '
                              '2020.0.0 Product Build 20191122 for Intel(R) 64 '
                              'architecture applications\n'
                              '  - Intel(R) MKL-DNN v2.6.0 (Git Hash '
                              '52b5f107dd9cf10910aaa19cb47f3abf9b349815)\n'
                              '  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
                              '  - LAPACK is enabled (usually provided by '
                              'MKL)\n'
                              '  - NNPACK is enabled\n'
                              '  - CPU capability usage: AVX2\n'
                              '  - Build settings: BLAS_INFO=mkl, '
                              'BUILD_TYPE=Release, CUDA_VERSION=11.7, '
                              'CUDNN_VERSION=8.5.0, '
                              'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
                              'CXX_FLAGS= -fabi-version=11 -Wno-deprecated '
                              '-fvisibility-inlines-hidden -DUSE_PTHREADPOOL '
                              '-fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM '
                              '-DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK '
                              '-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE '
                              '-DEDGE_PROFILER_USE_KINETO -O2 -fPIC '
                              '-Wno-narrowing -Wall -Wextra '
                              '-Werror=return-type -Werror=non-virtual-dtor '
                              '-Wno-missing-field-initializers '
                              '-Wno-type-limits -Wno-array-bounds '
                              '-Wno-unknown-pragmas -Wunused-local-typedefs '
                              '-Wno-unused-parameter -Wno-unused-function '
                              '-Wno-unused-result -Wno-strict-overflow '
                              '-Wno-strict-aliasing '
                              '-Wno-error=deprecated-declarations '
                              '-Wno-stringop-overflow -Wno-psabi '
                              '-Wno-error=pedantic -Wno-error=redundant-decls '
                              '-Wno-error=old-style-cast '
                              '-fdiagnostics-color=always -faligned-new '
                              '-Wno-unused-but-set-variable '
                              '-Wno-maybe-uninitialized -fno-math-errno '
                              '-fno-trapping-math -Werror=format '
                              '-Werror=cast-function-type '
                              '-Wno-stringop-overflow, LAPACK_INFO=mkl, '
                              'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
                              'PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, '
                              'USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, '
                              'USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, '
                              'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, '
                              'USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n',
 'Python': '3.10.12 (main, Jul  5 2023, 18:54:27) [GCC 11.2.0]',
 'TorchVision': '0.14.1+cu117',
 'numpy_random_seed': 2147483648,
 'opencompass': '0.1.2+b2d602f',
 'sys.platform': 'linux'}


### 重现问题 - 代码/配置示例

pls

### 重现问题 - 命令或脚本

CUDA_VISIBLE_DEVICES=5  python /home/opencompass/run.py /home/opencompass/configs/eval_demo.py -w /home/opencompass/outputs/demo --debug


### 重现问题 - 错误信息
```
/opt/miniconda/lib/python3.10/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
/opt/miniconda/lib/python3.10/site-packages/mmengine/utils/manager.py:113: UserWarning: <class 'mmengine.logging.logger.MMLogger'> instance named of OpenCompass has been created, the method `get_instance` should not accept any other arguments
  warnings.warn(
08/26 21:38:14 - OpenCompass - INFO - Partitioned into 40 tasks.
08/26 21:38:14 - OpenCompass - DEBUG - Task 0: [opt350m/siqa_0]
08/26 21:38:14 - OpenCompass - DEBUG - Task 1: [opt350m/siqa_1]
08/26 21:38:14 - OpenCompass - DEBUG - Task 2: [opt350m/siqa_2]
08/26 21:38:14 - OpenCompass - DEBUG - Task 3: [opt350m/siqa_3]
08/26 21:38:14 - OpenCompass - DEBUG - Task 4: [opt350m/siqa_4]
08/26 21:38:14 - OpenCompass - DEBUG - Task 5: [opt350m/siqa_5]
08/26 21:38:14 - OpenCompass - DEBUG - Task 6: [opt350m/siqa_6]
08/26 21:38:14 - OpenCompass - DEBUG - Task 7: [opt350m/siqa_7]
08/26 21:38:14 - OpenCompass - DEBUG - Task 8: [opt350m/siqa_8]
08/26 21:38:14 - OpenCompass - DEBUG - Task 9: [opt350m/siqa_9]
08/26 21:38:14 - OpenCompass - DEBUG - Task 10: [opt350m/siqa_10]
08/26 21:38:14 - OpenCompass - DEBUG - Task 11: [opt350m/siqa_11]
08/26 21:38:14 - OpenCompass - DEBUG - Task 12: [opt350m/siqa_12]
08/26 21:38:14 - OpenCompass - DEBUG - Task 13: [opt350m/siqa_13]
08/26 21:38:14 - OpenCompass - DEBUG - Task 14: [opt350m/siqa_14]
08/26 21:38:14 - OpenCompass - DEBUG - Task 15: [opt350m/siqa_15]
08/26 21:38:14 - OpenCompass - DEBUG - Task 16: [opt350m/siqa_16]
08/26 21:38:14 - OpenCompass - DEBUG - Task 17: [opt350m/siqa_17]
08/26 21:38:14 - OpenCompass - DEBUG - Task 18: [opt350m/siqa_18]
08/26 21:38:14 - OpenCompass - DEBUG - Task 19: [opt350m/siqa_19]
08/26 21:38:14 - OpenCompass - DEBUG - Task 20: [opt125m/siqa_0]
08/26 21:38:14 - OpenCompass - DEBUG - Task 21: [opt125m/siqa_1]
08/26 21:38:14 - OpenCompass - DEBUG - Task 22: [opt125m/siqa_2]
08/26 21:38:14 - OpenCompass - DEBUG - Task 23: [opt125m/siqa_3]
08/26 21:38:14 - OpenCompass - DEBUG - Task 24: [opt125m/siqa_4]
08/26 21:38:14 - OpenCompass - DEBUG - Task 25: [opt125m/siqa_5]
08/26 21:38:14 - OpenCompass - DEBUG - Task 26: [opt125m/siqa_6]
08/26 21:38:14 - OpenCompass - DEBUG - Task 27: [opt125m/siqa_7]
08/26 21:38:14 - OpenCompass - DEBUG - Task 28: [opt125m/siqa_8]
08/26 21:38:14 - OpenCompass - DEBUG - Task 29: [opt125m/siqa_9]
08/26 21:38:14 - OpenCompass - DEBUG - Task 30: [opt125m/siqa_10]
08/26 21:38:14 - OpenCompass - DEBUG - Task 31: [opt125m/siqa_11]
08/26 21:38:14 - OpenCompass - DEBUG - Task 32: [opt125m/siqa_12]
08/26 21:38:14 - OpenCompass - DEBUG - Task 33: [opt125m/siqa_13]
08/26 21:38:14 - OpenCompass - DEBUG - Task 34: [opt125m/siqa_14]
08/26 21:38:14 - OpenCompass - DEBUG - Task 35: [opt125m/siqa_15]
08/26 21:38:14 - OpenCompass - DEBUG - Task 36: [opt125m/siqa_16]
08/26 21:38:14 - OpenCompass - DEBUG - Task 37: [opt125m/siqa_17]
08/26 21:38:14 - OpenCompass - DEBUG - Task 38: [opt125m/siqa_18]
08/26 21:38:14 - OpenCompass - DEBUG - Task 39: [opt125m/siqa_19]
08/26 21:38:14 - OpenCompass - DEBUG - Modules of opencompass's task registry have been automatically imported from opencompass.tasks
08/26 21:38:14 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
08/26 21:38:14 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
/opt/miniconda/lib/python3.10/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
08/26 21:38:22 - OpenCompass - INFO - Task [opt350m/siqa_0]
/opt/miniconda/lib/python3.10/site-packages/mmengine/utils/manager.py:113: UserWarning: <class 'mmengine.logging.logger.MMLogger'> instance named of OpenCompass has been created, the method `get_instance` should not accept any other arguments
  warnings.warn(
/opt/miniconda/lib/python3.10/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Using the latest cached version of the module from /root/.cache/huggingface/modules/datasets_modules/datasets/social_i_qa/674d85e42ac7430d3dcd4de7007feaffcb1527c535121e09bab2803fbcc925f8 (last modified on Sat Aug 26 21:21:43 2023) since it couldn't be found locally at social_i_qa., or remotely on the Hugging Face Hub.
08/26 21:38:42 - OpenCompass - INFO - Start inferencing [opt350m/siqa_0]
[2023-08-26 21:38:42,334] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
  0%|                                                                                                | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 147, in <module>
    inferencer.run()
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 76, in run
    self._inference()
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 119, in _inference
    inferencer.inference(retriever,
  File "/home/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 122, in inference
    results = self.model.generate_from_template(
  File "/home/opencompass/opencompass/models/base.py", line 117, in generate_from_template
    return self.generate(inputs, max_out_len=max_out_len, **kwargs)
  File "/home/opencompass/opencompass/models/huggingface.py", line 145, in generate
    return sum((self._single_generate(
  File "/home/opencompass/opencompass/models/huggingface.py", line 145, in <genexpr>
    return sum((self._single_generate(
  File "/home/opencompass/opencompass/models/huggingface.py", line 213, in _single_generate
    outputs = self.model.generate(input_ids=input_ids,
  File "/opt/miniconda/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/generation/utils.py", line 1515, in generate
    return self.greedy_search(
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/generation/utils.py", line 2332, in greedy_search
    outputs = self(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 938, in forward
    outputs = self.model.decoder(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 651, in forward
    inputs_embeds = self.project_in(inputs_embeds)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 4716) of binary: /opt/miniconda/bin/python
Traceback (most recent call last):
  File "/opt/miniconda/bin/torchrun", line 33, in <module>
    sys.exit(load_entry_point('torch==1.13.1+cu117', 'console_scripts', 'torchrun')())
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/home/opencompass/opencompass/tasks/openicl_infer.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-08-26_21:38:44
  host      : e9106e301747
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 4716)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
08/26 21:38:45 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
08/26 21:38:45 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
/opt/miniconda/lib/python3.10/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
08/26 21:38:53 - OpenCompass - INFO - Task [opt350m/siqa_1]
/opt/miniconda/lib/python3.10/site-packages/mmengine/utils/manager.py:113: UserWarning: <class 'mmengine.logging.logger.MMLogger'> instance named of OpenCompass has been created, the method `get_instance` should not accept any other arguments
  warnings.warn(
/opt/miniconda/lib/python3.10/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
08/26 21:40:15 - OpenCompass - INFO - Start inferencing [opt350m/siqa_1]
[2023-08-26 21:40:15,223] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
  0%|                                                                                                | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 147, in <module>
    inferencer.run()
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 76, in run
    self._inference()
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 119, in _inference
    inferencer.inference(retriever,
  File "/home/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 122, in inference
    results = self.model.generate_from_template(
  File "/home/opencompass/opencompass/models/base.py", line 117, in generate_from_template
    return self.generate(inputs, max_out_len=max_out_len, **kwargs)
  File "/home/opencompass/opencompass/models/huggingface.py", line 145, in generate
    return sum((self._single_generate(
  File "/home/opencompass/opencompass/models/huggingface.py", line 145, in <genexpr>
    return sum((self._single_generate(
  File "/home/opencompass/opencompass/models/huggingface.py", line 213, in _single_generate
    outputs = self.model.generate(input_ids=input_ids,
  File "/opt/miniconda/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/generation/utils.py", line 1515, in generate
    return self.greedy_search(
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/generation/utils.py", line 2332, in greedy_search
    outputs = self(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 938, in forward
    outputs = self.model.decoder(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 651, in forward
    inputs_embeds = self.project_in(inputs_embeds)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 5071) of binary: /opt/miniconda/bin/python
Traceback (most recent call last):
  File "/opt/miniconda/bin/torchrun", line 33, in <module>
    sys.exit(load_entry_point('torch==1.13.1+cu117', 'console_scripts', 'torchrun')())
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/home/opencompass/opencompass/tasks/openicl_infer.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-08-26_21:40:20
  host      : e9106e301747
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 5071)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
08/26 21:40:21 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
08/26 21:40:21 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
/opt/miniconda/lib/python3.10/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
08/26 21:40:29 - OpenCompass - INFO - Task [opt350m/siqa_2]
/opt/miniconda/lib/python3.10/site-packages/mmengine/utils/manager.py:113: UserWarning: <class 'mmengine.logging.logger.MMLogger'> instance named of OpenCompass has been created, the method `get_instance` should not accept any other arguments
  warnings.warn(
'(ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: 0ce01cbb-c9f4-4644-b995-baad3e105cf6)')' thrown while requesting HEAD https://huggingface.co/facebook/opt-350m/resolve/main/generation_config.json
/opt/miniconda/lib/python3.10/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Using the latest cached version of the module from /root/.cache/huggingface/modules/datasets_modules/datasets/social_i_qa/674d85e42ac7430d3dcd4de7007feaffcb1527c535121e09bab2803fbcc925f8 (last modified on Sat Aug 26 21:21:43 2023) since it couldn't be found locally at social_i_qa., or remotely on the Hugging Face Hub.
08/26 21:40:50 - OpenCompass - INFO - Start inferencing [opt350m/siqa_2]
[2023-08-26 21:40:50,391] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
  0%|                                                                                                | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 147, in <module>
    inferencer.run()
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 76, in run
    self._inference()
  File "/home/opencompass/opencompass/tasks/openicl_infer.py", line 119, in _inference
    inferencer.inference(retriever,
  File "/home/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 122, in inference
    results = self.model.generate_from_template(
  File "/home/opencompass/opencompass/models/base.py", line 117, in generate_from_template
    return self.generate(inputs, max_out_len=max_out_len, **kwargs)
  File "/home/opencompass/opencompass/models/huggingface.py", line 145, in generate
    return sum((self._single_generate(
  File "/home/opencompass/opencompass/models/huggingface.py", line 145, in <genexpr>
    return sum((self._single_generate(
  File "/home/opencompass/opencompass/models/huggingface.py", line 213, in _single_generate
    outputs = self.model.generate(input_ids=input_ids,
  File "/opt/miniconda/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/generation/utils.py", line 1515, in generate
    return self.greedy_search(
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/generation/utils.py", line 2332, in greedy_search
    outputs = self(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 938, in forward
    outputs = self.model.decoder(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 651, in forward
    inputs_embeds = self.project_in(inputs_embeds)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 5426) of binary: /opt/miniconda/bin/python
Traceback (most recent call last):
  File "/opt/miniconda/bin/torchrun", line 33, in <module>
    sys.exit(load_entry_point('torch==1.13.1+cu117', 'console_scripts', 'torchrun')())
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/miniconda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/home/opencompass/opencompass/tasks/openicl_infer.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-08-26_21:40:51
  host      : e9106e301747
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 5426)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
08/26 21:40:51 - OpenCompass - DEBUG - Get class `OpenICLInferTask` from "task" registry in "opencompass"
08/26 21:40:51 - OpenCompass - DEBUG - An `OpenICLInferTask` instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
/opt/miniconda/lib/python3.10/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
08/26 21:40:59 - OpenCompass - INFO - Task [opt350m/siqa_3]
/opt/miniconda/lib/python3.10/site-packages/mmengine/utils/manager.py:113: UserWarning: <class 'mmengine.logging.logger.MMLogger'> instance named of OpenCompass has been created, the method `get_instance` should not accept any other arguments
  warnings.warn(
'(ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: d766e7b0-d587-41ff-aef5-8d96f0936e0c)')' thrown while requesting HEAD https://huggingface.co/facebook/opt-350m/resolve/main/tokenizer_config.json
```

### 其他信息

pls

Does
```python
import torch
print(torch.cuda.is_available())
```
return True? Similar issues are often caused by unsuccessful pytorch-cuda integration.

Feel free to re-open if needed.

pip uninstall torch
pip3 install torch torchvision torchaudio