Repository: open-compass/opencompass
Issue: Data type problem during eval[Bug] 
Root cause: Environment Incompatibility

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

During calculating ppl for the mmlu dataset, I run into this problem. My local llama2 7b model is of bp 16 type but it seems that it is not supported.

### Reproduces the problem - code/configuration sample

My eval script is like:
```python
from mmengine.config import read_base

with read_base():
    # from .datasets.collections.base_medium_llama import piqa_datasets, siqa_datasets
    from .datasets.mmlu.mmlu_ppl import mmlu_datasets
    from .models.hf_llama.hf_llama2_7b_local import models


datasets = [*mmlu_datasets]
```
and the local path is set in the models file

### Reproduces the problem - command or script

```python
python run.py configs/eval_hf_llama2_7b.py -w results_output --debug
```

### Reproduces the problem - error message

Here are the error info:

```
10/09 03:24:02 - OpenCompass - WARNING - pad_token_id is not set for the tokenizer.
10/09 03:24:02 - OpenCompass - WARNING - Using eos_token_id as pad_token_id.
10/09 03:24:02 - OpenCompass - WARNING - 
 la False
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  7.75it/s]
/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
10/09 03:24:03 - OpenCompass - INFO - Start inferencing [llama2-7b-hf-local/lukaemon_mmlu_high_school_physics]
100%|███████████████████████████████████████████████████████████████████████████████████████| 151/151 [00:00<00:00, 2372059.57it/s]
[2023-10-09 03:24:03,822] [opencompass.openicl.icl_inferencer.icl_ppl_inferencer] [INFO] Calculating PPL for prompts labeled 'A'
  0%|                                                                                                       | 0/19 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/tasks/openicl_infer.py", line 146, in <module>
    inferencer.run()
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/tasks/openicl_infer.py", line 76, in run
    self._inference()
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/tasks/openicl_infer.py", line 113, in _inference
    inferencer.inference(retriever,
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/openicl/icl_inferencer/icl_ppl_inferencer.py", line 179, in inference
    sub_res = self.model.get_ppl_from_template(
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/models/base.py", line 112, in get_ppl_from_template
    return self.get_ppl(inputs, mask_length)
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/models/huggingface.py", line 321, in get_ppl
    return np.concatenate([
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/models/huggingface.py", line 322, in <listcomp>
    self._get_ppl(inputs=[text], mask_length=mask_length)
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/models/huggingface.py", line 343, in _get_ppl
    outputs, inputs = self.get_logits(inputs)
  File "/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/models/huggingface.py", line 297, in get_logits
    outputs = self.model(input_ids)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1038, in forward
    outputs = self.model(
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 925, in forward
    layer_outputs = decoder_layer(
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 635, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 349, in forward
    query_states = self.q_proj(hidden_states)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'
[2023-10-09 03:24:08,543] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 155065) of binary: /alg_vepfs/public/miniconda_dirs/envs/opencompass/bin/python
Traceback (most recent call last):
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 806, in main
    run(args)
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
    elastic_launch(
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/alg_vepfs/public/miniconda_dirs/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/alg_vepfs/project/basellm/code/eval/opencompass/opencompass/tasks/openicl_infer.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-10-09_03:24:08
  host      : localhost
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 155065)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
```

### Other information

_No response_

Please check if you are using the CPU version of pytorch or the GPU version of pytorch. We require the use of the GPU version of pytorch.

![Screen Shot 2023-10-09 at 11 56 43 AM](https://github.com/open-compass/opencompass/assets/44200118/58aa64c7-4280-48b7-bab8-41cf2a9fbeac)
I got one a100 in the environment and my torch version is shown in the snapshot

Try to run the following command:

```bash
python3 -c "import torch; print(torch.cuda.is_available())"
```

Expected output is "True"

Thank you. Cuda is indeed unavailable. I cloned an old env and the eval is now running.