Repository: open-compass/opencompass
Issue: [Bug] Wrong word spell in code

### Prerequisite

- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass).

### Type

I'm evaluating with the officially supported tasks/models/datasets.

### Environment

-

### Reproduces the problem - code/configuration sample

-

### Reproduces the problem - command or script

-

### Reproduces the problem - error message

-

### Other information

This may have an impact on the naming of the output file, and there may be interference during secondary development.

<img width="597" alt="image" src="https://github.com/open-compass/opencompass/assets/30463691/0d3eea19-209d-46ec-a66c-7d4a9f20762c">



Thanks for your carefulness, we will be grateful if you could follow the contribution guideline to create a pr to fix this issue! https://opencompass.readthedocs.io/en/latest/notes/contribution_guide.html

I have submitted a pull request, and here is the link  https://github.com/open-compass/opencompass/pull/594 .
By the way, I would like to ask why hf_llama2 cannot use batch processing. In fact, if I change the default setting in OpenCampass to use the unk_token as the pad_token instead of the eos_token, hf_llama2 seems to be able to correctly use batch inference by hf.

Here is the experiment.

Init model
```
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("/xxx/llama2-7b-hf", use_fast=False,
                                          trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("/xxx/llama2-7b-hf", device_map="auto",
                                             trust_remote_code=True)
```

single infer:
```
inputs = tokenizer('北京这个城市', return_tensors='pt')
inputs = inputs.to('cuda:0')
pred = model.generate(**inputs, max_new_tokens=64, repetition_penalty=1.1)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
"""
北京这个城市很大，总是有新的地方可以去。
I've been to Beijing many times, but I still haven't seen all of it.
The city is so big that there are always new places to go.
"""


inputs = tokenizer('天津和北京的关系', return_tensors='pt')
inputs = inputs.to('cuda:0')
pred = model.generate(**inputs, max_new_tokens=64, repetition_penalty=1.1)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
"""
天津和北京的关系
The relationship between Beijing and Tianjin
Tianjin is a city in northern China, located on the Bohai Sea. It has a population of over 15 million people and is one of the most important economic centers in the country. The city is also home to many universities and

"""
```

batch infer:

```
tokenizer.pad_token = tokenizer.unk_token
tokenizer.padding_side = "left"
inputs = tokenizer(['北京这个城市', '天津和北京的关系'], padding=True, truncation=True,
                   return_tensors='pt')
inputs = inputs.to('cuda:0')
inputs.data.pop('attention_mask')
print(inputs)
pred = model.generate(**inputs, max_new_tokens=64, repetition_penalty=1.1)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
print(tokenizer.decode(pred.cpu()[1], skip_special_tokens=True))
"""
北京这个城市很大，总是有新的地方可以去。
I've been to Beijing many times, but I still haven't seen all of it.
The city is so big that there are always new places to go.
天津和北京的关系
The relationship between Beijing and Tianjin
Tianjin is a city in northern China, located on the Bohai Sea. It has a population of over 15 million people and is one of the most important economic centers in the country. The city is also home to many universities and

"""
```

It doesn't seem to make any difference.

However, the results I obtained from running llama2-7b seem to have some differences compared to the leaderboard. I am not sure if these differences are due to minor variations caused by factors such as CUDA or other environments, or if they are a result of batch inference. Nevertheless, this experiment suggests that it seems possible to conduct lossless batch inference with llama2. Have I made any mistakes in my approach?


You can set `batch_padding=True` in your model config for batch inference, wo use eos_token_id as defaults. You can also change it use `pad_token_id` parameter.

Our leaderboard use single generate as default, please provide more details(config, dataset, any modification) if you can not reproduce the accuracy of our config

Yes, I can see that the leaderboard uses single-instance generation. Therefore, I had a question, as OpenCampass's code seems to default to `batch_padding=False` only for hf_llama（and same to document demo ）. I checked models like Baichuan, and they don't have this parameter.
__->Is it because there are issues with batch inference in hf_llama?<-__  I'm not quite sure, so I did some validation, but it seems there is no such problem. 

Additionally, whether I add `pad_token_id` in the command line or in the model config's `tokenizer_kwargs`, it doesn't seem to work. In the end, I directly modified the OpenCampass source code at huggingface.py L69. Where should `pad_token_id` be added?

If use batch_padding=True.

The discrepancy may arise when using numerous examples, such as bs=4, 8, or 16, along with varying prompt lengths.


https://github.com/open-compass/opencompass/blob/8160cb84e3b4222b1cb5c90c8b09d304d922c386/opencompass/models/huggingface.py#L68 We use  `batch_padding=False` as default for all models .