Model generation is not stopping on eos token
Author: hengjiUSTC
Pull request: no

### System Info

I have a finetuned llama2 model: "HenryJJ/tangshi-llama2-7b-chat-qlora" loaded by:
```
from peft import AutoPeftModelForCausalLM
# Local directory where the model is saved
local_model_path = "HenryJJ/tangshi-llama2-7b-chat-qlora"


tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"
model = AutoPeftModelForCausalLM.from_pretrained(
    local_model_path,
    low_cpu_mem_usage=True,
    torch_dtype=torch.float16,
    load_in_4bit=True,
)
```
when i am running the inference:
```
prompt = f"""

 
你是一个唐诗助手,帮助用户写一首对应要求的唐诗

作者:李商隱
标签:黄河;咏物;抒情;鼓吹曲辞;乐府;咏物诗


"""
input_ids = tokenizer(prompt, return_tensors="pt", truncation=True).input_ids.cuda()
# with torch.inference_mode():
outputs = model.generate(
    input_ids=input_ids,
    max_new_tokens=400,
    top_p=0.9,
    temperature=0.7,
    pad_token_id= tokenizer.eos_token_id,
    eos_token_id= tokenizer.eos_token_id,
)

print(f"Prompt:\n{prompt}\n")
print(f"Generated output:\n{tokenizer.batch_decode(outputs.detach().cpu().numpy(), skip_special_tokens=True)[0][len(prompt):]}")
```

the result does not stop on eos token `
`
```
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:381: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.7` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:386: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.
  warnings.warn(
Prompt:


 
你是一个唐诗助手,帮助用户写一首对应要求的唐诗

作者:李商隱
标签:黄河;咏物;抒情;鼓吹曲辞;乐府;咏物诗




Generated output:
辭 黃河
黃河出塞西流幾千里，歷代將士戰死不知道。
黃河入海東流幾千里，歷代船員採藥不知道。
吹吹鼓鼓，喧喧鼓鼓。
黃河內流外流，兩岸淚水不相逢。
吹吹鼓鼓，哭哭鼓鼓。


鼓吹曲辭 黃河
黃河出塞西流幾千里，歷代將士戰死不知道。
黃河入海東流幾千里，歷代船員採藥不知道。
吹吹鼓鼓，喧喧鼓鼓。
黃河內流外流，兩岸淚水不相逢。
吹吹鼓鼓，哭哭鼓鼓。
黃河誰人知，從今古今新。
黃河誰人識，從今古今新。
吹吹鼓鼓，哭哭鼓鼓。

黃河出塞西流幾
```

Same thing happens when I use pipeline
```
from transformers import (
    pipeline,
)
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=300)
prompt = f"""
 
你是一个唐诗助手,帮助用户写一首对应要求的唐诗

作者:駱賓王
标签:思念;七言律诗;秋天;咏物


"""
result = pipe(f"{prompt}", eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id)
print(result[0]['generated_text'])
```
I got :
```

 
你是一个唐诗助手,帮助用户写一首对应要求的唐诗

作者:駱賓王
标签:思念;七言律诗;秋天;咏物


秋夜懷愁
薰茵寒煙濃，暮鴈飛斜堪。
漠漠江頭月，孤城殘鐘鐔。
歸來寄與憶，欲斷欲切斷。
欲斷欲切斷，欲斷欲折折。


題名:秋夜懷愁

題名:秋夜懷愁

題名:秋夜懷愁

題名:秋夜懷愁

題名:秋夜��
```

Is this a bug or i am using it wrong? reference colab: [https://colab.research.google.com/drive/1PkD_isiH7BA1pjgoB_1a55dcxQAkPCKJ?usp=sharing](https://colab.research.google.com/drive/1PkD_isiH7BA1pjgoB_1a55dcxQAkPCKJ?usp=sharing) please ignore the training code in front, reproduce code is in bottom.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://colab.research.google.com/drive/1PkD_isiH7BA1pjgoB_1a55dcxQAkPCKJ?usp=sharing

### Expected behavior

Output should stop on `
`.
Example:
```

 
你是一个唐诗助手,帮助用户写一首对应要求的唐诗

作者:駱賓王
标签:思念;七言律诗;秋天;咏物


秋夜懷愁
薰茵寒煙濃，暮鴈飛斜堪。
漠漠江頭月，孤城殘鐘鐔。
歸來寄與憶，欲斷欲切斷。
欲斷欲切斷，欲斷欲折折。
```


Comments:
Hey! This is a duplicate of a lot of issue, the eos token is set to `AddedToken(content = "
", normalized = True,...)`. This will work:
```python 
from transformers import AutoTokenizer, AddedToken 
tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf", eos_token = AddedToken("
", normalized = False, special=True))
```