RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
Author: 888yyh
Pull request: no

### System Info

windows cpu
i can only use gpu，and ran the code below

import torch
from transformers import AutoTokenizer, AutoModel
from modeling_chatglm import ChatGLMForConditionalGeneration

tokenizer = AutoTokenizer.from_pretrained(
    "./model", trust_remote_code=True)

model = ChatGLMForConditionalGeneration.from_pretrained(
    "./model").half()

while True:
    a = input("请输入您的问题：（输入q以退出）")
    if a.strip() == 'q':
        
        exit()
    response, history = model.chat(tokenizer, "问题：" + a.strip() + '\n答案：', max_length=256, history=[])
    print("回答：", response)

errors:RuntimeError: "LayerNormKernelImpl" not implemented for 'Half',however i use model = ChatGLMForConditionalGeneration.from_pretrained(
    "./model",torch_type=torch.float32),error "RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of bFloat "
if i use ChatGLMForConditionalGeneration.from_pretrained(
    "./model",torch_type=torch.Bfloat) errors"RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of Float"
how can i run the code correctly?

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

windows cpu
i can only use gpu，and ran the code below

import torch
from transformers import AutoTokenizer, AutoModel
from modeling_chatglm import ChatGLMForConditionalGeneration

tokenizer = AutoTokenizer.from_pretrained(
    "./model", trust_remote_code=True)

model = ChatGLMForConditionalGeneration.from_pretrained(
    "./model").half()

while True:
    a = input("请输入您的问题：（输入q以退出）")
    if a.strip() == 'q':
        
        exit()
    response, history = model.chat(tokenizer, "问题：" + a.strip() + '\n答案：', max_length=256, history=[])
    print("回答：", response)

errors:RuntimeError: "LayerNormKernelImpl" not implemented for 'Half',however i use model = ChatGLMForConditionalGeneration.from_pretrained(
    "./model",torch_type=torch.float32),error "RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of bFloat "
if i use ChatGLMForConditionalGeneration.from_pretrained(
    "./model",torch_type=torch.Bfloat) errors"RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of Float"
how can i run the code correctly?

### Expected behavior

windows cpu
i can only use gpu，and ran the code below

import torch
from transformers import AutoTokenizer, AutoModel
from modeling_chatglm import ChatGLMForConditionalGeneration

tokenizer = AutoTokenizer.from_pretrained(
    "./model", trust_remote_code=True)

model = ChatGLMForConditionalGeneration.from_pretrained(
    "./model").half()

while True:
    a = input("请输入您的问题：（输入q以退出）")
    if a.strip() == 'q':
        
        exit()
    response, history = model.chat(tokenizer, "问题：" + a.strip() + '\n答案：', max_length=256, history=[])
    print("回答：", response)

errors:RuntimeError: "LayerNormKernelImpl" not implemented for 'Half',however i use model = ChatGLMForConditionalGeneration.from_pretrained(
    "./model",torch_type=torch.float32),error "RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of bFloat "
if i use ChatGLMForConditionalGeneration.from_pretrained(
    "./model",torch_type=torch.Bfloat) errors"RuntimeError: mixed dtype (CPU): expect parameter to have scalar type of Float"
how can i run the code correctly?

Comments:
Hi @888yyh, thanks for raising this issue! 

So that we can help you, please make sure to follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) and [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md) as well as filling out all of the information in the [issue template](https://github.com/huggingface/transformers/blob/main/.github/ISSUE_TEMPLATE/bug-report.yml).

In this instance, could you: 
* Provide a code snippet we can run to recreate the error. We don't have access to `"./model"`
* Provide information on the running environment: run `transformers-cli env` in the terminal and copy-paste the output
* Clarify what hardware this is running on (CPU, GPU etc.)
* Format the code example to use markdown multiline code formatting. Any code and traceback errors should be between three backticks like so ` ``` CODE HERE ``` `

> Hi @888yyh, thanks for raising this issue!
> 
> So that we can help you, please make sure to follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) and [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md) as well as filling out all of the information in the [issue template](https://github.com/huggingface/transformers/blob/main/.github/ISSUE_TEMPLATE/bug-report.yml).
> 
> In this instance, could you:
> 
> * Provide a code snippet we can run to recreate the error. We don't have access to `"./model"`
> * Provide information on the running environment: run `transformers-cli env` in the terminal and copy-paste the output
> * Clarify what hardware this is running on (CPU, GPU etc.)
> * Format the code example to use markdown multiline code formatting. Any code and traceback errors should be between three backticks like so `` ``` CODE HERE ``` ``
"./model" is the path saving model weights
code:
import torch
from transformers import AutoTokenizer, AutoModel
from modeling_chatglm import ChatGLMForConditionalGeneration
# tokenizer = AutoTokenizer.from_pretrained(
#     ".\\model", cache_dir ='./model1' ,trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(
    "./model", trust_remote_code=True)
# model = ChatGLMForConditionalGeneration.from_pretrained(
    # "./model").half().cuda()
model = ChatGLMForConditionalGeneration.from_pretrained(
    "./model", torch_dtype=torch.float32)
# model = ChatGLMForConditionalGeneration.from_pretrained(
#     "./model", cache_dir = './model1').float()
while True:
    a = input("请输入您的问题：（输入q以退出）")
    if a.strip() == 'q':
        
        exit()
    response, history = model.chat(tokenizer, "问题：" + a.strip() + '\n答案：', max_length=256, history=[])
    print("回答：", response)


transformers==4.16.1 torch 2.0.0+cpu' windows

@888yyh Please follow the request above and format the code example properly using markdown code formatting like so: ` ``` CODE GOES HERE ``` `.

We don't have access to `model` and so cannot run to code to reproduce the error. You'll need to provide an example with a publicly available checkpoint in which the same error occurs. 

Please note that 4.16.1 is quite an old version of transformers, we are currently on 4.34.1. Please try upgrading your version of transformers. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.