AttributeError: 'str' object has no attribute 'shape'
Author: MARD1NO
Pull request: no

### System Info

transformers version 4.42.1

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run chatglm3 generation

### Expected behavior

raise Error like: 
```
    def get_masks(self, input_ids, past_key_values, padding_mask=None):
        batch_size, seq_length = input_ids.shape
        full_attention_mask = torch.ones(batch_size, seq_length, seq_length, device=input_ids.device)
        full_attention_mask.tril_()
        past_length = 0
        if past_key_values:
>           past_length = past_key_values[0][0].shape[0]
E           AttributeError: 'str' object has no attribute 'shape'

../../../.cache/huggingface/modules/transformers_modules/chatglm3-6b/modeling_chatglm.py:683: AttributeError
```

Comments:
I think it may be related to https://github.com/huggingface/transformers/pull/31679

Hi @MARD1NO, thanks for opening a PR! 

So that we can best help you, could you: 
* Share the full running env: run `transformers-cli env` in the terminal and copy-paste the output
* Share a minimal code snippet to reproduce the error 

It does look like the error is similar to the one in #31679. As the code in the description looks like it's custom, rather than from the transformers library, that code might need to be updated to handle this

cc @gante @zucchini-nlp 

> Hi @MARD1NO, thanks for opening a PR!
> 
> So that we can best help you, could you:
> 
> * Share the full running env: run `transformers-cli env` in the terminal and copy-paste the output
> * Share a minimal code snippet to reproduce the error
> 
> It does look like the error is similar to the one in #31679. As the code in the description looks like it's custom, rather than from the transformers library, that code might need to be updated to handle this
> 
> cc @gante @zucchini-nlp

Hi @amyeroberts, thanks for your quick reply :D 

env is: 
```
- `transformers` version: 4.42.1
- Platform: Linux-5.4.0-176-generic-x86_64-with-glibc2.31
- Python version: 3.11.5
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.1
- Accelerate version: 0.25.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 3090
```

The minimal code snippet is when using chatglm3 to generate like that:
```
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", padding_side="left", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("THUDM/chatglm3-6b", device_map="auto", trust_remote_code=True)
model = model.eval()

prompts = ["hello, how are you?", "Who are you?"]

inputs = tokenizer(prompts, padding=True, return_tensors='pt')
inputs = inputs.to(model.device)
pred = model.generate(**inputs, 
                      max_new_tokens=128,
                      do_sample=False,
                      repetition_penalty=1.0)
print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))

```

And I test success in transformers==4.40.1, I thinks there exist some bug

Hi @MARD1NO, thanks for sharing! 

As the modeling code is defined in https://huggingface.co/THUDM/chatglm3-6b/blob/main/modeling_chatglm.py, I'd suggest opening a discussion on the [THUDM/chatglm3-6b](https://huggingface.co/THUDM/chatglm3-6b) repo to report this error

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.