shape is invalid for input with gemma-7b
Author: battmanux
Pull request: no

Error with gemma-7b[-it] but not with gemma-2b[-it]

# Target

- CPU Intel (cuda not used)
- Linux mint 21.2
- Python 3.10.12 

# Source code

```python
import jinja2
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", torch_dtype=torch.float32, output_hidden_states=True
tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
streamer = TextStreamer(tokenizer)

s = """<start_of_turn>user
How does the brain work?<end_of_turn>
<start_of_turn>model
the way it"""

inputs = tokenizer(s, return_tensors="pt")
o = model.generate(**inputs, streamer=streamer, max_new_tokens=40)
```

# Error message
```
  File "/home/ebatt/.local/lib/python3.10/site-packages/transformers/models/gemma/modeling_gemma.py", line 280, in forward
    attn_output = attn_output.reshape(bsz, q_len, self.hidden_size)
RuntimeError: shape '[1, 18, 3072]' is invalid for input of size 73728
```
# Installed software
```
Requirement already satisfied: transformers in ./.local/lib/python3.10/site-packages (4.38.0)
Requirement already satisfied: filelock in /usr/lib/python3/dist-packages (from transformers) (3.6.0)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in ./.local/lib/python3.10/site-packages (from transformers) (0.20.3)
Requirement already satisfied: numpy>=1.17 in ./.local/lib/python3.10/site-packages (from transformers) (1.26.2)
Requirement already satisfied: packaging>=20.0 in ./.local/lib/python3.10/site-packages (from transformers) (23.2)
Requirement already satisfied: pyyaml>=5.1 in ./.local/lib/python3.10/site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in ./.local/lib/python3.10/site-packages (from transformers) (2023.3.23)
Requirement already satisfied: requests in ./.local/lib/python3.10/site-packages (from transformers) (2.31.0)
Requirement already satisfied: tokenizers<0.19,>=0.14 in ./.local/lib/python3.10/site-packages (from transformers) (0.15.2)
Requirement already satisfied: safetensors>=0.4.1 in ./.local/lib/python3.10/site-packages (from transformers) (0.4.2)
Requirement already satisfied: tqdm>=4.27 in ./.local/lib/python3.10/site-packages (from transformers) (4.66.1)
Requirement already satisfied: fsspec>=2023.5.0 in ./.local/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (2023.6.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.local/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.8.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.local/lib/python3.10/site-packages (from requests->transformers) (3.0.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers) (3.3)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./.local/lib/python3.10/site-packages (from requests->transformers) (1.26.17)
Requirement already satisfied: certifi>=2017.4.17 in ./.local/lib/python3.10/site-packages (from requests->transformers) (2023.7.22)
```

https://github.com/huggingface/transformers/blob/fc37f38915372c15992b540dfcbbe00a916d4fc6/src/transformers/models/gemma/modeling_gemma.py#L279



Comments:
oups, duplicated: https://github.com/huggingface/transformers/issues/29181


Closing as it's a duplicate :)