Wrong shape of  last layer hidden states when generating
Author: sunxiaojie99
Pull request: no

### System Info

transformers version: 4.38.0
Python version: 3.8.18
PyTorch version (GPU): 2.0.0+cu117

### Who can help?

@gante 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Hello! I have a question regarding this [issue.](https://github.com/huggingface/transformers/issues/26174)
If I aim to retrieve the last layer hidden states for all tokens, accounting for a total of 30 tokens (7 input tokens and 23 output tokens), it seems that the number of accessible hidden states is 29. This total comprises 7 hidden states from the initial generated token and 22 from the subsequent generated tokens. Is it not possible to obtain the hidden state for the final token, possibly the eos_token?

like this:
```
from transformers import MistralForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('mistralai/Mistral-7B-v0.1')
model = MistralForCausalLM.from_pretrained('mistralai/Mistral-7B-v0.1')
prompt = "Hey, are you conscious?"
inputs = tokenizer.encode(prompt, return_tensors="pt")  # shape=[1, 7]

output_sequences = model.generate(input_ids=inputs, max_length=30, output_attentions=True, output_hidden_states=True, return_dict_in_generate=True)

input_hidden = output_sequences.hidden_states[0][-1]  # hidden states of input prompt
output_hidden = torch.cat([item[-1].unsqueeze(0) for item in output_sequences.hidden_states[1:]], dim=0)  # hidden states of generate text

print(input_hidden.shape)  # [bs, 7, dim]
print(output_hidden.shape)  # [22, bs, 1, dim]
```

### Expected behavior

input_hidden: [bs, 7, dim]
output_hidden shape: [23, bs, 1, dim]

Comments:
I am getting `len(output_sequences.hidden_states)` `23` so not sure I understand what's going on here

> I am getting `len(output_sequences.hidden_states)` `23` so not sure I understand what's going on here

Yes, if I print detail shapes, the length of output_sequences is 23, as shown below:
```
for generated_token_index, hidden_states in enumerate(output_sequences.hidden_states):
    print(f"Generated token index: {generated_token_index}, last layer hidden state shape: {hidden_states[-1].shape}")
```
the output is:
```
Generated token index: 0, last layer hidden state shape: torch.Size([1, 7, 4096])
Generated token index: 1, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 2, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 3, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 4, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 5, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 6, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 7, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 8, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 9, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 10, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 11, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 12, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 13, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 14, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 15, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 16, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 17, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 18, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 19, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 20, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 21, last layer hidden state shape: torch.Size([1, 1, 4096])
Generated token index: 22, last layer hidden state shape: torch.Size([1, 1, 4096])
```

There are a total of 7+22=29 token hidden states, but the expected behavior is that we can obtain "30" token hidden states because there are "30" tokens (7 input tokens, 23 output tokens)

Hi @sunxiaojie99 👋 

If you have 30 tokens at the end of generation, you'll always have 29 hidden states :) 

The token with index N is used to produce hidden states with index N, which is then used to get the token with index N+1. The generation ends as soon as the target number of tokens is obtained so, when we obtain the 30th token, we don't spend compute to get the 30th set of hidden states. You can, however, manually run an additional forward pass to obtain the 30th set of hidden states, corresponding to the 30th token and used to obtain the 31st token.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> Hi [@sunxiaojie99](https://github.com/sunxiaojie99) 👋
> 
> If you have 30 tokens at the end of generation, you'll always have 29 hidden states :)
> 
> The token with index N is used to produce hidden states with index N, which is then used to get the token with index N+1. The generation ends as soon as the target number of tokens is obtained so, when we obtain the 30th token, we don't spend compute to get the 30th set of hidden states. You can, however, manually run an additional forward pass to obtain the 30th set of hidden states, corresponding to the 30th token and used to obtain the 31st token.

Does the output_sequences.hidden_states [0] has all input hidden state? Or only output id 0 hidden state