Mistral: CUDA error when generating text with a batch of inputs 
Author: plroit
Pull request: no

### System Info

I'm trying to decode a batch of outputs from a batch of inputs, with code that is working correctly with any encoder-decoder model (i.e. T5). I get the following error when I'm using Mistral:
` CUDA error: device-side assert triggered`
stack trace:
```python
File ~/miniconda3/lib/python3.11/site-packages/transformers/models/mistral/modeling_mistral.py:84, in MistralRMSNorm.forward(self, hidden_states)
     82 input_dtype = hidden_states.dtype
     83 hidden_states = hidden_states.to(torch.float32)
---> 84 variance = hidden_states.pow(2).mean(-1, keepdim=True)
     85 hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon)
     86 return self.weight * hidden_states.to(input_dtype)
```

- `transformers` version: 4.35.2
- Platform: Linux-5.4.0-122-generic-x86_64-with-glibc2.31
- Python version: 3.11.5
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: 0.24.1
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: MULTI_GPU
        - mixed_precision: no
        - use_cpu: False
        - debug: False
        - num_processes: 4
        - machine_rank: 0
        - num_machines: 1
        - gpu_ids: all
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
- PyTorch version (GPU?): 2.1.1+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: neither, single device


### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Example script to recreate:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
# required if I want a padded batch (Mistral does not define a padding token)
tokenizer.add_special_tokens({'pad_token': '[PAD]'})
device = "cuda"
model_inputs = tokenizer(["Hi there how are you? What's your name?", "Hi, sup?"], return_tensors="pt", padding=True).to(device)
generated_ids = model.generate(**model_inputs, max_new_tokens=10)
```


### Expected behavior

I should be able to use tokenizer.batch_decode on the outputs.

Comments:
Hey, you added a new token, this you need to resize the token embedding layer of the model with `model.resize_token_embeddings(len(tokenizer))`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.