[`Mistral`] Add Flash Attention-2 support for `mistral`
Author: younesbelkada
Pull request: yes

# What does this PR do?

Adds Flash Attention 2 for Mistral For Causal - we still need to discuss how to integrate it with local attention

cc @ArthurZucker @LysandreJik 

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    use_flash_attention_2=True,
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True
).to(0)

tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")

text = "Hello my name is"
inputs = tokenizer(text, return_tensors="pt").to(0)

out = model.generate(**inputs, max_new_tokens=4096, use_cache=True, do_sample=True)
print(tokenizer.batch_decode(out, skip_special_tokens=True))
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Might be worth adding support for sliding window attention ? 
params are window_size_left = config.sliding_window-1 // window_size_right = -1 I believe ? 
See this PR adding support in flashattentionv2
https://github.com/Dao-AILab/flash-attention/commit/083e8f525f1f8e1dde5044afdac79f9588302207

Sharing some results here vs mistral official implementation that uses `xformers.memory_efficient_attention`: 

##  Context length = 12 / max_new_tokens=512 / bs=1

HF transformers + FA-2
```
Latency: 15.1241201171875
33.85320904838279 tokens / s
Max allocated memory: 15218032640
```
Mistral + mem efficient:
```
Latency: 17.23331640625
29.709893785407036 tokens / s
Max allocated memory: 14636799488
```

## Context length = 11K / max_new_tokens=512 / bs=1

HF transformers + FA-2
```
Latency: 16.497216796875
31.03553807312431 tokens / s
Max allocated memory: 18673463808 
```
Mistral + mem efficient:
```
Latency: 22.50997265625
22.74547409802565 tokens / s
Max allocated memory: 17303250944
```

## Context length = 11K / max_new_tokens=512 / bs=2 with 11K padding tokens on the second batch

HF transformers + FA-2
```
Latency: 33.95778515625
15.077544004832287 tokens / s
Max allocated memory: 22320273408
```
Mistral + mem efficient:
```
Latency: 30.407841796875
16.83776189774238 tokens / s
Max allocated memory: 17841224192
```

## Context length = 11K / max_new_tokens=512 / bs=4 with 11K padding tokens on the second, third and fourth batch

HF transformers + FA-2
```
Latency: 48.86058984375
10.478792860203109 tokens / s
Max allocated memory: 29610738688
```
Mistral + mem efficient:
```
Latency: 45.27477734375
11.308724858272097 tokens / s
Max allocated memory: 18914968576
```

--> obviously the pad / unpad overhead takes it over for the HF implementation whereas the official repository deals with padding tokens differently. Note also that the max allocated memory increases if one adds padding token. Also note the current cache slicing mechanism assumes users are under `padding=left` regime. Generation should be performed with padding_side=left whereas this should have no impact for training as the cache is not used during training.

Here is a plot that compares pure forward on HF native vs HF + FA-2

<img width="957" alt="Screenshot 2023-10-02 at 17 49 12" src="https://github.com/huggingface/transformers/assets/49240599/336b5d1e-9ded-489f-a71e-8fdf75f04e0c">


<img width="957" alt="Screenshot 2023-10-02 at 17 49 06" src="https://github.com/huggingface/transformers/assets/49240599/35ec292d-19a1-43fd-a680-87b5ec1774f1">



For the sake of completeness,

Script I used to benchmark transformers + FA2: https://gist.github.com/younesbelkada/691c1dec3da2f0a7de29c1d1096d860f 

Script I used to benchmark mistral original source code: https://gist.github.com/younesbelkada/ada0d9c2c48ab034486dbaaf95d29fae (assuming you have cloned their repository and run it under the root folder of the repo)