Re-enable SDPA's FA2 path
Author: fxmarty
Pull request: yes

As per title, SDPA can't dispatch to FA2 in case the `attn_mask` argument is not `None`, which was always the case in recent releases following llama/gemma refactor, which led to memory issues https://github.com/huggingface/transformers/issues/30010.

In case we compile and use `fullgraph=True`, the graph is recaptured for different `q_len`, so having `is_causal=causal_mask is None and q_len > 1,` is fine

Reference https://github.com/pytorch/pytorch/issues/108108

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30070). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

```
FAILED tests/test_pipeline_mixin.py::TextGenerationPipelineTests::test_stop_sequence_stopping_criteria - AssertionError: Lists differ: [{'generated_text': 'Hello I believe in in in����������'}] != [{'generated_text': 'Hello I believe in fe fe fe fe fe fe fe fe fe fe fe fe'}]
```
is not failing locally, I don't get it. Besides, it is labeled as `tests_pipelines_tf`, not sure why

```
FAILED tests/models/whisper/test_modeling_whisper.py::WhisperModelTest::test_generate_with_prompt_ids_max_length - IndexError: index -1 is out of bounds for dimension 1 with size 0
```
is unrelated and also failing on main... Not sure why it appears here?

Other than that, Cohere/Llama/Gemma slow tests pass.

@ArthurZucker done, also running slow tests for llama/gemma/cohere gives

```
FAILED tests/models/code_llama/test_tokenization_code_llama.py::LlamaIntegrationTest::test_conversion - AssertionError: '{\n [226 chars]ed": true,\n      "special": true\n    },\n   [1796697 chars]}\n}' != '{\n [226 chars]ed": false,\n      "special": true...
FAILED tests/models/cohere/test_tokenization_cohere.py::CohereTokenizationTest::test_saving_tokenizer_trainer - pynvml.nvml.NVMLError_NotSupported: Not Supported
FAILED tests/models/llama/test_tokenization_llama.py::LlamaIntegrationTest::test_conversion - AssertionError: '{\n [964 chars]or": {\n    "type": "TemplateProcessing",\n   [1795198 chars]}\n}' != '{\n [964 chars]or": null,\n  "decoder": {\n    "t...
FAILED tests/models/recurrent_gemma/test_modeling_recurrent_gemma.py::RecurrentGemmaModelTest::test_cpu_offload - ValueError: model.normalizer doesn't have any device set.
FAILED tests/models/recurrent_gemma/test_modeling_recurrent_gemma.py::RecurrentGemmaModelTest::test_disk_offload_bin - ValueError: model.normalizer doesn't have any device set.
FAILED tests/models/recurrent_gemma/test_modeling_recurrent_gemma.py::RecurrentGemmaModelTest::test_disk_offload_safetensors - KeyError: 'model.normalizer'
FAILED tests/models/recurrent_gemma/test_modeling_recurrent_gemma.py::RecurrentGemmaIntegrationTest::test_2b_sample - AssertionError: Lists differ: ['Where is Paris ?\n\nChoose the word or phrase that is closest[463 chars]}}$'] != ['Where is Paris ?\n\nAnswer this quest...
FAILED tests/models/recurrent_gemma/test_modeling_recurrent_gemma.py::RecurrentGemmaIntegrationTest::test_model_2b_8bit - AssertionError: Lists differ: ['Hello I am doing a project on the topic o[162 chars]em>'] != ['<bos>Hello I am doing a project on the to[166 chars]<u>"]
```

which also fail on main on A100

@ArthurZucker 

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
import torch
from transformers.cache_utils import StaticCache

tokenizer = AutoTokenizer.from_pretrained(
    "meta-llama/Llama-2-7b-chat-hf", padding_side="left", pad_token="
"
)

with torch.device("cuda"):
    model = AutoModelForCausalLM.from_pretrained(
        "meta-llama/Llama-2-7b-chat-hf",
        torch_dtype=torch.float16,
        attn_implementation="sdpa",
    )

inputs = tokenizer(
    ["I would", "Today I am in Paris and", "I am"], padding=True, return_tensors="pt"
).to(model.device)

new_tokens = 10
gen_config = GenerationConfig(
    max_new_tokens=new_tokens,
    min_new_tokens=new_tokens,
    use_cache=True,
    pad_token_id=tokenizer.pad_token_id,
    num_beams=1,
    do_sample=False,
    eos_token_id=None,  # This is required for min_new_tokens to actually have an effect.
)
model.generation_config.eos_token_id = None  # greedy_search falls back on this eos_token_id that we need to set to None as well for min_new_tokens to have an effect.

print("----- GENERATE WITHOUT COMPILE")
gen_out = model.generate(**inputs, generation_config=gen_config, cache_implementation="static")

decoded = tokenizer.batch_decode(gen_out, skip_special_tokens=True)

print("decoded", decoded)

print("compiling...")

model.forward = torch.compile(model.forward, mode="reduce-overhead", fullgraph=True)
print("Finished compile call")

print("----- GENERATE WITH COMPILE")
gen_out = model.generate(**inputs, generation_config=gen_config, cache_implementation="static")
decoded = tokenizer.batch_decode(gen_out, skip_special_tokens=True)
print("decoded", decoded)

gen_out = model.generate(**inputs, generation_config=gen_config, cache_implementation="static")
decoded = tokenizer.batch_decode(gen_out, skip_special_tokens=True)
print("decoded", decoded)
```

works as expected as well (on 2.3 RC, see https://github.com/pytorch/pytorch/issues/121943 for 2.2.2, happening on main)