IDEFICS2: explicit error if SDPA is requested
Author: gante
Pull request: yes

# What does this PR do?

1. Raises an informative exception if we specify `attn_implementation="sdpa"` while loading IDEFICS2 -- the model has partial support, some components don't support SDPA
2. Relaxes `test_eager_matches_sdpa_generate` to handle partial SDPA support

The following command is fixed in this PR:
```
RUN_SLOW=1 python3 -m pytest -v --flake-finder --flake-runs=1 tests/models/idefics2/test_modeling_idefics2.py::Idefics2ForConditionalGenerationModelTest -k "( test_eager_matches_sdpa_generate) or (test_initialization)"
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34529). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp If we call 

```py
from transformers import Idefics2ForConditionalGeneration
import torch

model = Idefics2ForConditionalGeneration.from_pretrained(
    "HuggingFaceM4/idefics2-8b-base",
    torch_dtype=torch.bfloat16,
    attn_implementation="sdpa",
    device_map="auto",
)
```

We get the same exception as triggered by the tests

```
ValueError: Idefics2VisionTransformer does not support an attention implementation through torch.nn.functional.scaled_dot_product_attention yet. Please request the support for this architecture: https://github.com/huggingface/transformers/issues/28005. If you believe this error is a bug, please open an issue in Transformers GitHub repository and load your model with the argument `attn_implementation="eager"` meanwhile. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="eager")`
```

This means we can't specify `attn_implementation="sdpa"` and, therefore, the standard API is broken unless we set `_supports_sdpa = False`.

On the other hand, I understand what you're saying: a block within IDEFICS2 may be able to support SDPA. However, it only works when we DON'T set `attn_implementation`. 

Here's what I'm going to do, to preserve the feature while not blocking tests:
1 - keep `_supports_sdpa` as before
2 - raise an exception is a user passes `attn_implementation="sdpa"`

In the long run, IDEFICS2 should get a SDPA implementation on its own attention layers. It's not great practice to partially enable a flag.

@gante I see, thanks for details! That is indeed the expected behavior from the last release, where we started setting the user-requested attn implementation on all sub-models. I realize this is new and poorly documented so it is a source of confusion now. I am thinking of what would be the best way to document the feature and maybe any other changes in composite models

In fact the "sdpa" is expected to fail but that also means the test will fail for all models that support SDPA partially. I propose for now maybe skip the test for Idefics2 and then think about how we can make better tests with attn implementation in case of composite models. WDYT?

> In fact the "sdpa" is expected to fail but that also means the test will fail for all models that support SDPA partially. I propose for now maybe skip the test for Idefics2 and then think about how we can make better tests with attn implementation in case of composite models. WDYT?

Yes, agreed :) I'm about to commit a potential solution for you to check

@zucchini-nlp ready for a re-check :)

> Not sure about the error though, as we would anyway throw error that Idefics2VisionModel has no SDPA attn support

@zucchini-nlp in the original exception, it says that `Idefics2VisionModel` doesn't support SDPA, and suggests loading the \*top-level\* model with `attn_implementation="eager"` because the model has no SDPA support. 

Here we have a different case, where it partially supports SDPA if (and only if) the user doesn't specify `attn_implementation`. The new exception explains why and what can be done to get SDPA in the internal text model 🤗 

If we allow partial SDPA, the original exception doesn't tell the full story.

(closing, as arthur wrote this brings complexity with little benefit)