ModernBERT has been totally destroyed by PR #38974 and #38838
Author: rangehow
Pull request: no
Labels: name: bug

### System Info

transformers 4.54.0

### Who can help?

@zucchini-nlp @ArthurZucker 


### details

Regarding PR #38974, it modifies the model's default attention initialization behavior.

Referring to the ModernBERT code:
https://github.com/huggingface/transformers/blob/75794792ad6f23f09729674bc97a8338085f22b2/src/transformers/models/modernbert/modular_modernbert.py#L814-L832

The intention is to set the attention implementation to "FA2" if it is available and the user has not specified one.

The current situation is that this function is not even being called. Here is a simple script to reproduce it:

```python
from transformers import ModernBertConfig,ModernBertForMaskedLM

config = ModernBertConfig()

model = ModernBertForMaskedLM(config)


print(model.config._attn_implementation)
# sdpa

print(model._flash_attn_2_can_dispatch())
# You are attempting to use Flash Attention 2 without specifying a torch dtype. This might lead to unexpected behaviour
# You are attempting to use Flash Attention 2 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
# True
```

This behavior subtly degrades the experience for users of the ModernBERT model. In the previous version, if the user did not specify the attention mechanism, ModernBERT would automatically use FA2 attention (if available). Now, they might get a significant performance degradation just by simply updating their transformers version, because the ModernBERT team also carefully designed an unpadding process for the FA2 path.

Furthermore, modernbert is not a model that will function correctly simply by posteriorly setting model.config._attn_implementation, because its components, such as rotary embedding, can even differ depending on the attention implementation.

Comments:
https://github.com/huggingface/transformers/blob/896e9cea1ade521b2648f4798218550f6c72190c/src/transformers/modeling_utils.py#L2035

btw, `_check_attn_implementation`  and `super().set_attention_implementation` in https://github.com/huggingface/transformers/blob/75794792ad6f23f09729674bc97a8338085f22b2/src/transformers/models/modernbert/modular_modernbert.py#L814-L832 
is undefined..

<img width="2928" height="1360" alt="Image" src="https://github.com/user-attachments/assets/30811ba4-b285-4ecf-9eac-1d7b65281054" />

To be honest, isn't submitting this many breaking PRs getting a bit out of hand?
[https://github.com/huggingface/transformers/pull/38838](https://www.google.com/url?sa=E&q=https%3A%2F%2Fgithub.com%2Fhuggingface%2Ftransformers%2Fpull%2F38838)
This PR introduces another issue. It removes the default value for rope_theta. As a result, in fa2 attention mode, rope_theta is only defined when local attention is used. This causes a direct error on initialization, even if you try to hard-code the attention mode in the config.

<img width="2972" height="1194" alt="Image" src="https://github.com/user-attachments/assets/2c89af43-b79e-403e-aa5c-794e23864c1d" />

Sorry that this got through! We'll need to add tests as it's hard to make sure our code is up to the standards without updating anything! 
We'll do a patch with this!