Transformers 5.0.0 breaks custom model initialization
Author: umarbutler
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.0.0
- Platform: Linux-6.14.0-37-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 1.3.5
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0+cu128 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition

### Who can help?

@CyrilVallez

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

As shown in the MRE below, after initializing a custom model with a custom `_init_weights()` function, then filling a weight with a value, which is intended to simulate that weight having updated after training, and then loading the model again with `from_pretrained()`, it seems that the custom initialization function is being run again when it absolutely obviously should not be run again, causing the altered value to be overwritten.

```python
import torch

from transformers import PreTrainedModel, ModernBertConfig


class MyCustomModel(PreTrainedModel):
    config_class = ModernBertConfig
    all_tied_weights_keys = dict()

    def __init__(self, config: ModernBertConfig) -> None:
        # Init.
        super().__init__(config)

        # Define layers.
        self.classifier: torch.nn.Linear = torch.nn.Linear(1024, 2)
        self.post_init()

    def _init_weights(self, module):
        super()._init_weights(module)

        self.classifier.weight.data.fill_(0.01)


model = MyCustomModel(ModernBertConfig())
model.classifier.weight.data.fill_(0.05)
print(f"The weight after 'training' it: {model.classifier.weight.data[0][0]}")
model.save_pretrained("temp_model")
loaded_model = MyCustomModel.from_pretrained("temp_model")
print(f"The weight after loading should be `0.05` not `0.01`: {loaded_model.classifier.weight.data[0][0]}")
```

### Expected behavior

Custom initialization code should not be overwriting already trained weights when loading from a checkpoint.

Comments:
Hey! Indeed due to a change in how we reinit the parameters/buffers, you now need to use `torch.nn.init.constant_(param, value)` or `transformers.initialization.constant_(param, value)`, as we check a flag on the param itself to determine if it should be re-initialized.

Thanks for the clarification @Cyrilvallez 

I checked the v5 code and the re-init logic is based on the `_is_hf_initialized` flag, which is set in `_initialize_weights`:

```python
def _initialize_weights(self, module):
    if getattr(module, "_is_hf_initialized", False):
        return
    self._init_weights(module)
    module._is_hf_initialized = True
```
Direct tensor mutation (e.g. `param.data.fill_()`) does not set this flag, so the param is treated as uninitialized and _init_weights runs again when loading.

Even though this is intentional, I think a small DX improvement would help:

- a warning when .data is modified in _init_weights without marking the param initialized, and/or
- a short documentation note recommending torch.nn.init.*.

I’d be happy to open a PR for this if that’s helpful.

The flag is set both on weights directly and on modules. It's basically impossible to emit a warning when `.data` is used unfortunately, otherwise we would of course have done it 🥲