Fix accelerate hooks for all models using linear attention
Author: Cyrilvallez
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46978)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46978)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

Fixes https://github.com/huggingface/transformers/pull/46831.
It has been a long standing issue that`accelerate` is a problem for anything that uses weights without `forward`, or even direct `nn.Parameter` sometimes. For offloading for example, the hooks will not fire because the weights are used as-is in the function, with `F.conv1d` or the kernel, instead of through `forward` which would fire the hooks correctly. So the offloaded weights would stay on meta and result in silently wrong results.
This is kind of an "ugly" patch to fix it for all linear attention models for the moment, while we can work on removing dependence on accelerate, and improving hook dispatch (the issue was known, and one of my main motivation for the long-standing goal of removing/reworking `accelerate`).

Easy test code:

```python
import gc
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen3.5-0.8B"
input_text = "Hello, who are you?"
KWARGS = [
    {"device_map": 0},
    {"device_map": "auto", "max_memory": {0: "700MiB", "cpu": "12GiB"}}
]
tokenizer = AutoTokenizer.from_pretrained(model_id)
inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": input_text}],
    tokenize=True,
    return_tensors="pt",
    add_generation_prompt=True,
)

model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float16, **KWARGS[0])

out = model.generate(**inputs.to(model.device), max_new_tokens=20, do_sample=False)
output_text = tokenizer.batch_decode(out[:, inputs["input_ids"].shape[1]:])[0]
print(f"Output with GPU:\n{output_text}")

del model
gc.collect()

model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float16, **KWARGS[1])

out = model.generate(**inputs.to(model.device), max_new_tokens=20, do_sample=False)
output_text = tokenizer.batch_decode(out[:, inputs["input_ids"].shape[1]:])[0]
print(f"Output with offloading:\n{output_text}")

```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46978). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bamba, falcon_h1, falcon_mamba, granitemoehybrid, jamba, lfm2, lfm2_moe, mamba, nemotron_h, qwen3_5, qwen3_5_moe, qwen3_next, zamba2

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46978)
**Latest run:** [28500539984:1](https://github.com/huggingface/transformers/actions/runs/28500539984)
**Result:** `failure` | **Jobs:** 13 | **Tests:** 44,898 | **Failures:** 0 | **Duration:** 15h 37m

<!-- ci-dashboard-recap:end -->