[generate] Add proper MTP support
Author: Cyrilvallez
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46229)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46229)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

As per the title! Finally add MTP decoding to Transformers!
For models supporting it and having proper trained weights, it can be used as simply as:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zai-org/GLM-4.5-Air"

model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

input_text = "Hello, who are you and what can you do?"
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)

# Simply pass the kwarg `use_mtp=True` to use it!
out = model.generate(**inputs, use_mtp=True, do_sample=False, max_new_tokens=100)
output_text = tokenizer.batch_decode(out[:, inputs.input_ids.shape[1]:])[0]
print(output_text)
```

It will automatically scan the repo corresponding to the model calling `generate`, and load the weights corresponding to the MTP layers.

Note that it integrates nicely into the existing `_assisted_decoding`, thus avoiding to add yet another decoding method into `generate`.

## Benchmark and expected perfs

On the previous example, I benchmarked the speed (excluding loading time of the MTP weights of course) and token matching, and obtained an acceptance rate of 73.68% for the drafted tokens from the (unique) MTP layer, resulting in a decoding speedup ratio of about 1.4, or 40% faster decoding compared to not using MTP.
This is of course a single test on a small toy example, so proper benchmarking may be needed, but it gives a good idea of what we can expect in general I believe.

## Limitations

Note that this is unfortunately NOT supported for the recent Deepseek v4, as they changed the naming/conventions of the MTP layers (they mostly split the `eh_proj` layer in `e_proj` and `h_proj`, and use quantization on them). It should be relatively straightforward to fix it and make the weights match with simple `ConversionOps`, either on the general conversion mapping or on a model attribute such as `_mtp_conversions` or something.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46229). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Cyrilvallez — following your `add-mtp` branch closely. The training side for Qwen3.5 MTP doesn't exist yet in the repo.

**A bit of context:** I started prototyping this back in [#45638](https://github.com/huggingface/transformers/pull/45638) with an inline `Qwen3_5MTP` module and `mtp_num_hidden_layers` config — but that design created duplication with your inference-side `MtpLayer`. So I refactored it to align with #46229:

- Uses `from ...generation.mtp import MtpLayer` (same module as inference)
- Config field `num_nextn_predict_layers` matches your naming
- Model returns raw `mtp_loss`; `Trainer` applies `mtp_loss_coef` weighting
- ~+500 LOC on top of #46229


Hey @ArthurZucker @vasqu thanks for the reviews! I believe I answered all comments, let me know if something is unclear.
In general, I agree that this is mostly inference-centric, however the forward should be usable for (at least simple cases) of training as well - it would only require adding the proper additional loss, but the `forward` itself would not change. In general, they go a bit hand-in-hand, as later layers need to see both the last hidden states AND the new drafted token (which is basically inference), so the line is a bit blurry here. 
More complicated training would require quite a bit of additional efforts though - for example packed sequence training. Since we need to draft tokens and update the position_ids in the main mtp loop, packing format is not so straightforward.
Detection of exact layer type and norms for the mtp layers is also not so clear all the time, as @vasqu noted in the case of hybrid archs. The only alternative I see here however would be to have per-model mtp implementations, living in the main model file. This would allow explicit classes and variations, but will lead to duplication. Another strategy could be to use a `GenericForMTP` or something, implementing the forward in general, and then inherit from this and simply add the proper `__init__` to describe exact layers and norms in each model.
Let me know what you think!

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46229&sha=862db4

Awesome work overall!!

I think it would be nice to provide the basic training path for MTP then. Iiuc, it's just the "missing" labels in the forward. While they may be similar, I want to avoid that we encounter something unexpected. I don't think packed sequences are a big priority for now tbh so let's keep it simple for sure.

For the specific new patterns (e.g. hybrid linear attention). I think it makes sense to have the general implementation ("normal" attention) which we override to allow more specific patterns (hybrid linear attention). I would expect all hybrids to follow a similar pattern there so that we won't need the model specifics(?) and only variations of MTP if that makes sense

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deepseek_v3, glm4_moe, glm4v_moe, mimo_v2_flash, solar_open, youtu

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46229)
**Latest run:** [29232044411:1](https://github.com/huggingface/transformers/actions/runs/29232044411)
**Result:** `success` | **Jobs:** 15 | **Tests:** 169,586 | **Failures:** 0 | **Duration:** 16h 8m

<!-- ci-dashboard-recap:end -->