[glm-moe-dsa] Indexer uses interleaved rope
Author: pcuenca
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46842)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46842)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

~~This is a proof-of-concept for discussion, the real fix will be more general (read below).~~

History as I understand it:

- glm-moe-dsa was added in #43858. We added `indexer_rope_interleave` to [the config](https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L25), but it [was not implemented](https://github.com/huggingface/transformers/pull/43858/changes#diff-9e89bc97ac5f62ab9c3dca09e5b0682af597069035827e9c342ba22b64dce823R265) (we defaulted to full attention).
- Interleaved RoPE was added in #46372. [The Indexer was set to always use it](https://github.com/huggingface/transformers/pull/46372/changes#diff-9e89bc97ac5f62ab9c3dca09e5b0682af597069035827e9c342ba22b64dce823R416) (the config property was still ignored).
- The #41251 DeekSeek refactor made [GlmMoeDsaIndexer identical to DeepseekV32Indexer](https://github.com/huggingface/transformers/pull/41251/changes#diff-9e89bc97ac5f62ab9c3dca09e5b0682af597069035827e9c342ba22b64dce823R136). Property still not in use.

Another data point:
- [vLLM reads the property](https://github.com/vllm-project/vllm/blob/31ca9504b14c328291fda1ad1a2659180ae8eb34/vllm/model_executor/models/deepseek_v2.py#L1034)

## How this manifests in real-life

I'm testing again to exclude the transformers branch as a confounder; I think the result is a not-obvious quality degradation for long sequences.

Confirmed: generation degrades as sequence length increases.

~~Before this PR, I observed subtle degradation in long-context generation, but not obvious signals such as corrupted text. For example, asking to generate a space invaders game I saw syntax errors like in the following line (`0.0.4` is not a number): `this.tone(120, 0.0.4, 'sawtooth', 0.1, -80);`~~

~~These errors did not happen with the fix in this PR. But I can't still rule out the reason might have been different (my "bad output" test was run before the [on-going FP8/parallel fixes](https://github.com/huggingface/transformers/pull/46818)).~~

## Real Fix

If this is indeed a bug, I think the real fix should be to read `indexer_rope_interleave` in the DS implementation and act accordingly.

## Code Agent Policy

- [x] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://huggingface.co/docs/transformers/contributing) and the
      [Pull Request](https://huggingface.co/docs/transformers/pr_checks) checks?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes according to the [guidelines](https://github.com/huggingface/transformers/tree/main/docs)?
- [ ] Did you write any new necessary [tests](https://huggingface.co/docs/transformers/testing)?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46842). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Testing confirms the issue. I'll apply the fix to modular.

This is ready for review now.

> Thanks, let's avoid having that flag if its set in stone that DSv32 use interleaved and dsa does not

The property [exists in GLM models](https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L25). I will hardcode for now, we can revisit if/when more models use DSA.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm_moe_dsa

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46842)
**Latest run:** [28233301509:1](https://github.com/huggingface/transformers/actions/runs/28446840921)
**Result:** `success` | **Jobs:** 6 | **Tests:** 650 | **Failures:** 0 | **Duration:** 9m 22s

<!-- ci-dashboard-recap:end -->

Thanks a lot for the patience @ArthurZucker, merging now then!