Add GGUF loading support for GLM-4.5 MoE (glm4moe)
Author: mironable
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/40042

## What
Adds GGUF loading support for glm4_moe (GLM-4.5 / GLM-4.5-Air), closing #40042. Previously raised `ValueError: GGUF model with architecture glm4moe is not supported yet`.

## Changes
- adds `glm4moe` config mapping (attention + MoE + grouped routing; `attention.key_length -> head_dim` to preserve the partial-rotary default and a GPT2-style tokenizer converter
- merges ffn_gate_exps + ffn_up_exps into stacked `mlp.experts.gate_up_proj`; maps `exp_probs_b.bias` -> `mlp.gate.e_score_correction_bias`, `glm4_moe`<->`glm4moe` arch aliasing, and a fixup dropping MTP layers from `num_hidden_layers` and inferring attention_bias/use_qk_norm from tensor presence
- adds `test_glm4moe_f16` logit-parity test

Routed-expert handling mirrors the existing MiniMax-M2 processor (same merged `gate_up_proj` layout).

## Coordination
Issue #40042; maintainer greenlight from @SunMarc. No open PR addresses it (the earlier "base model missing" comment is outdated; `glm4_moe` modeling now exists)

## Testing
Verified against a tiny GGUF, converted from `trl-internal-testing/tiny-Glm4MoeForCausalLM`, hosted at `mironableee/tiny-glm4moe-gguf`:
- gguf vs original state_dict: 0 missing / 0 extra, max abs weight diff 2.98e-08
- logit parity (committed test): max abs diff 0.0, assert_close passes

Test is `@slow` + `@require_torch_accelerator` (inherited), so it runs on GPU CI


## AI assistance
Claude code was used in diagnosis. I reviewed every changed line, understood the implementation and ran the tests below myself

## Fixture hosting
The fixture currently lives under my personal [HF namespace `mironableee/tiny-glm4moe-gguf`](https://huggingface.co/mironableee/tiny-glm4moe-gguf). Please let me know if you 'd like it moved to `hf-internal-testing`or similar.


## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, 
**we ask that new users do not submit pure code agent PRs** at this time. 
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously. 

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result, 
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [x] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://huggingface.co/docs/transformers/contributing) and the
      [Pull Request](https://huggingface.co/docs/transformers/pr_checks) checks?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case https://github.com/huggingface/transformers/issues/40042#issuecomment-4708141770
- [ ] Did you make sure to update the documentation with your changes according to the [guidelines](https://github.com/huggingface/transformers/tree/main/docs)?
- [x] Did you write any new necessary [tests](https://huggingface.co/docs/transformers/testing)?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ggml

fyi @SunMarc (you greenlit this on [#40042](https://github.com/mironable/transformers/issues/40042)) and @Cyrilvallez for the GGUF loading path; ready for review when you get a moment

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46718)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46718&sha=e88663

tests/models/kosmos2_5/test_modeling_kosmos2_5.py::Kosmos2_5ModelTest::test_generate_with_cache_matches_no_cache (AssertionError: Lists differ on generated token IDs). 
It's a Kosmos-2.5 generation/cache nondeterminism unrelated to this GGUF change since it auto-reran 5 times and still flaked. The fetcher pulled it in because the diff touches modeling_gguf_pytorch_utils.py. The actual test_glm4moe_f16 is @slow+@require_torch_accelerator and runs on the GPU CI. Could a maintainer rerun the job? cc @SunMarc 

#45977 gguf support in transformers will be changed in those pr. So i think we have to wait until those changes are done.

Thanks @minatoyukinaa. @SunMarc @ArthurZucker this just adds the glm4moe arch mapping to the existing GGUF loading path. Should i hold until #45977 lands and rebase on top, or is it ok to merge this small additive mapping independently? Whatever is less churn for you; i'll mark this as draft in the meantime