Support loading Qwen3 MoE GGUF
Author: ctcanbol
Pull request: yes

# What does this PR do?

Currently, GGUF versions of Qwen3 MoE models raises "GGUF model with architecture qwen3moe is not supported yet" error. This PR resolves this issue.

Fixes #39721

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @SunMarc @MekkCyber


Comments:
Ready for review

run-slow: ggml

This comment contains run-slow, running the specified jobs: 

models: []
quantizations: ['quantization/ggml'] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ggml

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39638). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

There are still errors that report 

"ValueError: The checkpoint you are trying to load has model type `qwen3moe` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date." 
@ python3.11/site-packages/transformers/models/auto/configuration_auto.py, line 1271 in from_pretrained function.

When I run the Qwen3-235B-A22B-Thinking-2507-Q4_K_M.gguf file provided by unsloth.
The architecture of this file is `qwen3moe` which is different from `qwen3_moe` which is widely used in the configs.

There must have been a typo in this PR then. Would you like to fix this ? 

> There must have been a typo in this PR then. Would you like to fix this ?

I do not find the evidence that a typo exist.
But in gguf world the architecture name is 'qwen3moe', on otherside, in huggingface world it is 'qwen3_moe'.

Any suggestions for the fix? There is a fundamental incompability imo.

> Any suggestions for the fix? There is a fundamental incompability imo.

I think the same problem exist when qwen2moe and qwen2_moe system. 
I am not familiar with the architecture of transformers and vllm.

A patch might be in src\transformers\modeling_gguf_pytorch_utils.py file, like 392 line we could add qwen3moe related one.
Also,  it could be add at line 447 to replace the model_type only, if I remember right this could work.


https://github.com/huggingface/transformers/pull/39976 fixed here. Anyone can test is appreciated.