support loading qwen3 gguf
Author: 44670
Pull request: yes

# What does this PR do?

This PR adds gguf loading support for qwen3 models.


## Before submitting
- [No] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [Yes] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [No afaik] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [Yes] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [No] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
@SunMarc  Hi, can you take a look at this PR?

Thanks! 

@44670, Hi, may I know if you are still working on this issue? Thanks!

> @44670, Hi, may I know if you are still working on this issue? Thanks!

Yes! I am working on this and will be update this PR soon.

> Thanks ! Can you add some tests like the other gguf models ?

Hi @SunMarc !

I have added the testcase.

Hey, I came across this PR and now curious to know if your changes cover qwen3moe support as well? I am having following error with the latest transformers package, and was wondering if you are covering this as well:

```
File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/vllm/config.py", line 515, in __post_init__
    hf_config = get_config(self.hf_config_path or self.model,
  File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/vllm/transformers_utils/config.py", line 310, in get_config
    config_dict, _ = PretrainedConfig.get_config_dict(
  File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/transformers/configuration_utils.py", line 595, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/transformers/configuration_utils.py", line 686, in _get_config_dict
    config_dict = load_gguf_checkpoint(resolved_config_file, return_tensors=False)["config"]
  File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/transformers/modeling_gguf_pytorch_utils.py", line 407, in load_gguf_checkpoint
    raise ValueError(f"GGUF model with architecture {architecture} is not supported yet.")
ValueError: GGUF model with architecture qwen3moe is not supported yet.
```

There was another issue reported in vllm repo, a comment was saying qwen3moe was missing in one of the files you have updated in your PR: https://github.com/vllm-project/vllm/issues/18382#issuecomment-2907810276

> Hey, I came across this PR and now curious to know if your changes cover qwen3moe support as well? I am having following error with the latest transformers package, and was wondering if you are covering this as well:
> 
> ```
> File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/vllm/config.py", line 515, in __post_init__
>     hf_config = get_config(self.hf_config_path or self.model,
>   File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/vllm/transformers_utils/config.py", line 310, in get_config
>     config_dict, _ = PretrainedConfig.get_config_dict(
>   File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/transformers/configuration_utils.py", line 595, in get_config_dict
>     config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
>   File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/transformers/configuration_utils.py", line 686, in _get_config_dict
>     config_dict = load_gguf_checkpoint(resolved_config_file, return_tensors=False)["config"]
>   File "/home/user/.pyenv/versions/3.10.17/lib/python3.10/site-packages/transformers/modeling_gguf_pytorch_utils.py", line 407, in load_gguf_checkpoint
>     raise ValueError(f"GGUF model with architecture {architecture} is not supported yet.")
> ValueError: GGUF model with architecture qwen3moe is not supported yet.
> ```
> 
> There was another issue reported in vllm repo, a comment was saying qwen3moe was missing in one of the files you have updated in your PR: [vllm-project/vllm#18382 (comment)](https://github.com/vllm-project/vllm/issues/18382#issuecomment-2907810276)

Hi, unfortunately `qwen3moe` should be a different model type and maybe out of scope for now.


bump

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38645). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.