Repository: huggingface/transformers
Pull request: Support loading Gemma3 QAT GGUF models

# What does this PR do?

originally reported in vllm: https://github.com/vllm-project/vllm/issues/14723#issuecomment-2817093067
- Gemma3 QAT GGUF checkpoint doesn't have `general.basename` field in metadata
- This PR handles this edge case to avoid `KeyError: 'general.name'` raising


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@SunMarc @MekkCyber

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Changed files:
filename: src/transformers/modeling_gguf_pytorch_utils.py
patch: @@ -258,6 +258,8 @@ def process(self, weights, name, **kwargs):
 
 
 def read_field(reader, field):
+    if field not in reader.fields:
+        return []
     value = reader.fields[field]
     return [_gguf_parse_value(value.parts[_data_index], value.types) for _data_index in value.data]
 
@@ -369,6 +371,7 @@ def load_gguf_checkpoint(gguf_checkpoint_path, return_tensors=False, model_to_lo
     parsed_parameters = {k: {} for k in GGUF_TO_TRANSFORMERS_MAPPING}
 
     architecture = read_field(reader, "general.architecture")[0]
+    # NOTE: Some GGUF checkpoints may miss `general.name` field in metadata
     model_name = read_field(reader, "general.name")
 
     updated_architecture = None

filename: tests/quantization/ggml/test_ggml.py
patch: @@ -298,6 +298,7 @@ class GgufModelTests(unittest.TestCase):
     gemma2_model_id = "bartowski/gemma-2-2b-it-GGUF"
     original_gemma3_text_model_id = "google/gemma-3-1b-it"
     original_gemma3_vision_model_id = "google/gemma-3-4b-it"
+    gemma3_qat_model_id = "google/gemma-3-1b-it-qat-q4_0-gguf"
     gemma3_text_model_id = "unsloth/gemma-3-1b-it-GGUF"
     gemma3_vision_model_id = "unsloth/gemma-3-4b-it-GGUF"
 
@@ -329,7 +330,7 @@ class GgufModelTests(unittest.TestCase):
     q3_k_gemma2_model_id = "gemma-2-2b-it-Q3_K_L.gguf"
     q8_0_gemma2_model_id = "gemma-2-2b-it-Q8_0.gguf"
     fp32_gemma2_model_id = "gemma-2-2b-it-f32.gguf"
-    q2_k_gemma3_text_model_id = "gemma-3-1b-it-Q2_K.gguf"
+    q4_0_gemma3_qat_model_id = "gemma-3-1b-it-q4_0.gguf"
     bf16_gemma3_text_model_id = "gemma-3-1b-it-BF16.gguf"
     bf16_gemma3_vision_model_id = "gemma-3-4b-it-BF16.gguf"
 
@@ -889,19 +890,20 @@ def test_gemma2_weights_conversion_fp32(self):
             else:
                 raise ValueError(f"Layer {layer_name} is not presented in GGUF model")
 
+    @require_read_token
     @unittest.skipUnless(is_gguf_available("0.16.0"), "test requires gguf version >= 0.16.0")
-    def test_gemma3_text_q2_k(self):
+    def test_gemma3_qat_q4_0(self):
         model = AutoModelForCausalLM.from_pretrained(
-            self.gemma3_text_model_id,
-            gguf_file=self.q2_k_gemma3_text_model_id,
+            self.gemma3_qat_model_id,
+            gguf_file=self.q4_0_gemma3_qat_model_id,
             torch_dtype=torch.float16,
         )
 
-        tokenizer = AutoTokenizer.from_pretrained(self.gemma3_text_model_id, gguf_file=self.q2_k_gemma3_text_model_id)
+        tokenizer = AutoTokenizer.from_pretrained(self.gemma3_qat_model_id, gguf_file=self.q4_0_gemma3_qat_model_id)
         text = tokenizer(self.example_text, return_tensors="pt")["input_ids"]
         out = model.generate(text, max_new_tokens=10)
 
-        EXPECTED_TEXT = "Hello,\n\nI'm looking for a small,"
+        EXPECTED_TEXT = 'Hello with the prompt, "What is the best way'
         self.assertEqual(tokenizer.decode(out[0], skip_special_tokens=True), EXPECTED_TEXT)
 
     @require_read_token