Repository: huggingface/transformers
Pull request: Gemma3 processor typo

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #36701  (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->

---

fix a typo that may lead to inconsistent output from the `gemma3.processor` when `do_pan_and_scan` is True

I made a simple test following:
```python
from transformers import AutoProcessor, Gemma3ForConditionalGeneration
from PIL import Image

model_path = "./gemma-3-4b-it"

model = Gemma3ForConditionalGeneration.from_pretrained(
    model_path, device_map="auto"
).eval()

processor = AutoProcessor.from_pretrained(model_path)

image = Image.open("./bee.jpg") # download from "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"

crop_config ={
    'images_kwargs': {
        'do_pan_and_scan': True, # False
        "pan_and_scan_max_num_crops": 448,
        "pan_and_scan_min_crop_size": 32,
        "pan_and_scan_min_ratio_to_activate": 0.3,
        }
    }

inputs = processor(images=[image], text=f"{processor.boi_token} Describe this image in detail.", return_tensors="pt", **crop_config).to('cuda:0')

generate_ids = model.generate(**inputs, max_length=2048)

response = processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
print(response)

```
I got different responses below:
when `do_pan_and_scan=False`
```
This is a close-up photograph of a vibrant pink cosmos flower in a garden setting. The flower is the focal point, with a fuzzy bumblebee actively collecting pollen on its petals. The bee is dark with yellow stripes, and its body is covered in a light dusting of pollen. 

The cosmos flower has five delicate, layered petals, creating a star-like shape. The petals are a soft, pale pink color, and the center of the flower is a bright yellow. Several other cosmos flowers are visible in the background, some in full bloom and others beginning to wilt. 

The background is a mix of green foliage, including broad leaves and smaller green plants. A small red flower is also visible in the lower right corner of the image. The lighting is soft and natural, creating a warm and inviting atmosphere. The overall impression is one of a peaceful and thriving garden scene. 

The depth of field is shallow, with the foreground flower and bee in sharp focus, while the background is slightly blurred. This technique helps to draw the viewer's attention to the main subject of the photograph.
``` 

when `do_pan_and_scan=True` 

```
Here's a detailed description of the image:

**Overall Impression:**

The image is a close-up shot of a small patch of flowers in a garden setting. The focus is on several pink cosmos flowers, with a bee actively visiting one of them. The overall mood is slightly melancholic due to the presence of some withered and dying flowers.

**Detailed Breakdown:**

*   **Flowers:** The dominant elements are the pink cosmos flowers. They are in various stages of bloom – some are still vibrant and open, while others are drooping and beginning to wilt. The petals are a soft, pale pink color.
*   **Bee:** A bee is prominently featured on one of the cosmos flowers, actively collecting pollen. It's a dark-colored bee, likely a bumblebee, and its presence suggests the flowers are still productive.
*   **Withered Flowers:** Several flowers are clearly past their prime, with brown, dried petals and stems. This adds a touch of decay and suggests the end of the blooming season.
*   **Background:** The background is a blurred mix of green foliage, indicating a garden or natural setting. A red flower is visible in the background, adding a pop of color.
*   **Composition:** The image is tightly cropped, focusing on the flowers and the bee. The composition is somewhat chaotic, with the flowers overlapping and the bee positioned in the center of one of the blooms.

**Color Palette:**

The color palette is dominated by pinks and greens, with accents of brown (from the withered flowers) and red (from the background flower).

**Overall Tone:**

The image evokes a sense of the fleeting beauty of nature, with a reminder of the cycle of life and death. The presence of the bee amidst the dying flowers suggests resilience and the continuation of the natural world.

Do you want me to focus on a specific aspect of the image, such as the bee, the flowers, or the overall composition?
``` 

cc @zucchini-nlp 

Changed files:
filename: src/transformers/models/gemma3/image_processing_gemma3.py
patch: @@ -384,7 +384,7 @@ def preprocess(
             images_list = [images for images, _ in images_list_and_num_crops]
             num_crops = [num_crops for _, num_crops in images_list_and_num_crops]
         else:
-            num_crops = [[0] for images in images_list]
+            num_crops = [[0] for _ in images_list]
 
         processed_images = []
         for images in images_list:

filename: src/transformers/models/gemma3/processing_gemma3.py
patch: @@ -113,7 +113,6 @@ def __call__(
 
             # Replace image tokens by the full expanded sequence
             batch_num_crops = to_py_obj(image_inputs.pop("num_crops"))
-            text_with_crops = text
             for batch_idx, (prompt, images, num_crops) in enumerate(zip(text, batched_images, batch_num_crops)):
                 image_indexes = [m.start() for m in re.finditer(self.boi_token, prompt)]
 
@@ -130,7 +129,7 @@ def __call__(
                             + " ".join([self.boi_token] * num)
                         )
                         prompt = prompt[:idx] + formatted_image_text + prompt[idx + len(self.boi_token) :]
-                        text_with_crops[batch_idx] = prompt
+                        text[batch_idx] = prompt
 
             # Expand placeholder image tokens to the full image token sequence
             text = [prompt.replace(self.boi_token, self.full_image_sequence) for prompt in text]

filename: tests/models/gemma3/test_modeling_gemma3.py
patch: @@ -417,6 +417,39 @@ def test_model_4b_batch(self):
         ]  # fmt: skip
         self.assertEqual(output_text, EXPECTED_TEXTS)
 
+    def test_model_4b_crops(self):
+        model_id = "gg-hf-g/gemma-3-4b-it"
+
+        model = Gemma3ForConditionalGeneration.from_pretrained(
+            model_id, low_cpu_mem_usage=True, torch_dtype=torch.bfloat16
+        ).to(torch_device)
+
+        crop_config = {
+            "images_kwargs": {
+                "do_pan_and_scan": True,
+                "pan_and_scan_max_num_crops": 448,
+                "pan_and_scan_min_crop_size": 32,
+                "pan_and_scan_min_ratio_to_activate": 0.3,
+            }
+        }
+
+        inputs = self.processor.apply_chat_template(
+            self.messages,
+            tokenize=True,
+            return_dict=True,
+            return_tensors="pt",
+            add_generation_prompt=True,
+            **crop_config,
+        ).to(torch_device)
+
+        output = model.generate(**inputs, max_new_tokens=30, do_sample=False)
+        output_text = self.processor.batch_decode(output, skip_special_tokens=True)
+
+        EXPECTED_NUM_IMAGES = 3  # one for the origin image and two crops of images
+        EXPECTED_TEXTS = ["user\nYou are a helpful assistant.\n\nHere is the original image \n\n\n\n and here are some crops to help you see better \n\n\n\n \n\n\n\nDescribe this image in detail.\nmodel\nHere's a detailed description of the image:\n\n**Overall Impression:**\n\nThe image is a close-up shot of a garden scene featuring several"]  # fmt: skip
+        self.assertEqual(len(inputs["pixel_values"]), EXPECTED_NUM_IMAGES)
+        self.assertEqual(output_text, EXPECTED_TEXTS)
+
     def test_model_4b_multiimage(self):
         model_id = "gg-hf-g/gemma-3-4b-it"