Repository: huggingface/transformers
Pull request: Fix default image_rows and image_cols initialization in Idefics3 and SmolVLM processors

# What does this PR do?

This PR fixes a bug in the default initialization of `image_rows` and `image_cols` in the Idefics3 and SmolVLM processors.

## Problem

The original code incorrectly used `len(text)` to determine the size of the default `image_rows` and `image_cols` lists:
```python
image_rows = inputs.pop("rows", [[0] * len(text)])
image_cols = inputs.pop("cols", [[0] * len(text)])
```

This creates a single list with length equal to the number of text samples in the batch, which is incorrect. The correct behavior should create a list for each sample, where each inner list has length equal to the number of image tokens in that specific sample.

## Solution

Changed the default initialization to count image tokens per sample:
```python
image_rows = inputs.pop("rows", [[0] * sample.count(self.image_token) for sample in text])
image_cols = inputs.pop("cols", [[0] * sample.count(self.image_token) for sample in text])
```

This ensures that:
1. Each sample in the batch gets its own list
2. Each list has the correct length matching the number of image tokens in that sample

## Impact

This fix affects:
- `Idefics3Processor` in `src/transformers/models/idefics3/processing_idefics3.py`
- `SmolVLMProcessor` in `src/transformers/models/smolvlm/processing_smolvlm.py`

The bug would have caused issues when processing batches with multiple samples or samples with multiple images, as the default values would have incorrect dimensions.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@zucchini-nlp (multimodal models)



Changed files:
filename: src/transformers/models/idefics3/processing_idefics3.py
patch: @@ -286,8 +286,8 @@ def __call__(
                         f"The number of images in the text {n_images_in_text} and images {n_images_in_images} should be the same."
                     )
 
-                image_rows = inputs.pop("rows", [[0] * len(text)])
-                image_cols = inputs.pop("cols", [[0] * len(text)])
+                image_rows = inputs.pop("rows", [[0] * n_images for n_images in n_images_in_text])
+                image_cols = inputs.pop("cols", [[0] * n_images for n_images in n_images_in_text])
 
                 fake_image_token = self.fake_image_token
                 image_token = self.image_token

filename: src/transformers/models/smolvlm/processing_smolvlm.py
patch: @@ -172,8 +172,6 @@ def __init__(
 
     def expand_text_with_image_tokens(self, text, image_rows, image_cols):
         prompt_strings = []
-        image_rows = image_rows if image_rows is not None else [[0] * len(text)]
-        image_cols = image_cols if image_cols is not None else [[0] * len(text)]
         for sample, sample_rows, sample_cols in zip(text, image_rows, image_cols):
             # Replace the image token with fake tokens around the expanded image token sequence of length `image_seq_len`
             image_prompt_strings = []
@@ -330,6 +328,11 @@ def __call__(
                     raise ValueError(
                         f"The number of images in the text {n_images_in_text} and images {n_images_in_images} should be the same."
                     )
+                # Set default values for image_rows and image_cols if not provided
+                if image_rows is None:
+                    image_rows = [[0] * n_images for n_images in n_images_in_text]
+                if image_cols is None:
+                    image_cols = [[0] * n_images for n_images in n_images_in_text]
                 text = self.expand_text_with_image_tokens(text, image_rows=image_rows, image_cols=image_cols)
 
         elif videos is not None: