🚨[Fast Image Processor] Force Fast Image Processor for Qwen2_VL/2_5_VL + Refactor
Author: yonigozlan
Pull request: yes

# What does this PR do?

As discussed internally, this PR starts the process to make fast image processors the default in 🤗Transformers!

When instantiating a Processor or an Image Processor via `AutoProcessor.from_pretrained` or `AutoImageProcessor.from_pretrained` with a checkpoint using a `Qwen2VLImageProcessor`, the behavior will now be, to load `Qwen2VLImageProcessorFast` even if the processor was saved with a slow `Qwen2VLImageProcessor` originally.

For instance:
Old behavior:
```python
>> from transformers import AutoProcessor
>> processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")
>> print(type(processor.image_processor))
<class 'transformers.models.qwen2_vl.image_processing_qwen2_vl.Qwen2VLImageProcessor'>
```

New behavior:
```python
>> from transformers import AutoProcessor
>> processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")
>> print(type(processor.image_processor))
"""The image processor of type `Qwen2VLImageProcessor` is now loaded as a fast processor by default, even if the model checkpoint was saved with a slow processor. This is a breaking change and may produce slightly different outputs. To continue using the slow processor, instantiate this class with `use_fast=False`. Note that this behavior will be extended to all models in a future release."""
<class 'transformers.models.qwen2_vl.image_processing_qwen2_vl_fast.Qwen2VLImageProcessorFast'>
```
( The warning is a `warning_once`)


This PR also comes with a long overdue refactor (which should be 100% compatible with the slow image processor of qwen2 vl, and fix some existing inconsistencies with the fast one). Cc @zucchini-nlp for that :)

🚨The processed images in output between the slow and fast image processor are slightly different! This is expected as torchvision and PiL image processing functions are not fully equivalent.
Users can still force the use of a slow processor by loading the processor with `use_fast=False`

```python
>> from transformers import AutoProcessor
>> processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct", use_fast=False)
>> print(type(processor.image_processor))
<class 'transformers.models.qwen2_vl.image_processing_qwen2_vl.Qwen2VLImageProcessor'>
```

Here are some comparison between fast and slow processors with this refactor. 
Mixed various means images of different sizes are included in input. The images used for these benchmarks is [this one](http://images.cocodataset.org/val2017/000000039769.jpg)

***Summary of the summary: up to 30x speedup, between 5e-8 and 3e-3 average output pixel differences depending on the processing parameters and input image sizes***

Summary: Max Output Difference vs. Slow processor
-
This table shows the maximum difference at any single point between the output tensors of the Fast processors and the Slow processor.

|                     Batch Size                                 |         1 |         4 |         8 |        16 |        32 |        64 |
|:-----------------------------------------------------|----------:|----------:|----------:|----------:|----------:|----------:|
| ('mixed_various', 'Fast_cpu_grouping_disabled')      | 2.384e-07 | 0.0292    | 0.0292    | 0.0292    | 0.0292    | 0.0292    |
| ('mixed_various', 'Fast_cpu_grouping_enabled')       | 2.384e-07 | 0.0292    | 0.0292    | 0.0292    | 0.0292    | 0.0292    |
| ('uniform_1024x1024', 'Fast_cpu_grouping_disabled')  | 0.0292    | 0.0292    | 0.0292    | 0.0292    | 0.0292    | 0.0292    |
| ('uniform_1024x1024', 'Fast_cpu_grouping_enabled')   | 0.0292    | 0.0292    | 0.0292    | 0.0292    | 0.0292    | 0.0292    |
| ('uniform_224x224', 'Fast_cpu_grouping_disabled')    | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 |
| ('uniform_224x224', 'Fast_cpu_grouping_enabled')     | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 |
| ('uniform_512x512', 'Fast_cpu_grouping_disabled')    | 0.01501   | 0.01501   | 0.01501   | 0.01501   | 0.01501   | 0.01501   |
| ('uniform_512x512', 'Fast_cpu_grouping_enabled')     | 0.01501   | 0.01501   | 0.01501   | 0.01501   | 0.01501   | 0.01501   |
| ('mixed_various', 'Fast_cuda_grouping_disabled')     | 2.384e-07 | 0.09005   | 0.09005   | 0.09005   | 0.09005   | 0.09005   |
| ('mixed_various', 'Fast_cuda_grouping_enabled')      | 2.384e-07 | 0.09005   | 0.09005   | 0.09005   | 0.09005   | 0.09005   |
| ('uniform_1024x1024', 'Fast_cuda_grouping_disabled') | 0.04266   | 0.04266   | 0.04266   | 0.04266   | 0.04266   | 0.04266   |
| ('uniform_1024x1024', 'Fast_cuda_grouping_enabled')  | 0.04266   | 0.04266   | 0.04266   | 0.04266   | 0.04266   | 0.04266   |
| ('uniform_224x224', 'Fast_cuda_grouping_disabled')   | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 |
| ('uniform_224x224', 'Fast_cuda_grouping_enabled')    | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 | 2.384e-07 |
| ('uniform_512x512', 'Fast_cuda_grouping_disabled')   | 0.09005   | 0.09005   | 0.09005   | 0.09005   | 0.09005   | 0.09005   |
| ('uniform_512x512', 'Fast_cuda_grouping_enabled')    | 0.09005   | 0.09005   | 0.09005   | 0.09005   | 0.09005   | 0.09005   |

Summary: Mean Absolute Output Difference vs. Slow processor
-
This table shows the mean absolute difference between the output tensors of the Fast processors and the Slow processor for each configuration and image scenario.

|                Batch Size                                      |         1 |         4 |         8 |        16 |        32 |        64 |
|:-----------------------------------------------------|----------:|----------:|----------:|----------:|----------:|----------:|
| ('mixed_various', 'Fast_cpu_grouping_disabled')      | 5.315e-08 | 7.732e-05 | 7.452e-05 | 7.956e-05 | 7.892e-05 | 8e-05     |
| ('mixed_various', 'Fast_cpu_grouping_enabled')       | 5.315e-08 | 7.732e-05 | 7.452e-05 | 7.956e-05 | 7.892e-05 | 8e-05     |
| ('uniform_1024x1024', 'Fast_cpu_grouping_disabled')  | 9.615e-05 | 9.615e-05 | 9.615e-05 | 9.615e-05 | 9.615e-05 | 9.615e-05 |
| ('uniform_1024x1024', 'Fast_cpu_grouping_enabled')   | 9.615e-05 | 9.615e-05 | 9.615e-05 | 9.615e-05 | 9.615e-05 | 9.615e-05 |
| ('uniform_224x224', 'Fast_cpu_grouping_disabled')    | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 |
| ('uniform_224x224', 'Fast_cpu_grouping_enabled')     | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 |
| ('uniform_512x512', 'Fast_cpu_grouping_disabled')    | 2.832e-05 | 2.832e-05 | 2.832e-05 | 2.832e-05 | 2.832e-05 | 2.832e-05 |
| ('uniform_512x512', 'Fast_cpu_grouping_enabled')     | 2.832e-05 | 2.832e-05 | 2.832e-05 | 2.832e-05 | 2.832e-05 | 2.832e-05 |
| ('mixed_various', 'Fast_cuda_grouping_disabled')     | 5.315e-08 | 0.002611  | 0.002679  | 0.002686  | 0.0027    | 0.002701  |
| ('mixed_various', 'Fast_cuda_grouping_enabled')      | 5.315e-08 | 0.002611  | 0.002679  | 0.002686  | 0.0027    | 0.002701  |
| ('uniform_1024x1024', 'Fast_cuda_grouping_disabled') | 0.002783  | 0.002783  | 0.002783  | 0.002783  | 0.002783  | 0.002783  |
| ('uniform_1024x1024', 'Fast_cuda_grouping_enabled')  | 0.002783  | 0.002783  | 0.002783  | 0.002783  | 0.002783  | 0.002783  |
| ('uniform_224x224', 'Fast_cuda_grouping_disabled')   | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 |
| ('uniform_224x224', 'Fast_cuda_grouping_enabled')    | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 | 5.315e-08 |
| ('uniform_512x512', 'Fast_cuda_grouping_disabled')   | 0.002913  | 0.002913  | 0.002913  | 0.002913  | 0.002913  | 0.002913  |
| ('uniform_512x512', 'Fast_cuda_grouping_enabled')    | 0.002913  | 0.002913  | 0.002913  | 0.002913  | 0.002913  | 0.002913  |

Time per images:
-

<img width="4800" height="2700" alt="time_per_image_all_configs" src="https://github.com/user-attachments/assets/a8022f88-c164-4a05-abb7-265fd8837bf3" />


With different image sizes:

| | | 
|:-------------------------:|:-------------------------:|
|<img width="4800" height="2700" alt="time_per_image_all_configs" src="https://github.com/user-attachments/assets/a8022f88-c164-4a05-abb7-265fd8837bf3" /> | <img width="4800" height="2700" alt="time_per_image_all_configs" src="https://github.com/user-attachments/assets/2e4d7f4a-95a0-462c-a941-eaa8da2d6140" />|
|<img width="4800" height="2700" alt="time_per_image_all_configs" src="https://github.com/user-attachments/assets/04a6cc82-3dbe-47a5-9931-a18aa88032ef" /> |<img width="4800" height="2700" alt="time_per_image_all_configs" src="https://github.com/user-attachments/assets/02e606db-41cb-4e53-b213-eb2d87a7e0b0" />|

Speedups:
-
<img width="4800" height="2700" alt="speedup_vs_slow" src="https://github.com/user-attachments/assets/d38b0354-9ebf-4958-9488-5110745d77bf" />

With different image sizes:

| | | 
|:-------------------------:|:-------------------------:|
|<img width="4800" height="2700" alt="speedup_vs_slow" src="https://github.com/user-attachments/assets/d38b0354-9ebf-4958-9488-5110745d77bf" /> | <img width="4800" height="2700" alt="speedup_vs_slow" src="https://github.com/user-attachments/assets/3a6d68d8-9bd9-4dfb-97ab-1079e1235947" />|  
|<img width="4800" height="2700" alt="speedup_vs_slow" src="https://github.com/user-attachments/assets/b758938b-e0e3-46e8-ad10-0e0ad249ff6d" /> |<img width="4800" height="2700" alt="speedup_vs_slow" src="https://github.com/user-attachments/assets/04c5536c-68fd-42ae-bb9c-f80ae5474ad2" />|


Cc  @qubvel @ArthurZucker @Cyrilvallez 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39591). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for the review @qubvel @zucchini-nlp , made the modifications! I also needed to change quite a few processor tests for qwen vls/omni because of the change to fast image processor by default, but should be good now!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, colqwen2, glm4v, qwen2_5_omni, qwen2_5_vl, qwen2_vl

~~Hi~~
~~I noticed that this PR appears to change the behavior of the Qwen2-VL processor (I haven’t tested Qwen2.5-VL yet, but I suspect it shows a similar pattern).~~
~~Using the Qwen2.5-VL demo image (https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg) as an example:~~
~~- Before the PR, the processor produced pixel_values with shape [4988, 1176];~~
~~- After the PR, the shape becomes [14308, 1176].~~

~~In both cases, the model can correctly interpret the image.~~

~~Could you share whether this change is intentional and necessary? If model capability remains the same, a shorter/smaller pixel_values (i.e., shorter sequence length) should lead to faster inference.~~

Hi @yhyang201 , I'm not able to reproduce the issue, could you please provide a snippet to do so? Thanks.

Hi @yonigozlan , I sincerely apologize — I have double-checked and confirmed that this PR does not exhibit the issue I mentioned earlier.
Thank you very much for your contribution, and I’m truly sorry for the inconvenience and for taking up your time.


@yhyang201 no worries 🤗