image_transforms preprocess quite slow when run large image with qwen2vl
Author: zhjunqin
Pull request: no
Labels: name: Performance

name: bug

name: Vision

### System Info

- `transformers` version: 4.45.2
- Platform: Linux-5.4.0-132-generic-x86_64-with-glibc2.31
- Python version: 3.12.7
- Huggingface_hub version: 0.25.1
- Safetensors version: 0.4.5
- Accelerate version: 1.0.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.4.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 3090


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


funcitons in image_transforms, `rescale`, `normalize` quite slow when preprocess large image.
https://github.com/huggingface/transformers/blob/main/src/transformers/image_transforms.py

here is benchmark

![image](https://github.com/user-attachments/assets/b402be6e-e916-45ad-ac17-78d3dc84d62b)

please refer to https://github.com/vllm-project/vllm/issues/9238

### Expected behavior

how to improve performance?

Comments:
Hey @zhjunqin !

Can be related to https://github.com/huggingface/transformers/pull/28847, where we enabled image processing with torchvision but that only is supported in ViT model. Also @yonigozlan is working on optimizing image processing time in https://github.com/huggingface/transformers/issues/33810, so he might be your point of contact :)

Hey @zhjunqin !
Thanks a lot for raising this issue. Indeed I'm currently working on adding fast image processors to Transformers, and I'll try to address the Qwen one shortly. I'll ping this issue once a PR is opened!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@yonigozlan Hey :) Did you find time to address the qwen preprocessor?

Not yet, but it is still planned :). I will ping here when it's done

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @yonigozlan , is qwen's preprocessing factoring still planned ? We are using offline inference with qwen 2 vl 7B for document extraction tasks for approx 70 million images and the preprocessing time is a major slowdown for us. If it is not planned immediately, can you tell if there's any workaround to speed or skip the preprocessing as I am already sending the resized images using `smart_resize` function but still it somehow sends it to huggingface for resizing it again. Any pointers will help a lot...

Hi @Gladiator07. Sorry for the delay on this. I was waiting for this big refactoring PR on fast image processors https://github.com/huggingface/transformers/pull/35069 to be merged to continue adding new fast image processors.
But as this is taking longer than I thought and since there is a lot of demand for qwen2vl, I'll try to open a PR for a fast qwen2vl image processors by the end of the week. I'll ping here when it's opened.
Once it's out you'll be able to checkout my branch to use it.
Hope that sounds good!

PR is open here https://github.com/huggingface/transformers/pull/35733 !