Introduce numpy/numba optimization to `Qwen2VLImageProcessor`
Author: Isotr0py
Pull request: yes
Labels: name: Vision

name: optimization

name: Processing

# What does this PR do?

Discussion in vllm Slack: https://vllm-dev.slack.com/archives/C07QCGVDNUF/p1739079336572759
- Introduce numpy/numba optimization to `Qwen2VLImageProcessor` by fusing rescale, normalization and transposing operators.
- Thanks @imkero for providing valuable numba implementation for Fast Qwen2-VL preprocessing! https://gist.github.com/imkero/817c3087c00c0646abe077bb858d761e

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@zucchini-nlp @yonigozlan 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36356). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Very cool thanks for contributing this @Isotr0py!
Would be great to have some benchmarks comparing this implementation with the previous one, and with fast image processors on CPU as well. I'm especially curious to see how numba compares to a compiled fast image processor.
If it's faster, then we must have this in fast image processors 😉. That means we would need to process numpy arrays though whereas fast image processors only handle torch tensors for now. But if the speedup is worth it, we can try to change that :)


The magic of numba acceleration contains three points:

1. jit compiling python code to machine code
2. precomputed values (just 3 × 256 elems)
3. parallelism based on threading (just replace `range` with numba's `prange`, and mark parallel=True)

   It's important to choose which dimension to be in parallel.

   My original implementation is frame-wise parallel (that means when we input more than one images in same shape, they will be handled in parallel each other).

   I find this PR's code choose row/column-wise parallel (actually only the most outside prange will be in parallel, and prange-in-prange just works as normal range). I am a little worried that it may be "over-parallel" (launching too much threads, and threading overhead cost more than actual operations). Also, this leave multi-image be processed sequentially.

   I am not sure which implementation will be more efficient without actual benchmarking. Maybe we can also take this into consideration while benchmarking, and find the better one.

---

### UPDATE 1

I benchmark the numba-optimized function in my [snippet](https://gist.github.com/imkero/817c3087c00c0646abe077bb858d761e#file-z_benchmark-py) on my machine. It seems good to parallel in row-wise.

**1 frame, 2048x2048**
| numba threads | row-wise time cost | frame-wise time cost |
| - | - | - |
| 1 | 0.0062s | 0.0072s |
| 2 | 0.0031s | 0.0072s |
| 4 | 0.0016s | 0.0072s |
| 8 | 0.0008s | 0.0073s |
| 16 | 0.0008s | 0.0072s |
| 32 | 0.0007s | 0.0073s |

**16 frames, 2048x2048**
| numba threads | row-wise time cost | frame-wise time cost |
| - | - | - |
| 1 | 0.0989s | 0.1156s |
| 2 | 0.0497s | 0.0576s |
| 4 | 0.0266s | 0.0302s |
| 8 | 0.0176s | 0.0179s |
| 16 | 0.0158s | 0.0151s |
| 32 | 0.0159s | 0.0170s |

This conclusion requires the **frame-level loop is in numba scope**, loop outside numba will be slower:

```python
@jit("void(uint8[:,:,:,:],float32[:,:,:,:])", nopython=True, nogil=True, parallel=True)
def _fast_rescale_normalize_transpose_row_wise(vid_in: np.ndarray, vid_out: np.ndarray):
    for frame_idx in range(vid_in.shape[0]): # It's better to loop frames inside numba!
        for y in prange(vid_in.shape[1]):
            for x in range(vid_in.shape[2]):
                for chan in range(3):
                    vid_out[frame_idx, chan, y, x] = NORMALIZED_VALUES_MAP[chan, vid_in[frame_idx, y, x, chan]]
```

### Benchmark results

- Benchmark on Intel(R) Xeon(R) Silver 4116 CPU @ 2.10GHz with 24 cores
- To make the comparison fair, I disabled the cv2 fast_resize in the gist implementation. We should also optimize the resize function, but it will be in another PR (to keep this PR small and clean).


| n_frames | ImageSize | Main branch | This PR | Fast(CPU compiled) | Gist impl(cv2 disabled) |
| :---: | :---: | :---: | :---: | :---: | :---: |
| 2 | 512x512 | 0.0516s | 0.0500s | 0.1275s | 0.0335s |
| 2 | 1024x1024 | 0.2037s | 0.1666s | 0.1809s | 0.1241s |
| 2 | 2048x2048 | 0.8053s | 0.6280s | 0.4098s | 0.3731s |

| n_frames | ImageSize | Main branch | This PR | Fast(CPU compiled) | Gist impl(cv2 disabled) |
| :---: | :---: | :---: | :---: | :---: | :---: |
| 16 | 512x512 | 0.3480s | 0.2766s | 0.4860s | 0.2010s |
| 16 | 1024x1024 | 1.5970s | 1.2181s | 1.0465s | 0.8419s |
| 16 | 2048x2048 | 6.2483s | 4.9856s | 2.9700s | 3.0013s |