Use torchvision's native LANCZOS interpolation instead of PIL fallback
Author: NicolasHug
Pull request: yes

# What does this PR do?

This is sort of a follow-up to https://github.com/huggingface/transformers/pull/45195, and the proposed changes were discussed on the PyTorch slack channel with @yonigozlan .

[Torchvision 0.27](https://github.com/pytorch/vision/releases/tag/v0.27.0) added native support for LANCZOS interpolation. Previously, `transformers` needed two workarounds:

- the torchvision backend had to fall back to BICUBIC when LANCZOS was requested
-  some entire model pipelines had to fallback to PIL.

This PR removes both workarounds when torchvision >= 0.27 is installed. TorchVision is faster than PIL because it natively supports AVX2 and NEON paths for x86 and aarch64, and outputs are bitwise equivalent to PIL-SIMD.

### Benchmarks

```
~/dev/transformers (use_torchvision_lanczos*) » python benchmark_lanczos.py                                                                                                                                                                nicolashug@nicolashug-fedora-PW0H326Y
Scenario                                        PIL (ms)    TV (ms)    Speedup
------------------------------------------------------------------------------
Chameleon  (800x600 -> 682x512)                     6.63       1.88       3.5x
Idefics3   (1920x1080 -> 819x1456)                 26.40      11.02       2.4x
Flava CB   (224x224 -> 112x112)                     0.54       0.18       3.1x
```

Benchmark code:
<details> 

```py
"""Benchmark: torchvision LANCZOS resize vs PIL LANCZOS resize.

Requires torchvision >= 0.27 (which added LANCZOS support for tensors).
"""

import time

import PIL.Image
import numpy as np
import torch
from torchvision.transforms.v2.functional import resize as tv_resize
from torchvision.transforms import InterpolationMode


def bench_pil(pil_image, size, n=100):
    # Warmup
    for _ in range(10):
        pil_image.resize(size, PIL.Image.LANCZOS)

    t0 = time.perf_counter()
    for _ in range(n):
        pil_image.resize(size, PIL.Image.LANCZOS)
    return (time.perf_counter() - t0) / n * 1000  # ms


def bench_tv(tensor, size, n=100):
    # Warmup
    for _ in range(10):
        tv_resize(tensor, size, interpolation=InterpolationMode.LANCZOS, antialias=True)

    t0 = time.perf_counter()
    for _ in range(n):
        tv_resize(tensor, size, interpolation=InterpolationMode.LANCZOS, antialias=True)
    return (time.perf_counter() - t0) / n * 1000  # ms


# Typical sizes from the 4 affected models:
#   Chameleon:     shortest_edge=512 (we use 800x600 -> 682x512)
#   Idefics3/SmolVLM: longest_edge=1456 (we use 1920x1080 -> 1456x819)
#   Flava codebook:   112x112 (we use 224x224 -> 112x112)
scenarios = [
    ("Chameleon  (800x600 -> 682x512)", (800, 600), (512, 682)),
    ("Idefics3   (1920x1080 -> 819x1456)", (1920, 1080), (819, 1456)),
    ("Flava CB   (224x224 -> 112x112)", (224, 224), (112, 112)),
]

print(f"{'Scenario':<45} {'PIL (ms)':>10} {'TV (ms)':>10} {'Speedup':>10}")
print("-" * 78)

for label, (in_w, in_h), (out_h, out_w) in scenarios:
    rng = np.random.RandomState(0)
    arr = rng.randint(0, 256, (in_h, in_w, 3), dtype=np.uint8)

    pil_img = PIL.Image.fromarray(arr)
    tensor = torch.from_numpy(arr).permute(2, 0, 1).contiguous()  # CHW uint8

    pil_ms = bench_pil(pil_img, (out_w, out_h))
    tv_ms = bench_tv(tensor, (out_h, out_w))

    print(f"{label:<45} {pil_ms:>10.2f} {tv_ms:>10.2f} {pil_ms / tv_ms:>9.1f}x")

```

</details>


## Code Agent Policy

- [X] I confirm that this is not a pure code agent PR.

## Before submitting
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests? _I think existing tests properly cover the codepaths already_


## Who can review?

@vasqu  @zucchini-nlp since your reviewed https://github.com/huggingface/transformers/pull/45195. Thanks!


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
Agree, fixing that one test is the important bit, the rest are nits. Feel free to merge afterwards @zucchini-nlp 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, chameleon, flava

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46496)

@NicolasHug is the torch implementation of lanczos interpolation an exact replica of PIL? I see some numerical differences, so wanted to confirm. Would be happy if you could guide me to the tests asserting this version with the PIL version

edit: never mind, I see the issue. The bitwise equivalence is for PIL-SIMD, not regular PIL library as you mentioned above. One side effect of this will be that there will be discrepancies with libraries like mediapy that use PIL (my case - a jax based based model + mediapy processor)

@MHRDYN7 yes you'll have small discrepancies, but those aren't significant enough for models to notice. Models would notice the difference between Lanczos and Bicubic, or between antialias=True vs antialias=False, but not these.

Generally, we can't expect bitwise equal results for image preproc: even two compliant jpeg decoders may output slightly different pixel values (there are for example differences between libjpeg and libjpeg-turbo).

In this case for lanczos, the difference comes from the fact that intermediate value are stored as uint8 rather than float. This leads to small differences for lanczos and bicubic, less so for bilinear mode.