Potential Memory Leak or Caching in Fast Image Processor
Author: yhyang201
Pull request: no
Labels: name: bug

### System Info

Hi team,

Thank you for your great work on `transformers`!

While using the `AutoProcessor` with `use_fast=True`, I noticed that there seems to be a memory leak or possibly some form of persistent caching when processing images. Even after deleting the processor and clearing the CUDA cache, approximately 600MB of GPU memory remains occupied.

Here is a minimal reproducible example:

```python
from transformers import AutoProcessor
from PIL import Image
import time
import torch
import requests
from io import BytesIO

processor = AutoProcessor.from_pretrained(
    "Qwen/Qwen2.5-VL-7B-Instruct",
    use_fast=True,
    trust_remote_code=False,
    revision=None,
)

url = "https://github.com/sgl-project/sglang/blob/main/test/lang/example_image.png?raw=true"
response = requests.get(url)
images = [Image.open(BytesIO(response.content)).convert("RGB")]

result = processor(
    text=[
        "
system\nYou are a helpful assistant.
\n"
        "
user\nWhat’s in this image?



\n"
        "
assistant\n"
    ],
    padding=True,
    return_tensors="pt",
    images=images,
    device="cuda"
)

del result
del processor
torch.cuda.empty_cache()

print("You can now use nvidia-smi to observe GPU memory usage, which is around 600MB.")
while True:
    time.sleep(60)
```

I’d like to kindly ask:

1. If this is due to caching, is there a way to control or disable the cache?
2. If this is an unintended memory leak, would it be possible to investigate and potentially fix it?

Thanks again for your help and time!

Best regards

### Who can help?

tokenizers: @ArthurZucker and @itazap

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

As provided above.

### Expected behavior

It would be great if caching could be made optional, or if there could be an option to avoid any GPU memory usage entirely.

Comments:
cc @yonigozlan , maybe it's the fused means/std, couldn't see what else can be cached on a quick glance

Hmm not sure either, I'll have a look. Thanks for raising the issue!

Hi @yonigozlan , I did some debugging, it seems the GPU memory jumped after this line: https://github.com/huggingface/transformers/blob/main/src/transformers/image_processing_utils_fast.py#L469-L472

I tried an image shape with `[3, 4096, 4096]`, and dtype `torch.unit8`, after this line, `nvidia-smi` showed a GPU memory around 600 MB

Not sure it will help. Please ignore me if this is incorrect

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

not stale, I believe

cc @yonigozlan did you have updates? 🤗 

Hello @CatherineSue! I don't think there's anything weird going on really, this seems like expected behavior, it's just that the images are stored on the GPU before being processed, as you provided `device="cuda"`. Then I think the "leak" that you see is just from caching allocator.

In your example, if you do something like that:

```python
from transformers import AutoProcessor
from PIL import Image
import time
import torch
import requests
from io import BytesIO

processor = AutoProcessor.from_pretrained(
    "Qwen/Qwen2.5-VL-7B-Instruct",
    use_fast=True,
    trust_remote_code=False,
    revision=None,
)

url = "https://github.com/sgl-project/sglang/blob/main/test/lang/example_image.png?raw=true"
response = requests.get(url)
images = [Image.open(BytesIO(response.content)).convert("RGB")]

result = processor(
    text=[
        "
system\nYou are a helpful assistant.
\n"
        "
user\nWhat’s in this image?



\n"
        "
assistant\n"
    ],
    padding=True,
    return_tensors="pt",
    images=images,
    device="cuda"
)

del result
del processor
torch.cuda.empty_cache()

result2 = processor(
    text=[
        "
system\nYou are a helpful assistant.
\n"
        "
user\nWhat’s in this image?



\n"
        "
assistant\n"
    ],
    padding=True,
    return_tensors="pt",
    images=images,
    device="cuda"
)
```

You should still see the same GPU memory occupied, indicating there's no leak.

That's my understanding at least, but maybe I'm missing something!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.