Memory overflow occurs when calling the sam3-litetext model
Author: mcmingchang
Pull request: no
Labels: name: bug

### System Info

from io import BytesIO

import httpx
from PIL import Image

from transformers import AutoModel, AutoProcessor


model = AutoModel.from_pretrained("yonigozlan/sam3-litetext-s0", device_map="auto")
processor = AutoProcessor.from_pretrained("yonigozlan/sam3-litetext-s0")

image_url = "http://images.cocodataset.org/val2017/000000077595.jpg"
image = Image.open(BytesIO(httpx.get(image_url).content)).convert("RGB")

inputs = processor(images=image, text="ear", return_tensors="pt").to(model.device)

outputs = model(**inputs)

results = processor.post_process_instance_segmentation(
    outputs,
    threshold=0.5,
    mask_threshold=0.5,
    target_sizes=inputs.get("original_sizes").tolist(),
)[0]

print(f"Found {len(results['masks'])} objects")


I noticed that the model's volume is not particularly large, but it consumed most of the video memory during the first inference, and the video memory overflowed during the second inference

### Who can help?

win11
python ==3.12.5
torch == '2.8.0+cu129'
transformers == 5.9.0

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I noticed that the model's volume is not particularly large, but it consumed most of the video memory during the first inference, and the video memory overflowed during the second inference

### Expected behavior

It does not consume a large amount of video memory during inference

Comments:
This is caused by the running inference without torch inference mode.
You just need to wrap the inference with `with torch.inference_mode()` like:
```python
with torch.inference_mode():
        outputs = model(**inputs)
```

Hi! @mcmingchang  I’d like to work on this issue and submit a small docs fix for it.

I checked the current issue state and overlap:
- The issue is currently open and unassigned.
- I found no open PR linked to `#46369`.
- I found no open PR directly addressing the `sam3-litetext` memory/inference-mode docs issue.
- I saw @Bissmella’s helpful comment that the repro should run inference under `torch.inference_mode()`.

The repro code appears to match the official SAM3-LiteText docs example, where the forward pass currently runs as:
`outputs = model(**inputs)`
My proposed minimal fix is to update docs/source/en/model_doc/sam3_lite_text.md by:

adding import torch
wrapping only the forward pass with:

```
with torch.inference_mode():
    outputs = model(**inputs)
```
This keeps the model behaviour unchanged and only makes the official inference example avoid building autograd state, which should reduce GPU memory usage across repeated inference calls.

Could this issue please be assigned to me before I open the PR? I’ll include the issue link, duplicate-work check, and test result in the PR description.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.