SAM2/EdgeTAM produces different result on consecutive runs only when input_boxes is specified
Author: xenova
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.57.1
- Platform: macOS-15.6-arm64-arm-64bit-Mach-O
- Python version: 3.13.2
- Huggingface_hub version: 0.35.3
- Safetensors version: 0.5.3
- Accelerate version: 1.6.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0 (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no

### Who can help?

@yonigozlan 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run without input_boxes input:
```py
from transformers import Sam2Processor, EdgeTamModel
import torch
from PIL import Image
import requests

device = "cuda" if torch.cuda.is_available() else "cpu"

model = EdgeTamModel.from_pretrained("yonigozlan/EdgeTAM-hf").to(device)
processor = Sam2Processor.from_pretrained("yonigozlan/EdgeTAM-hf")

image_url = "https://huggingface.co/datasets/hf-internal-testing/sam2-fixtures/resolve/main/truck.jpg"
raw_image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB")


input_points = [[[[500, 375]]]]  # Single point click, 4 dimensions (image_dim, object_dim, point_per_object_dim, coordinates)
input_labels = [[[1]]]  # 1 for positive click, 0 for negative click, 3 dimensions (image_dim, object_dim, point_label)
input_boxes = None
inputs = processor(images=raw_image, input_points=input_points, input_labels=input_labels, input_boxes=input_boxes, return_tensors="pt").to(device)


with torch.no_grad():
    outputs_1 = model(**inputs)

with torch.no_grad():
    outputs_2 = model(**inputs)

for key in outputs_1.keys():
    if isinstance(outputs_1[key], torch.Tensor):
        diff = torch.abs(outputs_1[key] - outputs_2[key])
        print(f"{key} max diff: {diff.max().item()}")
```

produces
```
iou_scores max diff: 0.0
pred_masks max diff: 0.0
object_score_logits max diff: 0.0
```

---

2. Run with input_boxes inputs
```py
from transformers import Sam2Processor, EdgeTamModel
import torch
from PIL import Image
import requests

device = "cuda" if torch.cuda.is_available() else "cpu"

model = EdgeTamModel.from_pretrained("yonigozlan/EdgeTAM-hf").to(device)
processor = Sam2Processor.from_pretrained("yonigozlan/EdgeTAM-hf")

image_url = "https://huggingface.co/datasets/hf-internal-testing/sam2-fixtures/resolve/main/truck.jpg"
raw_image = Image.open(requests.get(image_url, stream=True).raw).convert("RGB")


input_points = [[[[500, 375]]]]  # Single point click, 4 dimensions (image_dim, object_dim, point_per_object_dim, coordinates)
input_labels = [[[1]]]  # 1 for positive click, 0 for negative click, 3 dimensions (image_dim, object_dim, point_label)
input_boxes = [[[400, 300, 800, 600]]]
inputs = processor(images=raw_image, input_points=input_points, input_labels=input_labels, input_boxes=input_boxes, return_tensors="pt").to(device)

with torch.no_grad():
    outputs_1 = model(**inputs)

with torch.no_grad():
    outputs_2 = model(**inputs)

for key in outputs_1.keys():
    if isinstance(outputs_1[key], torch.Tensor):
        diff = torch.abs(outputs_1[key] - outputs_2[key])
        print(f"{key} max diff: {diff.max().item()}")
```

produces
```
iou_scores max diff: 0.009837955236434937
pred_masks max diff: 0.20307767391204834
object_score_logits max diff: 0.08012199401855469
```

### Expected behavior

The values should be the same on consecutive runs, with or without input_boxes inputs.

Comments: