DINOV2 generate different outputs for the same inputs
Author: y22ma
Pull request: no
Labels: name: PyTorch

name: bug

name: Vision

### System Info

transformer: 4.44.2
python: 3.11
platform: ubuntu 20.04
torch: 2.2.2
cuda: 12.5

### Who can help?

@amyeroberts 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Here's a minimal example for me to reproduce this along with the image I used.
![sample](https://github.com/user-attachments/assets/24ab3a8b-ca6a-4e95-b03a-c7267e5bb774)
```
import torch
from PIL import Image
from transformers import AutoImageProcessor, AutoModel

# Initialize the DinoV2Embedding instance
model_name = "facebook/dinov2-large"
device = "cuda" if torch.cuda.is_available() else "cpu"
preprocess = AutoImageProcessor.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name).to(device).eval()

# Path to the image file
img_file_path = ""  # Replace with your image path

# Loop to get embeddings multiple times
for i in range(5):
    with torch.no_grad():

        image = Image.open(img_file_path)
        inputs = preprocess(images=image, return_tensors="pt").to(device)
        outputs = model(**inputs)
        print(f"Output {i + 1}: {outputs}")
```

### Expected behavior

Should generate consistent output such that the embedding could be used for retrieval.

Comments:
Hi @y22ma, thanks for opening the issue!

Hm, it is strange, but I was not able to reproduce the bug. I'm getting consistent results

```python
import torch
import requests
from PIL import Image
from transformers import AutoImageProcessor, AutoModel

# Initialize the DinoV2Embedding instance
model_name = "facebook/dinov2-large"
device = "cuda" if torch.cuda.is_available() else "cpu"
preprocess = AutoImageProcessor.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name).to(device).eval()

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

# Loop to get embeddings multiple times
for i in range(5):
    with torch.no_grad():

        inputs = preprocess(images=image, return_tensors="pt").to(device)
        outputs = model(**inputs)
        print(f"Output {i + 1}: {outputs.last_hidden_state[:, :3, :3]}")
```
<img width="1015" alt="Screenshot 2024-09-06 at 12 48 46" src="https://github.com/user-attachments/assets/43b5d6ba-87a0-4b88-afbb-123a018a8cb4">


Env:
```
- `transformers` version: 4.44.2
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.24.6
- Safetensors version: 0.4.4
- Accelerate version: 0.33.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.4.0+cu121 (False)
```

How different results did you get between iterations? Can you provide an output from my snippet above?


Thanks for getting back to me. Here's my output with your image as input (000000039769.jpg from coco val 2017):
```
Output 1: tensor([[[-2.2247, -2.0199,  0.5695],
         [ 1.3696,  2.1394,  3.7765],
         [ 1.1757,  2.2480,  3.4782]]], device='cuda:0')
Output 2: tensor([[[ 0.0169, -0.0767, -0.4897],
         [-0.6269, -0.8694,  0.5277],
         [-0.5765, -0.1864,  0.3269]]], device='cuda:0')
Output 3: tensor([[[ 0.0169, -0.0767, -0.4897],
         [-0.6269, -0.8694,  0.5277],
         [-0.5765, -0.1864,  0.3269]]], device='cuda:0')
Output 4: tensor([[[ 0.1425,  0.6349, -1.0352],
         [-1.6960, -0.3588,  0.7011],
         [-0.8738, -0.4230,  0.9493]]], device='cuda:0')
Output 5: tensor([[[ 0.8900, -0.6973,  0.1371],
         [ 1.3878, -0.8789, -1.0418],
         [ 0.8685, -0.7731, -0.8942]]], device='cuda:0')
```
It's interesting that the first output matches with yours, and it just goes haywire from there? I'm using a NVIDIA GeForce RTX 3080. I did verify that the input has consistent values.

Can you please provide full output of `transformers-cli env`?

Here it is:
Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.44.2
- Platform: Linux-5.15.0-117-generic-x86_64-with-glibc2.31
- Python version: 3.11.9
- Huggingface_hub version: 0.24.6
- Safetensors version: 0.4.5
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.2.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA GeForce RTX 3080

Thanks @y22ma, unfortunately I don't have an excess to 3080 RTX, and in my environment the issue is not appear. Do you have access to any other machine to reproduce the bug? Have you tried google colab?

I'm not able to reproduce this on a T4 machine on AWS unfortunately.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.