The features calculated by transformers dinov2 are different from the official ones
Author: aihao2000
Pull request: no
Labels: name: bug

### System Info

2024-07-24 07:32:04.151229: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-07-24 07:32:04.151287: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-07-24 07:32:04.152642: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-07-24 07:32:05.387377: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/transformers/commands/env.py:102: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
2024-07-24 07:32:07.837385: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:47] Overriding orig_value setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.42.4
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.23.5
- Safetensors version: 0.4.3
- Accelerate version: 0.32.1
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.3.1+cu121 (True)
- Tensorflow version (GPU?): 2.15.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.8.4 (gpu)
- Jax version: 0.4.26
- JaxLib version: 0.4.26
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: Tesla T4

### Who can help?

@Narsil

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

All the code can be run directly from here： https://colab.research.google.com/drive/1b_nOsILaoFGsQG3TcTQcCxQCOUtvFGEd?usp=sharing

### Expected behavior

I found that the features calculated by dinov2 obtained by torch.hub.load are inconsistent with the features calculated by transformers dinov2. I wonder where the gap is? Am I using it wrong?

Comments:
@aihao2000 The difference is happening because the image being passed to the respective models is not the same. Passing `image` to the processor will process the image for a second time. 

You can update your script as so: 

```py
import torch
from diffusers.utils import load_image
import torchvision.transforms as T
from transformers import AutoImageProcessor, AutoModel
import torch.nn.functional as F

dinov2_vitl14 = torch.hub.load("facebookresearch/dinov2", "dinov2_vitl14").cuda()
url = 'http://images.cocodataset.org/val2017/000000039769.jpg'

image = load_image(url)
transform = T.Compose(
        [
            T.Resize((518, 518)),
            T.ToTensor(),
            T.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
        ]
    )

image = transform(image).unsqueeze(0).cuda()
cls_token = dinov2_vitl14(image, is_training=False)

model = AutoModel.from_pretrained('facebook/dinov2-large').to('cuda')

outputs = model(pixel_values=image)
last_hidden_states = outputs.last_hidden_state

print(F.cosine_similarity(cls_token, outputs['pooler_output']))
print(F.mse_loss(cls_token, outputs['pooler_output']))
```

@amyeroberts Emm. I made a stupid mistake, thanks.