Repository: huggingface/transformers
Pull request: fix(models): Fix Perceiver interpolate_pos_encoding interpolating to the source size

### What does this PR do?

The following failing Perceiver use case was identified and fixed in this PR:

→ c6d2848a23 ([🚨 Fix torch.jit.trace for interpolate_pos_encoding in all vision models](https://github.com/huggingface/transformers/pull/33226)) refactored all vision models' `interpolate_pos_encoding` methods for torch.jit.trace; the canonical pattern used across other vision models (e.g. [modeling_vit.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/vit/modeling_vit.py#L82-L91), [modeling_deit.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/deit/modeling_deit.py#L82-L91)) is that they passes the target (height, width) to `nn.functional.interpolate`; but the Perceiver diff passed the <ins>source</ins> grid dims practically making the interpolation a no-op; this should fix that!
→ I also checked if other models have the exact same issue; and they don't, they compute `new_height = height // self.patch_size` (target patch grid) and pass that.

Fixes #44898

**Before the fix (feel free to cross-check; these errors are reproducible):**

<img width="500" height="500" alt="1" src="https://github.com/user-attachments/assets/5a478c28-2fdd-443d-bb75-612816b87e5f" />

**After the fix (feel free to cross-check):**

<img width="500" height="500" alt="2" src="https://github.com/user-attachments/assets/fc5eb0c7-09a4-44f6-9e3a-e6e18ec1860f" />

### Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you fix any necessary existing tests?

Changed files:
filename: src/transformers/models/perceiver/modeling_perceiver.py
patch: @@ -2567,7 +2567,7 @@ def interpolate_pos_encoding(self, position_embeddings: torch.Tensor, height: in
 
         position_embeddings = nn.functional.interpolate(
             position_embeddings,
-            size=(new_height, new_width),
+            size=(height, width),
             mode="bicubic",
             align_corners=False,
         )

filename: tests/models/perceiver/test_modeling_perceiver.py
patch: @@ -242,6 +242,18 @@ def create_and_check_for_image_classification_learned(
         result = model(inputs, attention_mask=input_mask, labels=sequence_labels)
         self.parent.assertEqual(result.logits.shape, (self.batch_size, self.num_labels))
 
+    def create_and_check_for_learned_image_interpolate_pos(
+        self, config, inputs, input_mask, sequence_labels, token_labels
+    ):
+        model = PerceiverForImageClassificationLearned(config=config)
+        model.to(torch_device)
+        model.eval()
+        higher_res_inputs = floats_tensor(
+            [self.batch_size, self.num_channels, self.image_size * 2, self.image_size * 2]
+        )
+        result = model(higher_res_inputs, attention_mask=input_mask, interpolate_pos_encoding=True)
+        self.parent.assertEqual(result.logits.shape, (self.batch_size, self.num_labels))
+
     def create_and_check_for_image_classification_fourier(
         self, config, inputs, input_mask, sequence_labels, token_labels
     ):
@@ -360,6 +372,12 @@ def test_for_image_classification_learned(self):
         )
         self.model_tester.create_and_check_for_image_classification_learned(*config_and_inputs)
 
+    def test_for_learned_image_interpolate_pos(self):
+        config_and_inputs = self.model_tester.prepare_config_and_inputs(
+            model_class=PerceiverForImageClassificationLearned
+        )
+        self.model_tester.create_and_check_for_learned_image_interpolate_pos(*config_and_inputs)
+
     def test_for_image_classification_fourier(self):
         config_and_inputs = self.model_tester.prepare_config_and_inputs(
             model_class=PerceiverForImageClassificationFourier