[RT-DETRv2] MPS crash: build_2d_sinusoidal_position_embedding hardcodes torch.float64, breaking Apple Silicon / MPS inference
Author: shubhammr21
Pull request: no
Labels: name: bug

### System Info

```
- `transformers` version: 5.9.0
- Platform: macOS (Apple Silicon, MPS backend)
- Python version: 3.13
- PyTorch version: (MPS-enabled build)
- `docling` version: 2.95.0
- `docling-ibm-models` version: 3.13.2
```

### Reproduction

Run `RTDetrV2ForObjectDetection` inference on any Apple Silicon Mac (MPS device). Triggered in practice via `docling` → `docling-ibm-models` → `transformers`, but reproducible with:

```python
import torch
from transformers import RTDetrV2ForObjectDetection, RTDetrImageProcessor
from PIL import Image
import requests

device = torch.device("mps")
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

processor = RTDetrImageProcessor.from_pretrained("PekingU/rtdetr_v2_r18vd")
model = RTDetrV2ForObjectDetection.from_pretrained("PekingU/rtdetr_v2_r18vd").to(device)

inputs = processor(images=image, return_tensors="pt")
inputs = {k: v.to(device) for k, v in inputs.items()}

with torch.no_grad():
    outputs = model(**inputs)  # crashes here
```

**Error:**
```
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
```

Traceback points to `modeling_rt_detr_v2.py` line 988, inside `build_2d_sinusoidal_position_embedding`:
```python
omega = torch.arange(pos_dim, dtype=torch.float64, device=device) / pos_dim
```

### Expected behavior

Inference should succeed on MPS devices.

**Root cause:** `build_2d_sinusoidal_position_embedding` already accepts a `dtype` parameter (defaulting to `torch.float32`), and the caller at line 1077 correctly passes `dtype=hidden_states.dtype`. However, all internal tensor allocations hardcode `torch.float64`, ignoring the parameter entirely.

Note: the same function body is the canonical source in `modeling_vit_mae.py` and gets inlined into the generated `modeling_rt_detr.py` and `modeling_rt_detr_v2.py`, so all three files (or the source + a regeneration) would need updating.

Verified locally on Apple Silicon (MPS, transformers 5.9.0): crash without the fix, `logits shape: torch.Size([1, 300, 80])` with it.


Comments:
**Suggested fix**

The function `build_2d_sinusoidal_position_embedding` already accepts a `dtype` parameter (defaulting to `torch.float32`), but all internal tensor allocations still hardcode `torch.float64`, bypassing it entirely. The fix is to replace every `torch.float64` inside the function body with `dtype`:

```diff
 def build_2d_sinusoidal_position_embedding(..., dtype: torch.dtype = torch.float32) -> torch.Tensor:
     ...
-        dtype: Output dtype; frequency arithmetic uses float64 internally.
+        dtype: Target dtype for all tensors.
     ...
-    omega  = torch.arange(pos_dim, dtype=torch.float64, device=device) / pos_dim
+    omega  = torch.arange(pos_dim, dtype=dtype,         device=device) / pos_dim
-    grid_h = torch.arange(height,  dtype=torch.float64, device=device)
+    grid_h = torch.arange(height,  dtype=dtype,         device=device)
-    grid_w = torch.arange(width,   dtype=torch.float64, device=device)
+    grid_w = torch.arange(width,   dtype=dtype,         device=device)
     ...
-        pos_embed = torch.cat([torch.zeros(1, embed_dim, dtype=torch.float64, device=device), pos_embed], dim=0)
+        pos_embed = torch.cat([torch.zeros(1, embed_dim, dtype=dtype,         device=device), pos_embed], dim=0)
```

The same function body appears verbatim in the generated `modeling_rt_detr.py` (imported from `vit_mae`) and `modeling_rt_detr_v2.py`, so all three files would need updating (or the generated ones regenerated after fixing the source in `modeling_vit_mae.py`).

Happy to open a PR if this direction looks right to you.

Hi @shubhammr21, thanks for the detailed analysis! I'd like to take this if you haven't started on a PR yet. The fix is straightforward — replace all hardcoded `torch.float64` with the `dtype` parameter in `build_2d_sinusoidal_position_embedding`. Happy to step aside if you're already working on it.

Thanks @Sriniketh24! Going with a slightly different approach than what I sketched above — PR is up at #46174.

cc @yonigozlan, and sorry about the late ping here. I thought this was just a hardware issue, but after some inspection I think the `float64` lines were introduced in #41693. I'm not sure that actually matches what the original models were doing in some of these cases, and it's breaking on MPS! Do we need that, or can we refactor some of them back to `float32`?

Hey, I dug into this one and I think I found the exact cause.

**build_2d_sinusoidal_position_embedding** already takes a dtype parameter that defaults to **float32**, but if you look at the actual implementation, every single tensor inside the function is created with **torch****.float64** directly:

python
_omega = torch.arange(pos_dim, dtype=torch.float64, device=device)
grid_h = torch.arange(height, dtype=torch.float64, device=device)
grid_w = torch.arange(width, dtype=torch.float64, device=device)_

So the dtype param is basically decoration, it gets completely ignored during the actual math. The **.to(dtype)** at the end was supposed to handle it, but on MPS the crash happens at line 988 when the first float64 tensor is created, so you never even get there.

Apple's MPS doesn't implement float64 at all, which is why this only surfaces on Macs. CUDA handles it fine so it's been hiding there for a while.

Fix is pretty mechanical, just swap all the torch.float64 references inside the function body to use dtype instead. Also updated the docstring which was actively misleading ("frequency arithmetic uses float64 internally").

Ran the full test suite locally, 95 passed. Also verified MPS inference works end-to-end on my M-series machine after the fix.

Want me to open a PR..?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Regarding **[RT-DETRv2] MPS crash: build_2d_sinusoidal_position_embedding hardcodes torch.float64, breaking Apple Silicon / MPS inference**:

**@shubhammr21** provided a concrete implementation note:
> **Suggested fix** The function `build_2d_sinusoidal_position_embedding` already accepts a `dtype` parameter (defaulting to `torch.float32`), but all internal tensor allocations still hardcode `torch.float64`, bypassing it entirely. The fix is to replace every `torch.float64` inside the function body with `dtype`: ``...

That should be validated first. If it does not resolve the issue, the next step is to determine whether the failure originates in configuration, dependencies, or the call site.

— 2000pd3rvr