align dtype of queries, keys in Grounding DINO
Author: Tungway1990
Pull request: yes

To support #32672

Support fp16 and bfloat16 by aligning dtype of queries and keys for Grounding DINO.

Models:
- vision models: @amyeroberts


Comments:
Thanks for contributing! Before I dive in: something we could do to make sure this works is adding a small test, something like that maybe in `test_modeling_groundingdino`?

```python
    @require_torch
    def test_functional_bf16_f16_forward(self):
        config, _ = self.model_tester.prepare_config_and_inputs_for_common()

        for model_class in self.all_model_classes:
            model = model_class(config)
            for dtype in [torch.float16, torch.bfloat16]:
                model = model.to(dtype)
                inputs = self._prepare_for_class(inputs_dict, model_class).to(dtype)
                outputs = model(**inputs)
```

because I just checked out your branch @Tungway1990  locally and got a dtype error when running
```python
 import requests
 
 import torch
 from PIL import Image
 from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection
 
 model_id = "IDEA-Research/grounding-dino-tiny"
 device = "cuda"
 
 dtype = torch.float16
 
 
 processor = AutoProcessor.from_pretrained(model_id)
 model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device).to(dtype)
 
 image_url = "http://images.cocodataset.org/val2017/000000039769.jpg"
 image = Image.open(requests.get(image_url, stream=True).raw)
 # Check for cats and remote controls
 text = "a cat. a remote control."
 
 inputs = processor(images=image, text=text, return_tensors="pt").to(device).to(dtype)
 with torch.no_grad():
     outputs = model(**inputs)
```

Specifically that one
```
    return F.conv2d(input, weight, bias, self.stride,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Input type (float) and bias type (c10::Half) should be the same
```

Other than that, there's a couple of `float()` upcasting to `float32` across the code. `generalized_box_iou` will return upcasted tensors but that should be manageable, however in here 
https://github.com/huggingface/transformers/blob/5fd7ca7bc99371b6bb7524acb482741b17fdc91d/src/transformers/models/grounding_dino/modeling_grounding_dino.py#L1236
there is indeed an upcast as well - wonder if it's necessary? 

Thanks. I will further check it.

Hi [molbap](https://github.com/molbap)

Seems I miss one line to cast the variable `reference_points` to align the dtype

```python
decoder_outputs = self.decoder(
            inputs_embeds=target,
            vision_encoder_hidden_states=encoder_outputs[0],
            vision_encoder_attention_mask=mask_flatten,
            text_encoder_hidden_states=encoder_outputs[1],
            text_encoder_attention_mask=~text_token_mask,
            reference_points=reference_points.to(target.dtype), #change here
            spatial_shapes=spatial_shapes,
            level_start_index=level_start_index,
            valid_ratios=valid_ratios,
            self_attn_mask=None,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            return_dict=return_dict,
        )

```
But I want to have your comment before making the change

I try to run the 
`inputs = processor(images=image, text=text, return_tensors="pt").to(device).to(dtype)`
A message is prompted: Attempting to cast a BatchEncoding to type torch.float16. This is not supported.

And the dtype of  `inputs['pixel_values'].dtype` is still torch.float32

The version of transformers and pytorch are 4.44.0 and 2.3.1+cu121 respectively.

Below is the code I use for testing:
```python
import requests

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection

model_id = "IDEA-Research/grounding-dino-tiny"
device = "cuda"

dtype = torch.float16

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device).to(dtype)

image_url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(image_url, stream=True).raw)
# Check for cats and remote controls
text = "a cat. a remote control."

inputs = processor(images=image, text=text, return_tensors="pt").to(device)
inputs['pixel_values'] = inputs['pixel_values'].to(dtype)

with torch.no_grad():
    outputs = model(**inputs)
```

> Hi [molbap](https://github.com/molbap)
> 
> Seems I miss one line to cast the variable `reference_points` to align the dtype
> 
> ```python
> decoder_outputs = self.decoder(
>             inputs_embeds=target,
>             vision_encoder_hidden_states=encoder_outputs[0],
>             vision_encoder_attention_mask=mask_flatten,
>             text_encoder_hidden_states=encoder_outputs[1],
>             text_encoder_attention_mask=~text_token_mask,
>             reference_points=reference_points.to(target.dtype), #change here
>             spatial_shapes=spatial_shapes,
>             level_start_index=level_start_index,
>             valid_ratios=valid_ratios,
>             self_attn_mask=None,
>             output_attentions=output_attentions,
>             output_hidden_states=output_hidden_states,
>             return_dict=return_dict,
>         )
> ```
Hi @Tungway1990 , i think the above is not sufficient, I also need to update the other functions that use reference_points=reference_points as well. i.e. (`self.deformable_layer(`, `encoder_layer(`) to run successfully.

Hi [molbap](https://github.com/molbap),

https://github.com/huggingface/transformers/blob/54b7703682aee4bc46817ebce96fdbdfcc82e262/src/transformers/models/grounding_dino/modeling_grounding_dino.py#L2200

I further trace the code which changing the dtype of `grid_y`, `grid_x` has the same effect as updating the dtype of `reference_points`.

Hi [Benjamin-Tan](https://github.com/Benjamin-Tan),

You may try the latest code, my test code is:

```python
import requests

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection

model_id = "IDEA-Research/grounding-dino-tiny"
device = "cuda"

dtype = torch.float16

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device).to(dtype)

image_url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(image_url, stream=True).raw)
# Check for cats and remote controls
text = "a cat. a remote control."

inputs = processor(images=image, text=text, return_tensors="pt").to(device)

inputs['pixel_values'] = inputs['pixel_values'].to(dtype)
with torch.no_grad():
    outputs = model(**inputs)

results = processor.post_process_grounded_object_detection(
    outputs,
    inputs.input_ids,
    box_threshold=0.4,
    text_threshold=0.3,
    target_sizes=[image.size[::-1]]
)
print(results)
```

> result:
[{'scores': tensor([0.4775, 0.4378, 0.4826], device='cuda:0'), 
'labels': ['a cat', 'a cat', 'a remote control'], 
'boxes': tensor([[344.6875,  23.0859, 637.1875, 374.0625],
                        [ 12.3438,  51.7969, 317.0312, 472.2656],
                        [ 38.5156,  69.9609, 176.8750, 118.2422]], device='cuda:0')}]

I tried with your latest. It is the same as my previous observations. Need to update the other functions that use reference_points=reference_points as well. i.e. (self.deformable_layer(, encoder_layer() to run successfully.

```
True
Could not load the custom kernel for multi-scale deformable attention: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/bentan/.cache/torch_extensions/py312_cu121/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 752, in forward
    output = MultiScaleDeformableAttentionFunction.apply(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/autograd/function.py", line 574, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 114, in forward
    output = MultiScaleDeformableAttention.ms_deform_attn_forward(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'ms_deform_attn_forward'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bentan/gdino.py", line 23, in <module>
    outputs = model(**inputs)
              ^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 3023, in forward
    outputs = self.model(
              ^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 2360, in forward
    encoder_outputs = self.encoder(
                      ^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 1753, in forward
    (vision_features, text_features), attentions = encoder_layer(
                                                   ^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 1280, in forward
    (vision_features, vision_deformable_attn) = self.deformable_layer(
                                                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 1146, in forward
    hidden_states, attn_weights = self.self_attn(
                                  ^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 762, in forward
    output = multi_scale_deformable_attention(value, spatial_shapes, sampling_locations, attention_weights)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 606, in multi_scale_deformable_attention
    sampling_value_l_ = nn.functional.grid_sample(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bentan/venv/lib/python3.12/site-packages/torch/nn/functional.py", line 4381, in grid_sample
    return torch.grid_sampler(input, grid, mode_enum, padding_mode_enum, align_corners)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: expected scalar type Half but found Float
```

Hi [Benjamin-Tan](https://github.com/Benjamin-Tan),

I can reproduce the issue in Linux environment. Upon examining the traceback, it shows that the error message is triggered by the function `load_cuda_kernels`, where the C++ build has failed. You may refer to the two code snippets below.

https://github.com/huggingface/transformers/blob/85345bb439652d3f03bb4e123cef7a440f2ba95b/src/transformers/models/grounding_dino/modeling_grounding_dino.py#L634-L639

https://github.com/huggingface/transformers/blob/85345bb439652d3f03bb4e123cef7a440f2ba95b/src/transformers/models/grounding_dino/modeling_grounding_dino.py#L71-L98

As I originally tested the code in a Windows environment, there were no issues at all. When comparing the files in `.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/` (Linux) and `.AppData\Local\torch_extensions\torch_extensions\Cache\py311_cu121\MultiScaleDeformableAttention` (Window)  , there are a few missing files due to the failed C++ build.

(Linux)
![image](https://github.com/user-attachments/assets/274d68c8-922b-4cfe-be2c-2fc296ff9079)

(Window)
![image](https://github.com/user-attachments/assets/99732d61-4adf-428b-96c5-1f2b375b9634)

Below is the error message that I received:
```
Could not load the custom kernel for multi-scale deformable attention: 
Error building extension 'MultiScaleDeformableAttention': [1/3] c++ -MMD -MF vision.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/TH -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /usr/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -DWITH_CUDA=1 -c /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/vision.cpp -o vision.o
FAILED: vision.o
c++ -MMD -MF vision.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/TH -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /usr/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -DWITH_CUDA=1 -c /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/vision.cpp -o vision.o
In file included from /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/Device.h:4,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/python.h:8,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/extension.h:9,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/cpu/ms_deform_attn_cpu.h:12,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/ms_deform_attn.h:13,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/vision.cpp:11:
/home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/python_headers.h:12:10: fatal error: Python.h: No such file or directory
   12 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.
[2/3] /usr/local/cuda/bin/nvcc --generate-dependencies-with-compile --dependency-output ms_deform_attn_cuda.cuda.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/TH -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /usr/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu -o ms_deform_attn_cuda.cuda.o
FAILED: ms_deform_attn_cuda.cuda.o
/usr/local/cuda/bin/nvcc --generate-dependencies-with-compile --dependency-output ms_deform_attn_cuda.cuda.o.d -DTORCH_EXTENSION_NAME=MultiScaleDeformableAttention -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/TH -isystem /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /usr/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu -o ms_deform_attn_cuda.cuda.o
/home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu:19:9: warning: #pragma once in main file
   19 | #pragma once
      |         ^~~~
In file included from /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/Device.h:4,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/python.h:8,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/extension.h:9,
                 from /home/ec2-user/.local/lib/python3.9/site-packages/transformers/kernels/deformable_detr/cuda/ms_deform_attn_cuda.cu:20:
/home/ec2-user/.local/lib/python3.9/site-packages/torch/include/torch/csrc/python_headers.h:12:10: fatal error: Python.h: No such file or directory
   12 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 756, in forward
    output = MultiScaleDeformableAttentionFunction.apply(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/autograd/function.py", line 574, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 114, in forward
    output = MultiScaleDeformableAttention.ms_deform_attn_forward(
AttributeError: 'NoneType' object has no attribute 'ms_deform_attn_forward'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ec2-user/document/dino.py", line 25, in <module>
    outputs = model(**inputs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 3027, in forward
    outputs = self.model(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 2364, in forward
    encoder_outputs = self.encoder(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 1757, in forward
    (vision_features, text_features), attentions = encoder_layer(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 1284, in forward
    (vision_features, vision_deformable_attn) = self.deformable_layer(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 1150, in forward
    hidden_states, attn_weights = self.self_attn(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 766, in forward
    output = multi_scale_deformable_attention(value, spatial_shapes, sampling_locations, attention_weights)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 606, in multi_scale_deformable_attention
    sampling_value_l_ = nn.functional.grid_sample(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/nn/functional.py", line 4381, in grid_sample
    return torch.grid_sampler(input, grid, mode_enum, padding_mode_enum, align_corners)
RuntimeError: expected scalar type Half but found Float
```
I print the status of `kernel_loaded()`, `is_torch_cuda_available()` and `is_ninja_available()` when running the same test code in float16.

(Window)
```
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
kernel_loaded: True
is_torch_cuda_available: True
is_ninja_available: True
[{'scores': tensor([0.4781, 0.4367, 0.4814], device='cuda:0'), 'labels': ['a cat', 'a cat', 'a remote control'], 'boxes': tensor([[344.6875,  23.0859, 637.1875, 374.5312],
        [ 12.3438,  51.7969, 317.0312, 472.2656],
        [ 38.5156,  69.9609, 176.8750, 118.2422]], device='cuda:0')}]
```

(Linux)
```
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
kernel_loaded: False
is_torch_cuda_available: True
is_ninja_available: True
Could not load the custom kernel for multi-scale deformable attention: /home/ec2-user/.cache/torch_extensions/py39_cu124/MultiScaleDeformableAttention/MultiScaleDeformableAttention.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 756, in forward
    output = MultiScaleDeformableAttentionFunction.apply(
  File "/home/ec2-user/.local/lib/python3.9/site-packages/torch/autograd/function.py", line 574, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/ec2-user/.local/lib/python3.9/site-packages/transformers/models/grounding_dino/modeling_grounding_dino.py", line 114, in forward
    output = MultiScaleDeformableAttention.ms_deform_attn_forward(
AttributeError: 'NoneType' object has no attribute 'ms_deform_attn_forward'
```

The status of `kernel_loaded()` returns True after the first occurrence in Windows while remaining false in Linux. I believe the root cause of the error is the failed C++ build. Let's see if we can fix the build failure issue.

See if [molbap](https://github.com/molbap) and [amyeroberts](https://github.com/amyeroberts) have any thoughts on it.


Hi @Tungway1990 , if the issue of `kernel_loaded()` is not solved, is it better to update the reference_points=reference_points for now? (self.deformable_layer, encoder_layer) to run successfully