TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
Author: byrkbrk
Pull request: no

### System Info

- `transformers` version: 4.41.1
- Platform: macOS-14.5-arm64-arm-64bit
- Python version: 3.11.9
- Huggingface_hub version: 0.23.2
- Safetensors version: 0.4.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.3.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

#### Reproduction

The example is from https://huggingface.co/docs/transformers/en/tasks/zero_shot_object_detection#zero-shot-object-detection-pipeline

#### The Code

~~~
from transformers import pipeline
import skimage
import numpy as np
from PIL import Image



if __name__ == "__main__":
    image = skimage.data.astronaut()
    image = Image.fromarray(np.uint8(image)).convert("RGB")
    
    detector = pipeline(model="google/owlv2-base-patch16-ensemble", 
                        task="zero-shot-object-detection",
                        device="mps")
    predictions = detector(
        image,
        candidate_labels=["human face", "rocket", "nasa badge", "star-spangled banner"])
~~~

#### The Error

~~~
Traceback (most recent call last):
  File "/Users/burakbayrak/folder-to-del/zero_shot_detection.py", line 15, in <module>
    predictions = detector(
                  ^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/pipelines/zero_shot_object_detection.py", line 135, in __call__
    results = super().__call__(inputs, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/pipelines/base.py", line 1235, in __call__
    return next(
           ^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/pipelines/pt_utils.py", line 124, in __next__
    item = next(self.iterator)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/pipelines/pt_utils.py", line 269, in __next__
    processed = self.infer(next(self.iterator), **self.params)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/pipelines/base.py", line 1150, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/pipelines/zero_shot_object_detection.py", line 172, in _forward
    outputs = self.model(**model_inputs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/models/owlv2/modeling_owlv2.py", line 1705, in forward
    (pred_logits, class_embeds) = self.class_predictor(image_feats, query_embeds, query_mask)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/models/owlv2/modeling_owlv2.py", line 1395, in class_predictor
    (pred_logits, image_class_embeds) = self.class_head(image_feats, query_embeds, query_mask)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/burakbayrak/anaconda3/envs/zero-shot-detection/lib/python3.11/site-packages/transformers/models/owlv2/modeling_owlv2.py", line 1279, in forward
    pred_logits = pred_logits.to(torch.float64)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
~~~


### Expected behavior

When the device information (i.e., `device="mps"`) is passed to `pipeline` object, no error should occur.

Comments:
cc @qubvel if you have bandwidth to give it a look

Hi @byrkbrk thank you for the issue!

It seems like you can just remove the line with the `torch.float64` casting and reinstall the modified repo (in case you need a quick fix). 

See https://github.com/huggingface/transformers/pull/31071

Hello @qubvel,

Casting to `torch.float32` rather than `torch.float64` solves the problem indeed, thanks. 
However, such forcefully casting can have some side effects; that is, it changes the type regardless of the value of the argument `torch_dtype` passed to `pipeline` object.
Anyhow, it is not in the scope of the current issue.