Issue 1:
System Info

transformers version: 4.29.0.dev0

Platform: Linux-3.10.0-957.12.2.el7.x86_64-x86_64-with-glibc2.10

Python version: 3.8.3

Huggingface_hub version: 0.13.4

Safetensors version: not installed

PyTorch version (GPU?): 1.5.0 (True)

Tensorflow version (GPU?): not installed (NA)

Flax version (CPU?/GPU?/TPU?): not installed (NA)

Jax version: not installed

JaxLib version: not installed

Using GPU in script?:

Using distributed or parallel set-up in script?:

Who can help?

No response

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction

img_url = "https://huggingface.co/ybelkada/segment-anything/resolve/main/assets/car.png"

raw_image = Image.open(requests.get(img_url, stream=True).raw).convert("RGB")

input_points = [[[450, 600]]] # 2D location of a window in the image

inputs = processor(raw_image, input_points=input_points, return_tensors="pt").to(device)

outputs = model(**inputs)

masks = processor.image_processor.post_process_masks(

outputs.pred_masks.cpu(), inputs["original_sizes"].cpu(), inputs["reshaped_input_sizes"].cpu()

)

scores = outputs.iou_scores

Expected behavior

RuntimeError Traceback (most recent call last)

in

4

5 inputs = processor(raw_image, input_points=input_points, return_tensors="pt").to(device)

----> 6 outputs = model(**inputs)

7

8 masks = processor.image_processor.post_process_masks(

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)

548 result = self._slow_forward(*input, **kwargs)

549 else:

--> 550 result = self.forward(*input, **kwargs)

551 for hook in self._forward_hooks.values():

552 hook_result = hook(self, input, result)

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/sam/modeling_sam.py in forward(self, pixel_values, input_points, input_labels, input_boxes, input_masks, image_embeddings, multimask_output, output_attentions, output_hidden_states, return_dict, **kwargs)

1331 )

1332

-> 1333 sparse_embeddings, dense_embeddings = self.prompt_encoder(

1334 input_points=input_points,

1335 input_labels=input_labels,

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)

548 result = self._slow_forward(*input, **kwargs)

549 else:

--> 550 result = self.forward(*input, **kwargs)

551 for hook in self._forward_hooks.values():

552 hook_result = hook(self, input, result)

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/sam/modeling_sam.py in forward(self, input_points, input_labels, input_boxes, input_masks)

669 if input_labels is None:

670 raise ValueError("If points are provided, labels must also be provided.")

--> 671 point_embeddings = self._embed_points(input_points, input_labels, pad=(input_boxes is None))

672 sparse_embeddings = torch.empty((batch_size, point_batch_size, 0, self.hidden_size), device=target_device)

673 sparse_embeddings = torch.cat([sparse_embeddings, point_embeddings], dim=2)

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/sam/modeling_sam.py in _embed_points(self, points, labels, pad)

619 padding_point = torch.zeros(target_point_shape, device=points.device)

620 padding_label = -torch.ones(target_labels_shape, device=labels.device)

--> 621 points = torch.cat([points, padding_point], dim=2)

622 labels = torch.cat([labels, padding_label], dim=2)

623 input_shape = (self.input_image_size, self.input_image_size)

RuntimeError: Expected object of scalar type double but got scalar type float for sequence element 1.

Issue 2:
System Info

transformers version: 4.29.0.dev0

Platform: macOS-13.2-arm64-arm-64bit

Python version: 3.10.6

Huggingface_hub version: 0.13.4

Safetensors version: 0.3.0

PyTorch version (GPU?): 1.13.0 (False)

Tensorflow version (GPU?): not installed (NA)

Flax version (CPU?/GPU?/TPU?): 0.6.9 (cpu)

Jax version: 0.4.8

JaxLib version: 0.4.7

Using GPU in script?: NO

Using distributed or parallel set-up in script?: NO

Dependencies

torch = 1.13.0

numpy = 1.23.4

Who can help?

No response

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction

Pull SAM Notebook example

Run notebook up until

masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs["original_sizes"].cpu(), inputs["reshaped_input_sizes"].cpu())

Get error

TypeError: upsample_bilinear2d() received an invalid combination of arguments - got (Tensor, list, bool, NoneType), but expected one of:
* (Tensor input, tuple of SymInts output_size, bool align_corners, tuple of floats scale_factors)
didn't match because some of the arguments have invalid types: (Tensor, !list!, bool, !NoneType!)
* (Tensor input, tuple of SymInts output_size, bool align_corners, float scales_h, float scales_w, *, Tensor out)

Expected behavior

original_sizes/output_sizes to be of the expected type, is this a dependency issue?