SAM: Notebook example not working
Author: antoinemacia
Pull request: no

### System Info

- `transformers` version: 4.29.0.dev0
- Platform: macOS-13.2-arm64-arm-64bit
- Python version: 3.10.6
- Huggingface_hub version: 0.13.4
- Safetensors version: 0.3.0
- PyTorch version (GPU?): 1.13.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.6.9 (cpu)
- Jax version: 0.4.8
- JaxLib version: 0.4.7
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

Dependencies
- torch = 1.13.0
- numpy = 1.23.4

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Pull [SAM Notebook example](https://github.com/huggingface/notebooks/blob/main/examples/segment_anything.ipynb)
2. Run notebook up until 
```
masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs["original_sizes"].cpu(), inputs["reshaped_input_sizes"].cpu())
```
3. Get error
```
TypeError: upsample_bilinear2d() received an invalid combination of arguments - got (Tensor, list, bool, NoneType), but expected one of:
 * (Tensor input, tuple of SymInts output_size, bool align_corners, tuple of floats scale_factors)
      didn't match because some of the arguments have invalid types: (Tensor, !list!, bool, !NoneType!)
 * (Tensor input, tuple of SymInts output_size, bool align_corners, float scales_h, float scales_w, *, Tensor out)
```

### Expected behavior

original_sizes/output_sizes to be of the expected type, is this a dependency issue?

Comments:
I have similar issue when i run 

```
img_url = "https://huggingface.co/ybelkada/segment-anything/resolve/main/assets/car.png"
raw_image = Image.open(requests.get(img_url, stream=True).raw).convert("RGB")
input_points = [[[450, 600]]]  # 2D location of a window in the image

inputs = processor(raw_image, input_points=input_points, return_tensors="pt").to(device)
outputs = model(**inputs)
```

```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-6-abdc2d7068b8> in <module>
      4 
      5 inputs = processor(raw_image, input_points=input_points, return_tensors="pt").to(device)
----> 6 outputs = model(**inputs)
      7 
      8 masks = processor.image_processor.post_process_masks(

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/sam/modeling_sam.py in forward(self, pixel_values, input_points, input_labels, input_boxes, input_masks, image_embeddings, multimask_output, output_attentions, output_hidden_states, return_dict, **kwargs)
   1331             )
   1332 
-> 1333         sparse_embeddings, dense_embeddings = self.prompt_encoder(
   1334             input_points=input_points,
   1335             input_labels=input_labels,

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/sam/modeling_sam.py in forward(self, input_points, input_labels, input_boxes, input_masks)
    669             if input_labels is None:
    670                 raise ValueError("If points are provided, labels must also be provided.")
--> 671             point_embeddings = self._embed_points(input_points, input_labels, pad=(input_boxes is None))
    672             sparse_embeddings = torch.empty((batch_size, point_batch_size, 0, self.hidden_size), device=target_device)
    673             sparse_embeddings = torch.cat([sparse_embeddings, point_embeddings], dim=2)

~/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/models/sam/modeling_sam.py in _embed_points(self, points, labels, pad)
    619             padding_point = torch.zeros(target_point_shape, device=points.device)
    620             padding_label = -torch.ones(target_labels_shape, device=labels.device)
--> 621             points = torch.cat([points, padding_point], dim=2)
    622             labels = torch.cat([labels, padding_label], dim=2)
    623         input_shape = (self.input_image_size, self.input_image_size)

RuntimeError: Expected object of scalar type double but got scalar type float for sequence element 1.
```

```

- `transformers` version: 4.29.0.dev0
- Platform: Linux-3.10.0-957.12.2.el7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.3
- Huggingface_hub version: 0.13.4
- Safetensors version: not installed
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

cc @younesbelkada @ArthurZucker 

Thanks for reporting! Will fix this asap

Same here.

TypeError: upsample_bilinear2d() received an invalid combination of arguments - got (Tensor, list, bool, NoneType),
but expected one of:
 * (Tensor input, tuple of ints output_size, bool align_corners, tuple of floats scale_factors)
      didn't match because some of the arguments have invalid types: (Tensor, !list!, bool, !NoneType!)
 * (Tensor input, tuple of ints output_size, bool align_corners, float scales_h, float scales_w, *, Tensor out)

Hi @antoinemacia @xiao2mo 
I can confirm now the colab scripts works as expected if you re-install the library from source!
@YubinXie could you open another ticket for your issue to keep track of it? 
Have a great weekend everyone!

@younesbelkada @ArthurZucker its working on my end, thanks for looking at it so promptly 😄 

Good week end yall!