Repository: transformers
Issue: RLE of SAM can't handle masks with no change
Labels: bug

### System Info

- `transformers` version: 4.49.0.dev0
- Platform: Windows-10-10.0.26100-SP0
- Python version: 3.11.11
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.5.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I'm fine-tuning the SamModel and using the fine-tuned model in a mask-generation pipeline afterward. 
After some time in the training, I suddenly get the following error when using the fine-tuned model in the pipeline:


```
Traceback (most recent call last):
  File "***.py", line 17, in <module>
    outputs = generator(image)
              ^^^^^^^^^^^^^^^^
  File "transformers\pipelines\mask_generation.py", line 166, in __call__
    return super().__call__(image, *args, num_workers=num_workers, batch_size=batch_size, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\base.py", line 1354, in __call__
    return next(
           ^^^^^
  File "transformers\pipelines\pt_utils.py", line 124, in __next__
    item = next(self.iterator)
           ^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\pt_utils.py", line 269, in __next__
    processed = self.infer(next(self.iterator), **self.params)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\base.py", line 1269, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\pipelines\mask_generation.py", line 237, in _forward
    masks, iou_scores, boxes = self.image_processor.filter_masks(
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\models\sam\image_processing_sam.py", line 847, in filter_masks
    return self._filter_masks_pt(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\models\sam\image_processing_sam.py", line 945, in _filter_masks_pt
    masks = _mask_to_rle_pytorch(masks)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "transformers\models\sam\image_processing_sam.py", line 1386, in _mask_to_rle_pytorch
    counts += [cur_idxs[0].item()] + btw_idxs.tolist() + [height * width - cur_idxs[-1]]
               ~~~~~~~~^^^
IndexError: index 0 is out of bounds for dimension 0 with size 0
```



Note: this error doesn't occur on every image, but just on some.

Code used to produce error:
```
    image = Image.open("PATH_TO_MY_IMAGE")

    model = SamModel.from_pretrained("PATH_TO_MY_CHECKPOINT")
    processor = SamImageProcessor.from_pretrained("facebook/sam-vit-huge")
    generator = pipeline(
        "mask-generation",
        model=model,
        device="cpu",
        points_per_batch=64,
        image_processor=processor
    )  # MaskGenerationPipeline

    outputs = generator(image)
```

### Expected behavior

No error should be thrown and the RLE should be computed correctly.