SAM2 Video support fp16
Author: Guppy16
Pull request: yes

# What does this PR do?

Fix SAM2 Video inference processor so that it can support float16 (currently just works for fp32 and bfloat16).

## How to reproduce

Demo source from [here](https://huggingface.co/docs/transformers/en/model_doc/sam2_video#streaming-video-inference)

- This demo will work for: `dtype = torch.bfloat16` and `dtype = torch.float32`, 
- and this PR fixes it for the case: `dtype = torch.float16` 
- (pls note that fp8 / int8 / etc don't work)

```py
import torch
from transformers import Sam2VideoModel, Sam2VideoProcessor
from transformers.video_utils import load_video

device = torch.device("cuda")
dtype = torch.float16
model_name = "facebook/sam2.1-hiera-tiny"

model = Sam2VideoModel.from_pretrained(model_name).to(device, dtype=dtype)

processor = Sam2VideoProcessor.from_pretrained(model_name)


video_url = "https://huggingface.co/datasets/hf-internal-testing/sam2-fixtures/resolve/main/bedroom.mp4"

video_frames, _ = load_video(video_url)

# Initialize session for streaming
inference_session = processor.init_video_session(
    inference_device=device,
    dtype=dtype,
)

# Process frames one by one
for frame_idx, frame in enumerate(video_frames[:10]):  # Process first 10 frames
    inputs = processor(images=frame, device=device, return_tensors="pt")
    if frame_idx == 0:
        # Add point input on first frame
        processor.add_inputs_to_inference_session(
            inference_session=inference_session,
            frame_idx=0,
            obj_ids=1,
            input_points=[[[[210, 350], [250, 220]]]],
            input_labels=[[[1, 1]]],
            original_size=inputs.original_sizes[
                0
            ],  # need to be provided when using streaming video inference
        )
    # Process current frame
    sam2_video_output = model(
        inference_session=inference_session, frame=inputs.pixel_values[0]
    )
    video_res_masks = processor.post_process_masks(
        [sam2_video_output.pred_masks],
        original_sizes=inputs.original_sizes,
        binarize=False,
    )[0]
    print(f"Frame {frame_idx}: mask shape {video_res_masks.shape}")
```


## Who can review?
@yonigozlan @molbap


Comments:
@yonigozlan bump. there are a few cicd pipelines which are failing; smth to do with code quality and consistency between sam 2 and sam 3 (but not sure entirely)

Thanks! The checks are failing because you are modifying a `modular` file which serves to auto-generate one or several `modeling` files. https://huggingface.co/docs/transformers/v5.0.0rc2/modular_transformers#create-a-modelingpy-file 

Thank you for raising this issue @Guppy16 ! Indeed I could reproduce the problem. I simplified the fix a bit to convert to the correct dtype as late as possible, and added tests. 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: edgetam_video, sam2_video, sam3_tracker_video, sam3_video

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43268). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.