SAM3 Video: Remove call to pin_memory if the target device is not CUDA
Author: atultw
Pull request: yes

# What does this PR do?
Remove the call to pin_memory if the target device is not CUDA. 

This library has been recommended for people who can't use the main SAM3 repo due to CUDA requirement, for example on MPS: https://huggingface.co/facebook/sam3/discussions/11 

I believe pin_memory is only effective for CUDA type devices. There hasn't been movement on pin_memory fix for MPS since 2022: https://github.com/pytorch/pytorch/issues/86060 

Transformers is the easiest way to run SAM3 on a Mac, and this change would let us properly use SAM3 Video. 


<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


@yonigozlan


Comments:
@atultw  The logic is sound and low risk, but it would be good to add a small test covering the non-CUDA path to prevent future regressions, and a brief inline comment explaining why `pin_memory()` is gated to CUDA would improve maintainability.


Hi @atultw! I recently made a PR that fixes MPS output for SAM3.
https://github.com/huggingface/transformers/pull/43042 

If fixes the input, actually, so the output is correct :)

As far as I know `pin_memory` is ignored if device is not CUDA, but if it actually makes it work on your machine - great!

> As far as I know `pin_memory` is ignored if device is not CUDA

This may be the case if you run everything on CPU, let me test that and report back. 

When running on MPS, it does not appear to be ignored because I get the same error as https://github.com/pytorch/pytorch/issues/86060. Maybe we need to only remove the call if the target device is MPS? Rather than all non-CUDA. 

Edit: I just reviewed your PR and reproduced your issue with sam3_tracker_video on MPS. Thanks for finding that root cause. I did not run into the pin_memory error with sam3_tracker_video. This PR is for sam3_video. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43061). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam3_video