Fix flaky SAM-HQ integration tests by adding set_seed
Author: raimbekovm
Pull request: yes

# What does this PR do?

Fixes flaky `SamHQModelIntegrationTest` by adding `set_seed(0)` in `setUp()`.

The root cause: HF SAM-HQ creates two independent `SamHQPositionalEmbedding` instances (`shared_image_embedding` and `prompt_encoder.shared_embedding`), but the checkpoint only loads weights to `shared_image_embedding`. The other one stays randomly initialized, causing non-deterministic test results.

Note: This is a workaround. The proper fix would be to share the embedding between both (like the original SAM-HQ) or copy weights to both during loading. Happy to implement that instead if preferred.

Fixes #42890

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #42890
- [ ] Did you make sure to update the documentation with your changes?
- [ ] Did you write any new necessary tests?

## Who can review?

@NielsRogge @molbap @yonigozlan

Comments:
`_keys_to_ignore_on_load_missing` will only suppress the warning if the key is missing from the checkpoint, it shouldn't change the loading behaviour. Is the checkpoint missing that tensor for some reason?

You're right. I investigated the architecture:

The original SAM-HQ has one shared `pe_layer.positional_encoding_gaussian_matrix`. But the HF model creates two independent `SamHQPositionalEmbedding` instances:
- `shared_image_embedding` (SamHQModel.__init__)
- `prompt_encoder.shared_embedding` (SamHQPromptEncoder.__init__)

During conversion, the tensor is mapped only to `shared_image_embedding.positional_embedding`. The `prompt_encoder.shared_embedding.positional_embedding` remains randomly initialized.

The proper fix would be to either share the embedding between both (like the original) or copy the tensor to both locations during checkpoint loading. The `set_seed` approach is a workaround that masks this architectural mismatch.

Should I update the PR to fix the root cause instead?

Yes, I think so! At the very least a PR that did that would give us a good starting point to check what it fixes and if there are any regressions

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam_hq

Implemented the proper fix mentioned in the PR description.

**What changed:**
- Share `positional_embedding` between `shared_image_embedding` and `prompt_encoder.shared_embedding` (matching original SAM-HQ architecture)
- Use `_tied_weights_keys` to handle weight loading correctly
- Changed buffer to `nn.Parameter(requires_grad=False)` because `mark_tied_weights_as_initialized()` only works with parameters

**Why this approach:**
Tried several alternatives first:
- `_register_load_state_dict_pre_hook` / `register_load_state_dict_post_hook` — don't work with safetensors/accelerate loading
- `_load_from_state_dict` override — not supported by modular converter

The `_tied_weights_keys` mechanism works but requires `get_expanded_tied_weights_keys` override since the default implementation checks `tie_word_embeddings` (designed for language models).

**Tests:**
- Embeddings are now properly shared (same object)
- Weights load correctly from checkpoint
- Deterministic across multiple loads
- Forward pass works correctly

The `set_seed` workaround from the first commit is no longer needed but I left it as a safety net.

cc @NielsRogge @molbap @yonigozlan for review, since I'm not the expert on SAM models!

hello @raimbekovm ! Thanks for raising this issue. The issue appears to be more important than the scope of this PR, so I opened another one to fix everything that's wrong here https://github.com/huggingface/transformers/pull/43420

Glad I could help surface the issue — feel free to ping me if you need any help!