Repository: huggingface/transformers
Pull request: fix(models, testing): Fix Llama4 vision rotary meta tensor initialization and MyT5 get_tokenizer signature

### What does this PR do?

The following issues were identified and fixed in this PR:

→ **Llama-4 Vision:** [freqs_ci is stored as a plain attr](https://github.com/huggingface/transformers/blob/153894c013/src/transformers/models/llama4/modeling_llama4.py#L1032) in [Llama4VisionRotaryEmbedding](https://github.com/huggingface/transformers/blob/153894c013/src/transformers/models/llama4/modeling_llama4.py#L1013). When `from_pretrained` initializes the model with `device_map="auto"`, all tensors become meta tensors, but `freqs_ci` is not registered and never materialized to device giving an error when copying out of the meta tensor. Fixed by registering it as a buffer and adding a meta-device recompute guard in `forward`. 
→ **MyT5:** 05c0e1d390 ("rm slow tokenizers") refactored [TokenizerTesterMixin.get_tokenizer](https://github.com/huggingface/transformers/blob/main/tests/test_tokenization_common.py#L417) to accept `pretrained_name` as a positional argument and changed the call sites in the base class tests accordingly, but [MyT5TokenizationTest.get_tokenizer](https://github.com/huggingface/transformers/blob/main/tests/models/myt5/test_tokenization_myt5.py#L88) was never updated to match; this should fix that. Took the canonical pattern from the other models (BARTpho, CANINE, CLVP, etc.) and used it here.

cc: @Rocketknight1 @itazap

**[CI Failures](https://github.com/huggingface/transformers/actions/runs/22885280567)**

**Before the fixes (feel free to cross-check; these errors are reproducible):**

<img width="800" height="700" alt="4" src="https://github.com/user-attachments/assets/0df6bc05-8ba7-41ad-b6ce-a7d206aa6d44" />
<img width="400" height="500" alt="image" src="https://github.com/user-attachments/assets/aaa7b2ae-301f-4e38-9ddb-4dcb364061b4" />
<img width="400" height="500" alt="image" src="https://github.com/user-attachments/assets/c2b3ecef-f7ee-4e28-9884-ec2fa66d817f" />

**After the fixes (feel free to cross-check):**

<img width="800" height="400" alt="2" src="https://github.com/user-attachments/assets/0b3a095c-ad19-4fc0-98df-25610392dee1" />
<img width="400" height="700" alt="image" src="https://github.com/user-attachments/assets/66616bfd-8319-48a9-b840-d0886d1e5063" />
<img width="400" height="700" alt="image" src="https://github.com/user-attachments/assets/b29ed25d-e715-4c4c-a0a4-9524473f7574" />

### Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you fix any necessary existing tests?

Changed files:
filename: src/transformers/models/llama4/modeling_llama4.py
patch: @@ -485,6 +485,8 @@ def _init_weights(self, module):
         if isinstance(module, Llama4TextExperts):
             init.normal_(module.gate_up_proj, mean=0.0, std=std)
             init.normal_(module.down_proj, mean=0.0, std=std)
+        elif isinstance(module, Llama4VisionRotaryEmbedding):
+            init.copy_(module.freqs_ci, module._compute_freqs_ci(module.config))
         elif isinstance(module, Llama4VisionModel):
             init.normal_(module.class_embedding, std=module.scale)
             init.normal_(module.positional_embedding_vlm, std=module.scale)
@@ -1001,6 +1003,11 @@ def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
 class Llama4VisionRotaryEmbedding(nn.Module):
     def __init__(self, config: Llama4VisionConfig):
         super().__init__()
+        self.config = config
+        self.register_buffer("freqs_ci", self._compute_freqs_ci(config), persistent=False)
+
+    @staticmethod
+    def _compute_freqs_ci(config):
         idx = config.image_size // config.patch_size
         img_idx = torch.arange(idx**2, dtype=torch.int32).reshape(idx**2, 1)
         img_idx = torch.cat([img_idx, img_idx[:1]], dim=0)
@@ -1017,7 +1024,7 @@ def __init__(self, config: Llama4VisionConfig):
         freqs = torch.cat([freqs_x, freqs_y], dim=-1).float().contiguous()[..., ::2]
         freqs = freqs.masked_fill(img_idx.reshape(-1, 1, 1) < 0, 0)
         freq_cis = torch.view_as_complex(torch.stack([torch.cos(freqs), torch.sin(freqs)], dim=-1))
-        self.freqs_ci = freq_cis  # idx**2, idx**2, idx * 2
+        return freq_cis  # idx**2, idx**2, idx * 2
 
     def forward(self, hidden_states):
         return self.freqs_ci.to(hidden_states.device)

filename: tests/models/myt5/test_tokenization_myt5.py
patch: @@ -85,7 +85,7 @@ class MyT5TokenizationTest(TokenizerTesterMixin, unittest.TestCase):
     from_pretrained_id = "Tomlim/myt5-base"
     test_rust_tokenizer = False
 
-    def get_tokenizer(cls, **kwargs) -> MyT5Tokenizer:
+    def get_tokenizer(cls, pretrained_name=None, **kwargs) -> MyT5Tokenizer:
         return cls.tokenizer_class.from_pretrained("Tomlim/myt5-base", **kwargs)
 
     @unittest.skip(reason="inputs cannot be pretokenized as ids depend on whole input string")