noamaanMulla-03/transformers-issues huggingface dataset:
[serge] Fix 8 integration tests for model `mamba2` failing with `OOM` (other (8))
Author: sergereview[bot]
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47282)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47282)
<!-- ci-dashboard-badge:end -->

## Original CI failure

- Failure group: `1/1: 8 integration tests for model `mamba2` failing with `OOM` (other (8))`
- `tests/models/mamba2/test_modeling_mamba2.py::Mamba2IntegrationTest::test_mamba2_mixer_train_vs_eval_equivalence` [multi-gpu] (OOM, seen 6/7)

<!-- serge-task:integration-failure-triage:sha256:2c9bcdbaeccfe5658f7aeec2e208cfa203a284a9a878fb77289d0457648c91e8 -->
Relates to #47279

The mamba2 integration tests OOM on 22 GiB CUDA devices when loading `mistralai/Mamba-Codestral-7B-v0.1` in bfloat16 and running batched/long-sequence checks. This patch lowers the test scale to fit the CI GPU budget without changing what is being verified:

- `test_mamba2_mixer_train_vs_eval_equivalence`: halve batch size and sequence length (`B=2, T=256`).
- `test_batched_equivalence_with/without_cache`: use 2 prompts instead of 3, generate 20 tokens instead of 30, and compare the first 80 characters.
- `test_simple_generate`: generate 20 tokens instead of 30 and truncate the device-specific expected strings accordingly.
- Add `torch.cuda.empty_cache()` after model setup in the GPU tests to reduce fragmentation between runs.

Note: the reported normalizer failure (`No module named 'transformers.models.tipsv2'` / `Tipsv2Config`) is unrelated to this patch — it stems from a missing model module in the repository and is outside the scope of the mamba2 OOM fix.

---
_This change was produced automatically by serge from a CI failure report. The patch was generated by an LLM and applied by serge; review before merging._

_serge `v0.1.0` · model: `moonshotai/Kimi-K2.7-Code` · 67 LLM turns · 68 tool calls · 312.6s · 2307542 in / 23747 out tokens_

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mamba2

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47282)
**Latest run:** [29216227806](https://github.com/huggingface/transformers/actions/runs/29216227806)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47282). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Revisit the condition for calling `compute_loss` at eval
Author: qgallouedec
Pull request: no

During eval, `Trainer` calls `prediction_step`. If no labels are present in the inputs, it only runs forward and
returns logits, and don't call `compute_loss`. Consequently you can't get your custom loss at eval

In trainers like DPO, we need to override the `prediction_step` like this to force the call to `compute_loss`:

```
    def prediction_step(self, model, inputs, prediction_loss_only, ignore_keys: list[str] | None = None):
        inputs = self._prepare_inputs(inputs)
        with torch.no_grad(), self.compute_loss_context_manager():
            if prediction_loss_only:
                loss = self.compute_loss(model, inputs, return_outputs=False)
                logits, labels = None, None
            else:
                loss, outputs = self.compute_loss(model, inputs, return_outputs=True)
                logits, labels = outputs.logits, inputs["input_ids"]
        return loss, logits, labels
```

see https://github.com/huggingface/trl/blob/7a530ba6d2ebf4101aaeb8002cd51c9bde4fb721/trl/trainer/dpo_trainer.py#L1412-L1423

Comments:
@qgallouedec Can I work on this and #43602?


Thanks, 
I think the first step is to discuss what exactly do we want before working on the change

@qgallouedec changes I think should be made are first there should be a new opt-in trainer attribute (compute_loss_in_eval), this attribute will declare wether a trainer subclass intends its compute_loss method to be invoked during evaluaton, even if inputs does not contain labels.
second, currently if there is labels present only then compute loss is called, (should_compute_loss = has_labels or self.compute_loss_in_eval) trainer should define wether loss should be computed.
What do you think about these changes ?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @qgallouedec, I'd like to pick this up. Building on Aaraviitkgp's proposal: add an opt-in compute_loss_in_eval flag on Trainer, and update prediction_step to call compute_loss when that flag is set even without labels. Does that match what you had in mind, or is there a different direction you'd prefer?
Fix failing tests for zaya
Author: kaixuanliu
Pull request: yes

This PR fixes 3 failed test cases. 1 common and 2 for XPU:
```
FAILED tests/models/zaya/test_modeling_zaya.py::ZayaModelTest::test_generate_with_quant_cache - ValueError: `QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer types: {'hybrid', 'hybrid_sliding'}
FAILED tests/models/zaya/test_modeling_zaya.py::ZayaIntegrationTest::test_model_generation - AssertionError: Lists differ: [107,[23 chars]888, 2088, 740, 564, 6361, 611, 3124, 236881, [15 chars] 735] != [107,[23 chars]888, 1030, 5724, 1133, 611, 236789, 500, 7467,[18 chars]5213]
FAILED tests/models/zaya/test_modeling_zaya.py::ZayaIntegrationTest::test_model_logits - AssertionError: Tensor-likes are not close!
```
@ydshieh pls help review, thx!

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: zaya

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47268)
**Latest run:** [29184344056](https://github.com/huggingface/transformers/actions/runs/29184344056)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->
Shard on read Dtensor aware
Author: 3outeille
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46717)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46717)
<!-- ci-dashboard-badge:end -->

# What does this PR do ?

This PR adds `shard-on-read` support for loading DTensor-sharded model parameters during `from_pretrained`. Instead of materializing the full checkpoint tensor on every rank and sharding afterward, each rank reads only the slice it needs from the checkpoint, based on the target parameter’s DTensor mesh and placements.

For now I don't want to break the integration with the existing TP on main (since we haven't merged the inference-compatible DTensor TP yet). So here's what I'm doing:

- I'm doing 2 loading paths for the moment (Dtensor and the old one for TP only)
- I fully integrate FSDP and get it working with shard-on-read to keep the distributed PR as small as possible
- Once only TP is left, I remove the second TP loading path and add DTensor inference for TP

Comments:
View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46717&sha=672bbb

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46717). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46717)
**Latest run:** [29221237811:1](https://github.com/huggingface/transformers/actions/runs/29221237811)
**Result:** `success` | **Jobs:** 15 | **Tests:** 170,779 | **Failures:** 0 | **Duration:** 15h 27m

<!-- ci-dashboard-recap:end -->
Fix failing tests for mimo_v2_flash
Author: kaixuanliu
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47284)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47284)
<!-- ci-dashboard-badge:end -->

Fix following test cases for mimo_v2_flash:
```
FAILED tests/models/mimo_v2_flash/test_modeling_mimo_v2_flash.py::MiMoV2FlashModelTest::test_generate_with_quant_cache - ValueError: `QuantizedCache` is only supported for models with only full attention layers. We found the following invalid layer types: {'sliding_attention'}
FAILED tests/models/mimo_v2_flash/test_modeling_mimo_v2_flash.py::MiMoV2FlashModelTest::test_kernels_can_load_without_crashing - ValueError: Attempted to register a kernel for apply_rotary_pos_emb, but it was not a `torch.nn.Module`. This means the underlying function needs to be decor
ated with `@use_kernel_func_from_hub`. Please submit and issue to the transformers repo: `https://github.com/huggingface/tra...
FAILED tests/models/mimo_v2_flash/test_modeling_mimo_v2_flash.py::MiMoV2FlashIntegrationTest::test_small_model_generation - ValueError: No matching expectation found for ('xpu', 5, None)
FAILED tests/models/mimo_v2_flash/test_modeling_mimo_v2_flash.py::MiMoV2FlashIntegrationTest::test_small_model_logits_batched - ValueError: No matching expectation found for ('xpu', 5, None)
```
@ydshieh pls help review, thx!

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mimo_v2_flash

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47284)
**Latest run:** [29229098200:2](https://github.com/huggingface/transformers/actions/runs/29229476328)
**Result:** `success` | **Jobs:** 7 | **Tests:** 349 | **Failures:** 0 | **Duration:** 3m 47s

<!-- ci-dashboard-recap:end -->
tipsv2_dpt: fix failing tests for XPU
Author: kaixuanliu
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47292)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47292)
<!-- ci-dashboard-badge:end -->

Fixed 4 test(s):
- tests/models/tipsv2_dpt/test_modeling_tipsv2_dpt.py::Tipsv2DptForDensePredictionIntegrationTest::test_inference_depth_estimation
- tests/models/tipsv2_dpt/test_modeling_tipsv2_dpt.py::Tipsv2DptForDensePredictionIntegrationTest::test_inference_model
- tests/models/tipsv2_dpt/test_modeling_tipsv2_dpt.py::Tipsv2DptForDensePredictionIntegrationTest::test_inference_normal_estimation
- tests/models/tipsv2_dpt/test_modeling_tipsv2_dpt.py::Tipsv2DptForDensePredictionIntegrationTest::test_inference_semantic_segmentation

@ydshieh pls help review, thx!

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: tipsv2_dpt

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47292)
**Latest run:** [29230313601](https://github.com/huggingface/transformers/actions/runs/29230313601)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->
Add KORMo model
Author: mjkmain
Pull request: no

### Model description

I'd like to add **KORMo** (Korean Open Reasoning Model) to Transformers as a native model.

KORMo is a fully open bilingual (Korean–English) LLM — weights, training code and training data
are all openly released. Paper: https://huggingface.co/papers/2510.09426

Architecturally, KORMo is identical to Llama (RMSNorm, grouped-query attention, RoPE, SwiGLU MLP).
The **only** difference is naming: the two decoder-layer RMSNorms are called
`pre_attention_layernorm` / `pre_mlp_layernorm` instead of Llama's
`input_layernorm` / `post_attention_layernorm`.

Because of this, it can be implemented cleanly with the `modular` mechanism (inheriting from
Llama and overriding just the decoder layer). Keeping the KORMo norm names means the existing
public checkpoints load without any weight renaming, which matters since the KORMo repos publish
many training-dynamics checkpoints.

I'm part of the KORMo team, so I can maintain this model.

### Open source status

- [x] The model implementation is available (currently via `trust_remote_code`)
- [x] The model weights are available: https://huggingface.co/KORMo-Team

### Provide useful links for the implementation

- Paper: https://huggingface.co/papers/2510.09426 (arXiv:2510.09426)
- Weights: https://huggingface.co/KORMo-Team/KORMo-10B-base , https://huggingface.co/KORMo-Team/KORMo-10B-sft

I already have a working `modular`-based implementation (inherits from Llama) that passes
`tests/models/kormo/` and the repo-consistency / code-quality checks locally, and I'm happy to
open the PR. Would a native integration be welcome?


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[DiffusionGemma] Cast the decoder padding mask to bool
Author: kashif
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47295)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47295)
<!-- ci-dashboard-badge:end -->

`create_diffusion_decoder_attention_mask` calls the low-level mask interface directly. That one keeps the dtype of the 2D mask it is given, unlike the high-level mask creation functions which cast it. So a `1`/`0` integer padding mask, like the one a tokenizer returns, ends up propagating its dtype and we get an int64 mask back instead of a boolean one, which then blows up in sdpa.

The existing mask tests all pass a `torch.bool` mask, so this never showed up. Added a test with an integer mask, it fails on main and passes here.

Found while training DiffusionGemma from TRL, which builds its `decoder_attention_mask` with `torch.long`.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: diffusion_gemma

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47295)
**Latest run:** [29239545345](https://github.com/huggingface/transformers/actions/runs/29239545345)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47295). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[DeepSpeed] Composite models are not initialized correctly under ZeRO-3
Author: kashif
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47297)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47297)
<!-- ci-dashboard-badge:end -->

Opening this as a draft with just a failing test, no fix, since the right fix is not obvious to me.

Under ZeRO-3, `AutoModel.from_config` on a composite model raises:

```
ValueError: Fan in and fan out can not be computed for tensor with fewer than 2 dimensions
```

`_from_config` builds the model under `zero.Init()` and then calls `initialize_weights_zero3`, which gathers each module's parameters before initializing them. For a composite model the sub-models are not covered, so `_init_weights` runs on the empty partitioned shards and the init functions that need the shape of the tensor blow up.

A plain model like Llama is fine, so this only shows up once there is a sub-model. The added test uses a tiny Gemma3.

This was hidden because the image running `tests/models` has no deepspeed, so every `require_deepspeed` test there is skipped. I put the test in the deepspeed file instead, as that one does run with it.

Happy to work on the fix if you can point me at how you want the sub-models handled.

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47297)
**Latest run:** [29241935700](https://github.com/huggingface/transformers/actions/runs/29241935700)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47297). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix MPS crash in build_2d_sinusoidal_position_embedding (float64 → float32)
Author: sridhar-3009
Pull request: yes
Labels: name: Code agent slop

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47234)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47234)
<!-- ci-dashboard-badge:end -->

Fixes #46159

## What happened

`build_2d_sinusoidal_position_embedding` (defined in `modular_vit_mae.py`, reused by RT-DETR and RT-DETRv2) already accepts a `dtype` parameter that defaults to `torch.float32` and converts its output via `.to(dtype)`. However, every internal tensor allocation hardcoded `torch.float64`, which MPS does not support, crashing with:

```
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
```

## Fix

Replace the three hardcoded `dtype=torch.float64` arguments inside the function body with `dtype=torch.float32`. The output is still cast to the caller-requested `dtype` at the end, so behaviour on CUDA/CPU is unchanged. Float32 precision is more than sufficient for sinusoidal position embeddings.

Changed files:
- `src/transformers/models/vit_mae/modular_vit_mae.py` (source of truth)
- `src/transformers/models/vit_mae/modeling_vit_mae.py` (generated)
- `src/transformers/models/rt_detr/modeling_rt_detr.py` (generated)
- `src/transformers/models/rt_detr_v2/modeling_rt_detr_v2.py` (generated)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: rt_detr, rt_detr_v2, vit_mae

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47234)
**Latest run:** [29069080309:2](https://github.com/huggingface/transformers/actions/runs/29069080309)
**Result:** `failure` | **Jobs:** 2 | **Tests:** 11 | **Failures:** 1 | **Duration:** 1m 7s

<!-- ci-dashboard-recap:end -->
Fix wrong method name call in TimesFM 2.5 preprocessing
Author: sridhar-3009
Pull request: yes
Labels: name: Code agent slop

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47229)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47229)
<!-- ci-dashboard-badge:end -->

## What

`_preprocess_inputs` in `modeling_timesfm2_5.py` calls `self._timesfm_moving_average` at line 780, but the method defined in the same class is `_timesfm2_5_moving_average`. The name was copied from the original TimesFM 1.0 model and never updated.

## Reproduction

Any call that passes `window_size` to a TimesFM 2.5 model raises `AttributeError: 'TimesFM2_5ForPrediction' object has no attribute '_timesfm_moving_average'`. Python also helpfully suggests `Did you mean: '_timesfm2_5_moving_average'?`.

## Fix

One-character change at the call site: rename `_timesfm_moving_average` → `_timesfm2_5_moving_average` to match the actual definition.

Closes #46821.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: timesfm2_5

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47229)
**Latest run:** [29068416248:2](https://github.com/huggingface/transformers/actions/runs/29068543449)
**Result:** `success` | **Jobs:** 2 | **Tests:** 11 | **Failures:** 1 | **Duration:** 47s

<!-- ci-dashboard-recap:end -->
Fix misleading T5Stack error messages when stack is used as decoder
Author: sridhar-3009
Pull request: yes
Labels: name: Code agent slop

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47232)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47232)
<!-- ci-dashboard-badge:end -->

## What

`T5Stack.forward` builds error messages like:

```
"You cannot specify both decoder_input_ids and decoder_inputs_embeds at the same time"
```

by prepending `"decoder_"` to the argument names when `self.is_decoder=True`. But when `T5Stack` is called directly (not through `T5ForConditionalGeneration`), the actual parameter names are `input_ids` / `inputs_embeds`, not `decoder_input_ids`. The prefix creates a mismatch that confuses users trying to debug the error.

## Fix

Change the prefix from `"decoder_"` to `"decoder's "` at both error sites (lines 647 and 657). The messages now read:

```
"You cannot specify both decoder's input_ids and decoder's inputs_embeds at the same time"
"You have to specify either decoder's input_ids or decoder's inputs_embeds"
```

These are accurate whether the stack is called directly or through the high-level API.

Fixes #46463.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: switch_transformers, t5

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47232)
**Latest run:** [29068650667:2](https://github.com/huggingface/transformers/actions/runs/29070287335)
**Result:** `failure` | **Jobs:** 6 | **Tests:** 782 | **Failures:** 1 | **Duration:** 3m 39s

<!-- ci-dashboard-recap:end -->