noamaanMulla-03/transformers-issues huggingface dataset:
Guard torchcodec imports against OSError
Author: Functionhx
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47206)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47206)
<!-- ci-dashboard-badge:end -->

Fixes #47070

torchcodec can be installed but fail to import (OSError) when its native extension has missing dependencies. Wrap the import in the ASR and audio-classification pipelines, plus video_utils, so these failures don't crash pipelines that don't actually need torchcodec.

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47206)
**Latest run:** [29018873432:2](https://github.com/huggingface/transformers/actions/runs/29018873432)
**Result:** `failure` | **Jobs:** 2 | **Tests:** 22 | **Failures:** 1 | **Duration:** 2m 14s

> **Code quality check failed**: test jobs were skipped. Fix the code quality issues and push again to run tests.

<!-- ci-dashboard-recap:end -->

Closed: maintainer has previously rejected this defensive-import approach (see #47071). The position is that broken torchcodec installations are a user environment issue, not something transformers should work around.
No agent PR descriptions
Author: Rocketknight1
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=45790)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=45790)
<!-- ci-dashboard-badge:end -->

Experimenting with a rule against agent-written PR descriptions. This is deliberate friction, basically; it's intended to filter out low-effort contributions. We're still getting quite a lot of spam PRs where the user has clearly not bothered to even read the code they're submitting, so we're requiring some "proof of work" before we review a PR.

I'm also removing the "fail-closed" block in `AGENTS.md` because I think it mostly reiterates things already said above.

Comments:
@qgallouedec yeah, good suggestions and thanks for spotting the typo!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45790). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Can AI be used to paraphrase & fix grammar in the intended message to bring clarity? I'm wondering if that counts as ai written PR

I think that's fine, we don't want to discriminate against people just because English isn't their native language. AI translation / correction isn't a problem!

> Can AI be used to paraphrase & fix grammar in the intended message to bring clarity? I'm wondering if that counts as ai written PR

@rigen1048 to be clear, the only usage we’re trying to discourage right now is fully automated contributions. E.g. opening an agent and prompting it to `"find an issue, solve it, and open a PR.`"

Everything else is welcome. You’re absolutely encouraged to use AI extensively to **assist** you with your contributions, as long as you remain in the loop.

So why do we target PR description here? Because a common pattern to all purely AI generated PR is a purely AI generated PR description.

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=45790)
**Latest run:** [28876102121:1](https://github.com/huggingface/transformers/actions/runs/28876645649)
**Result:** `success` | **Jobs:** 2 | **Tests:** 24 | **Failures:** 0 | **Duration:** 2m 4s

<!-- ci-dashboard-recap:end -->
Make executorch exporter tests always use xnnpack backend
Author: tarekziade
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47201)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47201)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

The ExecuTorch export tests target CPU (via xnnpack on CPU), and its deps now live in the CPU torch-light CI image. Mark test_executorch_export with a new require_non_gpu decorator so they run on the CPU image, not the GPU nightly.

Lower DynamoExporter.min_versions torch 2.11.0 -> 2.10.0 so the tests are not skipped on the pinned torch<=2.10.0 CPU image.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47201). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

now running with https://github.com/huggingface/transformers/pull/47204 via the docker `:dev` image via `push-ci-image`

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47201)
**Latest run:** [29010616783:1](https://github.com/huggingface/transformers/actions/runs/29010616783)
**Result:** `success` | **Jobs:** 15 | **Tests:** 170,731 | **Failures:** 0 | **Duration:** 17h 12m

<!-- ci-dashboard-recap:end -->
Trigger a scheduled run
Author: ydshieh
Pull request: yes

# What does this PR do?

Today's run is failing, we don't have results.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47209). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix off-by-one in MoonshineStreaming sliding_window_mask_function right window
Author: namantiwari2002
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47162)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47162)
<!-- ci-dashboard-badge:end -->

## What

`sliding_window_mask_function` in `modeling_moonshine_streaming.py` (and its
`modular_` source) builds the right (future) side of the sliding-window
attention mask with a strict inequality:

```python
right_mask = (dist < 0) & (-dist < right_window_size)
```

With `dist = q_idx - kv_idx`, a future key is `dist < 0`, and `-dist` is the
number of frames the key is ahead of the query. The strict `<` drops the
frame exactly `right_window_size` steps ahead, so a configured right window of
`w_right = 4` only attends to 3 future frames instead of 4.

Concretely, `sliding_window_mask_function((16, 4))` for a query at position 10
returned future positions `{11, 12, 13}` instead of the expected
`{11, 12, 13, 14}`.

This does not match the Moonshine v2 definition, where `w_right = 4` implies an
algorithmic lookahead of `4 x 20 ms = 80 ms` (4 frames at 50 Hz), not 60 ms.

## Change

Use `<=` for the right-window comparison so the full right window is included:

```python
right_mask = (dist < 0) & (-dist <= right_window_size)
```

The left window and causal semantics are unchanged. The same generated copy in
`gemma4/modeling_gemma4.py` (which reuses this function via `modular`) is
updated to stay in sync.

Fixes #46305

## How tested

Added a unit test (`MoonshineStreamingSlidingWindowMaskTest`) asserting that a
query at position 10 with window `(16, 4)` attends to exactly
`{11, 12, 13, 14}`, that the frame 5 steps ahead is excluded, and that the left
window is unaffected.

```
python -m pytest tests/models/moonshine_streaming/test_modeling_moonshine_streaming.py::MoonshineStreamingSlidingWindowMaskTest -v
```

The fix logic was confirmed with a standalone reproduction script mirroring
the issue's repro case — output changed from `[11, 12, 13]` to `[11, 12, 13, 14]`.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma4, moonshine_streaming

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47162)
**Latest run:** [28924902300:2](https://github.com/huggingface/transformers/actions/runs/28924902300)
**Result:** `success` | **Jobs:** 7 | **Tests:** 1,045 | **Failures:** 0 | **Duration:** 15m 8s

<!-- ci-dashboard-recap:end -->

cc @eustlb @ebezzam since this is an audio sliding window issue. I think the bug is legit, and this code is imported into Gemma4! It's only used for Gemma4 audio, though, and I think the actual Gemma4 configs might mean it doesn't have much effect, but it's still probably important enough to check/fix.
Switch mlinter to 0.1.2
Author: tarekziade
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47172)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47172)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

runs on current mlinter 0.1.2 

models that fail to pass the linter

- TRF014 (trust_remote_code) │ auto,  legitimate loader exemption we can keep that
- TRF016 (dead do_* flags)   │ kimi_k25, minimax_m3_vl    
- TRF018 (super()._init_weights) | cosmos3_omni, diffusion_gemma, eomt_dinov3, exaone4_5, gemma4, higgs_audio_v2, kimi_k25, olmo3, pp_formulanet                      
- TRF019 (processor _default) |  blip_2, grounding_dino, hunyuan_vl, kosmos2, omdet_turbo, qwen3_asr

We will address failing models in follow-up PRs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47172). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Also I think current CI doesn't run on new mlinter, so we need to check it locally, but might be wrong

> Also I think current CI doesn't run on new mlinter, so we need to check it locally, but might be wrong

the current CI runs on mlinter 0.1.1 - this branch runs on main (future 0.1.2) see https://github.com/huggingface/transformers/actions/runs/28942558779/job/85868216161?pr=47172

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47172)
**Latest run:** [29018294686:1](https://github.com/huggingface/transformers/actions/runs/29020542489)
**Result:** `success` | **Jobs:** 14 | **Tests:** 170,474 | **Failures:** 0 | **Duration:** 15h 14m

<!-- ci-dashboard-recap:end -->
Fix double label shift in CohereAsr training loss
Author: Diwak4r
Pull request: yes
Labels: name: Code agent slop

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47200)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47200)
<!-- ci-dashboard-badge:end -->

## What does this PR do?

Fixes a training-loss bug in `CohereAsrForConditionalGeneration` where labels are shifted twice.

`CohereAsrForConditionalGeneration.forward()` (in both `modular_cohere_asr.py` and the generated `modeling_cohere_asr.py`) calls `shift_tokens_right()` on `labels` to build `decoder_input_ids`, then passes the **same** `labels` to `self.loss_function()`. That maps to `ForCausalLMLoss`, which shifts `labels` again internally (`labels[..., 1:]` in `src/transformers/loss/loss_utils.py`). The result is a double shift: the model trains against `labels[..., 1:]` instead of the real `labels`, degrading training.

This is the exact pattern that was fixed for Moonshine in PR #46784. CohereAsr inherits from Moonshine and overrides `forward()` but was never updated.

### Fix
Replace `self.loss_function(logits=..., labels=labels, vocab_size=...)` with a plain `CrossEntropyLoss` against the already-aligned `labels`, matching Whisper, Bart, and the fixed Moonshine:

```python
loss_fct = CrossEntropyLoss()
loss = loss_fct(logits.reshape(-1, self.config.vocab_size), labels.reshape(-1))
```

Applied to the modular source file (source of truth) and mirrored in the auto-generated `modeling_cohere_asr.py`.

## Verification
- Confirmed at the loss level that `ForCausalLMLoss(logits, labels)` produces a value that differs from a direct `CrossEntropyLoss` on `(logits, labels)` (i.e. the double shift is real).
- Added `test_training_loss_no_double_shift` to `tests/models/cohere_asr/test_modeling_cohere_asr.py`, mirroring the Moonshine regression test: for both unpadded and `-100`-padded labels, the reported loss equals an aligned CE and is **not** equal to a double-shifted CE.
- `ruff check` passes on both edited files.

Fixes #46894.

## Before submitting
- [x] This PR fixes a bug (does not add new models/features)
- [x] Tests added/updated

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cohere_asr

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47200)
**Latest run:** [29003879081:2](https://github.com/huggingface/transformers/actions/runs/29003879081)
**Result:** `success` | **Jobs:** 5 | **Tests:** 327 | **Failures:** 0 | **Duration:** 3m 49s

<!-- ci-dashboard-recap:end -->

Duplicate of #46895, blocked for just running your code agent without checking
Fix YOLOS device mismatch with device_map="auto"
Author: swankystark
Pull request: yes
Labels: name: Code agent slop

# What does this PR do?

Fixes #46869.

When loading YOLOS with `device_map="auto"`, `mid_position_embeddings` remains on the device where it was initialized, while `hidden_states` follows the device of the current `YolosLayer`. When layers are distributed across multiple GPUs, the addition

```python
hidden_states = hidden_states + interpolated_mid_position_embeddings[i]
```

can therefore fail with:

```text
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
```

This PR moves `interpolated_mid_position_embeddings[i]` to `hidden_states.device` immediately before the addition, matching the device-alignment pattern already used by several other models in the repository (e.g. HuBERT, Wav2Vec2, Data2Vec Audio).

The existing `ModelTesterMixin.test_model_parallelism` already exercises this code path on multi-GPU CI by loading the model with `device_map="auto"` and executing a forward pass, so no additional regression test is added.

## Before submitting

* [x] I confirm that this is **not** a pure code agent PR. I investigated the issue, traced the root cause in `YolosEncoder.forward()`, implemented the fix, and validated it against existing model-parallel patterns in the repository.
* [ ] This PR fixes a typo or improves the docs.
* [x] I read the contributor guidelines and PR checks.
* [x] This change was discussed in Issue #46869.
* [ ] Documentation update is not required.
* [x] No new tests were added because the existing `ModelTesterMixin.test_model_parallelism` already covers this multi-GPU execution path.

## Who can review?

@molbap


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46886)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: yolos

Thanks for catching that. I've updated the PR accordingly.

Reverted the .to(hidden_states.device) workaround.
Added _no_split_modules = ["YolosEmbeddings", "YolosLayer"] following the existing ViT-family pattern.

I also ran ruff check and ruff format --check on the modified file. Thanks again for the review!
[docs] n-d parallelism
Author: stevhliu
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=44775)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=44775)
<!-- ci-dashboard-badge:end -->

refactors the current [Parallelism methods](https://huggingface.co/docs/transformers/main/en/perf_train_gpu_many#zero-data-parallelism-pipeline-parallelism-and-model-parallelism-3d-parallelism) doc to:

- focus on practical examples of combining strategies
- adds use-case framing for each combination to make it more actionable
- removes strategies that aren't currently supported yet (PP) or are experimental (EP)

todo:
- [ ] add link to #44420 once merged
- [x] add link to #44605 once merged
- [x] add link to #44613 once merged

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_44775). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=44775)
**Latest run:** [29037662996:1](https://github.com/huggingface/transformers/actions/runs/29037662996)
**Result:** `success` | **Jobs:** 2 | **Tests:** 24 | **Failures:** 0 | **Duration:** 1m 57s

<!-- ci-dashboard-recap:end -->
Fix chain-pair ipTM to use max-of-row-mean in ESMFold2 head
Author: faustomilletari
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47221)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47221)
<!-- ci-dashboard-badge:end -->

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47221)
**Latest run:** [29040612123:2](https://github.com/huggingface/transformers/actions/runs/29040612123)
**Result:** `failure` | **Jobs:** 14 | **Tests:** 158,584 | **Failures:** 7 | **Duration:** 21h 24m

<!-- ci-dashboard-recap:end -->

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: esmfold2

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47221). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Revert 45777
Author: Rocketknight1
Pull request: yes

#45777 removed some lines in the conftest which work around a torch issue, but the Torch fix hasn't propagated through to our CI version of Torch yet, so we need to put them back for now!

cc @vasqu 

Comments:
cc @ydshieh @khushali9, what version of Torch do we need in the CI before we can remove those lines again?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45942). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

it's torch 2.12, not released yet I think

 @Rocketknight1 I checked it is not 2.12 but the next release 2.13 on June 8.  I will keep a check and create my PR again to have it merge once Pytorch version is released. cc @ydshieh Thanks!

Oh yes, sorry indeed 2.13. Thank you for handling it.

You can use  `is_torch_greater_or_equal` from `src/transformers/utils/__init__.py` that is defined in `src/transformers/utils/import_utils.py`

Sure @ydshieh thanks

@ydshieh I checked pytorch 2.13 release is not out yet, I am keeping an eye, I will make this changes as soon as they have release out. 

@ydshieh my PR is out as Pytorch has 2.13 release with my bug fixes in. Also kept version aware fix in as you mentioned earlier. https://github.com/huggingface/transformers/pull/47222 Can you review it. Thanks
fix: resolve serge integration test failures (#47182)
Author: maslinedwin
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47224)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47224)
<!-- ci-dashboard-badge:end -->

## Summary

This PR addresses multiple model integration test failures from the 2026-07-08 serge triage report (#47182):

- **Whisper generation** (22+ failures): Fix missing max_length assignment
- **EdgeTAM model** (12 failures): Filter out original_sizes kwarg before passing to backbone

## Details

### 1. Whisper Generation Fix
File: src/transformers/models/whisper/generation_whisper.py

In _set_max_new_tokens_and_length method, max_length is calculated but never assigned back to generation_config.max_length, causing generation to use the original smaller length.

Fix: Add assignment: generation_config.max_length = max_length

### 2. EdgeTAM Model Fix
Files: src/transformers/models/edgetam/modular_edgetam.py and modeling_edgetam.py

The original_sizes parameter is passed through kwargs to the backbone model which doesn't accept it.

Fix: Filter out original_sizes from kwargs before passing to backbone.

## Testing
- Syntax validation passed for all modified files
- Changes follow existing patterns in the codebase

## Related
Addresses issue #47182 (serge integration failure triage from 2026-07-08)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: edgetam, whisper

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47224)
**Latest run:** [29049164040:2](https://github.com/huggingface/transformers/actions/runs/29049164040)
**Result:** `failure` | **Jobs:** 15 | **Tests:** 170,662 | **Failures:** 1 | **Duration:** 18h 33m

<!-- ci-dashboard-recap:end -->
Add SAM 3.1 Object Multiplex video models
Author: Aravind-11
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47226)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47226)
<!-- ci-dashboard-badge:end -->

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, sam3, sam3_1_tracker_video, sam3_1_video, sam3_tracker_video, sam3_video

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47226)
**Latest run:** [29055017727](https://github.com/huggingface/transformers/actions/runs/29055017727)
**Result:** `failure` | Grafana metrics are not available yet.

> **Code quality check failed**: test jobs were skipped. Fix the code quality issues and push again to run tests.

<!-- ci-dashboard-recap:end -->

agent slop. Maintainers can ignore. I will delete this .