noamaanMulla-03/transformers-issues huggingface dataset:
test: add SDPA gradient health tests (fixes #44928)
Author: Lemniscate-world
Pull request: yes
Labels: name: Code agent slop

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47024)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47024)
<!-- ci-dashboard-badge:end -->

Fixes #44928

## Summary
Adds gradient health validation tests for scaled_dot_product_attention with dense attention masks. These tests catch gradient explosion bugs such as #44928, where 3D position_ids force SDPA Math fallback, causing BF16 collapse and NaN gradients in Qwen3.5 RLHF training.

## Tests added
- **test_sdpa_finite_gradient_standard**: SDPA must produce finite gradients with standard inputs
- **test_sdpa_finite_gradient_dense_mask**: Dense causal mask must not cause NaN/exploding gradients
- **test_sdpa_gradient_scaling_consistency**: Gradient scales approximately linearly with query scaling

## Motivation
Issue #44928 reports catastrophic gradient explosion (NaN) when using Qwen3.5 with 3D position_ids that force SDPA Math backend fallback under BF16. These tests establish baseline gradient health expectations and would have caught the regression.

## Detection
These patterns are monitored by [NeuralDBG](https://github.com/LambdaSection/NeuralDBG), a causal inference engine for DL training, via its gradient health transition events.

Comments:
Detected via [NeuralDBG](https://github.com/LambdaSection/NeuralDBG) gradient health monitoring. The test patterns here (finite check, dense mask safety, scaling consistency) mirror the gradient_health_transition events that NeuralDBG captures at runtime.

CC @ArthurZucker for visibility (related to SDPA attention).

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47024)
**Latest run:** [28593785482](https://github.com/huggingface/transformers/actions/runs/28593785482)
**Result:** `failure` | Grafana metrics are not available yet.

> **Code quality check failed**: test jobs were skipped. Fix the code quality issues and push again to run tests.

<!-- ci-dashboard-recap:end -->

This PR was closed but the issue #44928 remains open. Was there a CI failure I should address? Happy to fix and reopen. The tests validate SDPA gradient correctness with dense masks — a real bug affecting Qwen3.5 RLHF training.
Add HunYuan VL model
Author: Mi-Jiazhi
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46417)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46417)
<!-- ci-dashboard-badge:end -->

This PR adds support for **HunYuanVL**, Tencent’s open-source vision–language model. The text backbone **reuses `HunYuanDenseV1`**; vision, multimodal `forward` / `generate`, and the processor/image stack are implemented via the **modular** path.

Comments:
Great model to have in the library imo! Let's resolve merge conflicts and make CI green first, and I will try to come and make a first pass next week

I've updated `hunyuan_vl.md` and refactored the `xdrope section`. The remaining comments will be addressed later.

@Mi-Jiazhi I would focus on @zucchini-nlp's last comments and on the position ids / rope portion. A lot of my commens are rather nits or things we can cleanup afterwards (nothing major)

I fixed the comment of @zucchini-nlp and most of the comments of @vasqu.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46417). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: auto, hunyuan_vl

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/28650887566)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/auto", "models/hunyuan_vl"]
quantizations: []

run-slow: hunyuan_vl

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/28653917084)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/hunyuan_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/28653917084)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [311eb320](https://github.com/huggingface/transformers/commit/311eb32051d3b74008a481b7e1c278c06ef9ce95) | workflow commit (merge commit) |
| PR | [4bb10eeb](https://github.com/Mi-Jiazhi/transformers/commit/4bb10eeb16e83ebf0a814df5f6cb1789373b1d93) | branch commit (from PR) |
| main | [9d011e83](https://github.com/huggingface/transformers/commit/9d011e8375ab802f1b9ada2d488af35232f6dd3e) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !


Anton, feel free to merge if you're fine with current state

Merging then :hugs: thanks a lot everyone

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, hunyuan_vl

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46417)
**Latest run:** [28661723445:2](https://github.com/huggingface/transformers/actions/runs/28661723445)
**Result:** `success` | **Jobs:** 14 | **Tests:** 47,440 | **Failures:** 0 | **Duration:** 17h 58m

<!-- ci-dashboard-recap:end -->
Fix/qwen3 variants tp plan
Author: tengomucho
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47041)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47041)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

This is to fix #46846, and it is based on #46847 by @muhamedfazalps, as discussed in the original PR with @vasqu. 

This PR adds the missing tensor parallel sharding plan entries for the GatedDeltaNet linear attention projections across the Qwen3 variants. The five projections used by the linear attention layers, `in_proj_qkv`, `in_proj_z`, `in_proj_b`, `in_proj_a`, and `out_proj`, are now sharded as `colwise_gather_output`. The change is applied consistently to qwen3_5, `qwen3_5_moe`, and `qwen3_next`, keeping the modular file and the generated configuration in sync for each model.
Without these entries the linear attention weights stayed unsharded at TP greater than one, causing an OOM on load for the sharded checkpoints TP>1. The additions mirror the existing attention projection plan already present for the full attention layers.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_5, qwen3_5_moe, qwen3_next

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47041)
**Latest run:** [28661082360](https://github.com/huggingface/transformers/actions/runs/28661082360)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47041). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update workflow callers to use `transformers-ci`
Author: ydshieh
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47040)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47040)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

[See https://github.com/huggingface/transformers-test-ci/pull/3](https://github.com/huggingface/transformers-ci/pull/11)

As discussed offline, this transformers-ci would be our lovely home for CI.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47040). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: vit

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47040)
**Latest run:** [28665121318:1](https://github.com/huggingface/transformers/actions/runs/28666107815)
**Result:** `success` | **Jobs:** 12 | **Tests:** 49,391 | **Failures:** 0 | **Duration:** 6h 46m

<!-- ci-dashboard-recap:end -->
Add use_temp_cache_if_readonly decorator to testing_utils
Author: ydshieh
Pull request: yes

## Summary

- Adds a `use_temp_cache_if_readonly` decorator to `src/transformers/testing_utils.py` for tests that call `from_pretrained` with a model not yet present in the shared, read-only CI cache.
- On first attempt the test runs normally (hitting the cache if the model is there). If `OSError: [Errno 30] Read-only file system` is raised, the test is transparently retried inside a writable `TemporaryDirectory`.
- Patches `hf_constants.HF_HUB_CACHE` and `hf_constants.HF_XET_CACHE` directly (via `mock.patch.object`) rather than env vars, since both constants are resolved at import time.

## Motivation

Contributors adding new tests with `from_pretrained("org/new-model")` where the model is not yet in the shared CI cache would get an opaque `[Errno 30]` failure. This decorator lets them opt in to a graceful fallback without boilerplate `TemporaryDirectory` + `mock.patch.dict` setup in every test.

## Usage

```python
from transformers.testing_utils import use_temp_cache_if_readonly

@use_temp_cache_if_readonly
def test_my_new_model(self):
    model = AutoModel.from_pretrained("org/new-model")
```

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46768)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46768). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update dev
Author: vasqu
Pull request: yes

As per title cc @ArthurZucker @Cyrilvallez 

Merging as it's just versioning 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47044). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
ImportError: cannot import name 'AutoModelWithLMHead' from 'transformers'
Author: hannahlhargrove
Pull request: no

Hi, I am in a virtual conda environment, trying to import AutoModelWithLMHead, AutoTokenizer, and EsmModel. I can't change the models being used, because I am trying to recreate a peer-reviewed source model code. I have installed pytorch (2.12.0), and transformers (5.10.0.dev0). When I go to load in the requisite packages, I get the error that the module(s) listed does not exist. Any assistance on this would be very appreciated, thank you!

Comments:
```AutoModelWithLMHead``` is a deprecated API that was removed in newer Transformers releases. Since you're reproducing older research code and can't modify the model implementation, pin to an older stable version instead of the ```5.10.0.dev0``` dev build:

```python
pip install transformers==4.28.0
```

If you hit PyTorch compatibility issues:

```python
pip install transformers==4.28.0 torch==1.13.1
```

Also check the original paper's repo for a ```requirements.txt``` or ```environment.yml``` — research code is usually only validated against specific stable versions, and dev/nightly builds like ```5.10.0.dev0``` remove deprecated APIs aggressively.

Hey @hannahlhargrove,

AutoModelWithLMHead was deprecated 
a while back and has been fully removed 
in recent versions of transformers.

Depending on your model, try one of these:

# For causal/decoder models (GPT etc.)
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "your-model-name")

# For ESM specifically
from transformers import EsmModel
model = EsmModel.from_pretrained(
    "facebook/esm2_t6_8M_UR50D")

AutoTokenizer is fine as-is, no changes needed.

What model are you trying to load? 
Can help narrow it down further.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Could not import module 'Qwen3_5ForCausalLM
Author: Arslan-Mehmood1
Pull request: no
Labels: name: bug

### System Info

unable to run https://huggingface.co/NemoStation/Marlin-2B

tried the given requirements as well as transformers==5.9.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

.

### Expected behavior

.

Comments:
Hey, could you share a minimal repro script along with your error traceback? Hard to debug without it.

Hey @matdou , 

https://huggingface.co/NemoStation/Marlin-2B
```
import torch
from transformers import AutoModelForCausalLM

marlin = AutoModelForCausalLM.from_pretrained(
    "NemoStation/Marlin-2B",
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map={"": "cuda"},
)
marlin.compile()  # optional — wraps torch.compile, faster after first call

```

Hi @Arslan-Mehmood1 

I've tried running your repro script but it does work correctly for me. A few things to check:
1. `NemoStation/Marlin-2B` is a gated repo. You need to visit https://huggingface.co/NemoStation/Marlin-2B, fill out the access form, and wait for approval (it's auto-approved, so it's instant). Then authenticate locally ([docs](https://huggingface.co/docs/huggingface_hub/en/guides/cli#hf-auth-login)): `hf auth login` or set `HF_TOKEN=<your token>` in your environment.
2. Using `device_map` requires `accelerate`, make sure you have it installed on your environment.

Once those things are in place the model loads and runs fine. What error are you seeing exactly?

https://huggingface.co/learn/llm-course/chapter8/5

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
is_torch_hpu_available permanently monkeypatches global torch functions
Author: olliezyuhscripta
Pull request: no

﻿`is_torch_hpu_available` in import_utils.py permanently monkeypatches global torch functions

```python
torch.gather = patched_gather
torch.Tensor.gather = patched_gather
torch.take_along_dim = patched_take_along_dim
torch.linalg.cholesky = safe_cholesky  # adds jitter when NaN detected
torch.scatter = patched_scatter        # clones src in certain cases
torch.Tensor.scatter = patched_scatter
torch.compile = hpu_backend_compile    # overrides backend to "hpu_backend"
```

these are cached with @lru_cache so they run once and permanently change core torch behavior globally. anyone who imports transformers on an HPU system (or in an env where torch.hpu is detectable) gets these patches applied to everything, not just transformers code

the `safe_cholesky` one is especially concerning - it silently adds noise to numerical results when cholesky fails, which could mask bugs or change outputs in other libraries that depend on stock torch behavior

the `torch.compile` override silently changes the backend to hpu_backend even when users pass a different backend kwarg

should at minimum warn users that this is happening, or ideally scope these patches to only apply within transformers' own model code


Comments:
Hmn, I think this is okay? We're globally monkeypatching, but only in HPU systems (systems using Intel's Habana/Gaudi hardware). I don't think that will affect most users, and there's probably good reasons for this. The only side effect would be in extremely rare mixed HPU/CUDA systems, or something like that.

Hi, I'd like to work on this. I'll submit a PR shortly.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
CohereASR training-loss double-shift bug (same pattern as Moonshine #46784)
Author: sharmax-vikas
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.13.0.dev0 (main branch at commit dc5a497c18)
- - Platform: Windows
- - - Python version: 3.x
- - - - Using GPU: N/A (code-level bug, not runtime)

### Who can help?

@eustlb @ebezzam @vasqu

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

aIn `src/transformers/models/cohere_asr/modular_cohere_asr.py`, `CohereAsrForConditionalGeneration.forward()` has a double-shift bug:

1. Labels are shifted right via `shift_tokens_right()` (line 483) to create `decoder_input_ids`
2. 2. Then `self.loss_function()` is called (line 503), which maps to `ForCausalLMLoss`
3. 3. `ForCausalLMLoss` shifts labels again internally (`labels[..., 1:]` in `loss/loss_utils.py` line 63-64)
This means the model trains against `labels[..., 1:]` instead of `labels`.

This is the **exact same bug** that was fixed in Moonshine in PR #46784 (commit d8c235494e). The Moonshine fix replaced `self.loss_function()` with `CrossEntropyLoss()`, but `CohereAsr` — which inherits from Moonshine and overrides `forward` — was not updated.

```python
# Buggy code (line 503 in modular_cohere_asr.py):
loss = self.loss_function(logits=logits, labels=labels, vocab_size=self.config.vocab_size)

# Should be:
loss_fct = CrossEntropyLoss()
loss = loss_fct(logits.reshape(-1, self.config.vocab_size), labels.reshape(-1))
```

### Expected behavior

Training loss should be computed directly against `labels` without any additional shifting, since `shift_tokens_right()` already handles the decoder input alignment. This matches the behavior of Whisper, Bart, and the now-fixed Moonshine model. The fix should use `CrossEntropyLoss()` instead of `self.loss_function()`, as done in Moonshine PR #46784.

Comments:
Reproduced on current main (`b70d02fc72`). With a tiny random-init `CohereAsrForConditionalGeneration`, the returned loss exactly equals `CE(logits[..., :-1, :], labels[..., 1:])` and not `CE(logits, labels)`:

```
model loss:            4.990426
aligned CE (correct):  4.985212
double-shift CE (bug): 4.990426
```

So the model does train against `labels[..., 1:]` — `forward` already right-shifts labels into `decoder_input_ids`, then `ForCausalLMLoss` shifts a second time. `CohereAsr` overrides `forward`, which is why the Moonshine fix in #46784 didn't reach it.

I'll open a PR mirroring that fix: plain `CrossEntropyLoss` in `modular_cohere_asr.py` + regenerated modeling file, with the same regression test (with and without `-100` padding).
Cache config max_cache_length only honored for Quantized Cache
Author: dacorvo
Pull request: no
Labels: name: bug

### System Info

`GenerationConfig.cache_config` is documented as carrying "arguments used in the key-value cache class", but on the static-cache path it is silently ignored. A user setting `cache_config={"max_cache_len": N}` together with `cache_implementation="static"` gets a cache sized to `max_new_tokens + input_length - 1`, not `N`.

Concrete repro: prompt of 32 tokens, `max_new_tokens=16`, `cache_config={"max_cache_len": 2048}` → `model._cache.max_cache_len == 47`, `layers[0].keys.shape == [1, 8, 47, 64]`. Verified in [evidence/cache_config_max_cache_len.py](https://github.com/dacorvo/transformers-generation-compile/blob/main/evidence/cache_config_max_cache_len.py).

The miswiring is in `_prepare_cache_for_generation` ([generation/utils.py:1932-1935](https://github.com/huggingface/transformers/blob/v5.10.1/src/transformers/generation/utils.py#L1932-L1935)): `cache_config` is read on the quantized branch only. The static branch hardcodes `max_cache_len=max_cache_length` (= `max_length - 1`) and never looks at `cache_config`.

Cost on Llama-3.2-1B, A10G, bf16, 1024-token prompt, `prefill_chunk_size=1024`: a user calling `generate()` with `max_new_tokens=128` and then `max_new_tokens=256` pays a full `StaticCache` realloc + Inductor recompile on the second call (+19 artifacts, 27 s instead of ~2.7 s), because the auto-allocated cache is sized for the first call's `max_length` and grows on the second. The natural workaround — pin the cache to the worst-case `max_cache_len` via `cache_config` — is the one that doesn't work.

### Who can help?

@ArthurZucker @Cyrilvallez 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

https://github.com/dacorvo/transformers-generation-compile/blob/main/README.md

### Expected behavior

When using `cache_implementation="static"`, setting `generation_config.cache_config["max_cache_len"]` should size the auto-allocated `StaticCache` accordingly, so repeated calls with `max_new_tokens` / prompt length up to that ceiling reuse the same cache without realloc + recompile.

Comments:
cc @cyrilvallez for generation I guess?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Resolved in #46446
fix: use _added_tokens_encoder cache instead of property in convert_tokens_to_string and get_vocab
Author: Achyuthan-S
Pull request: yes
Labels: name: Code agent slop

Fixes #46396

Follow-up to #46323, which fixed `_convert_token_to_id_with_added_voc`.

The `added_tokens_encoder` property rebuilds and re-sorts the full token
map on every access. This PR replaces remaining per-token calls to the
property with direct reads from the maintained `_added_tokens_encoder`
cache in `convert_tokens_to_string()` and `get_vocab()` across four
slow tokenizers: byt5, myt5, dia, and perceiver.

9 replacements across 4 files. The property itself and
`tokenization_cpmant.py` are intentionally untouched (Option B from
the issue).

**Tests run:**
- `tests/models/byt5/test_tokenization_byt5.py`
- `tests/models/myt5/test_tokenization_myt5.py`
- `tests/models/dia/test_tokenization_dia.py`
- `tests/models/perceiver/test_tokenization_perceiver.py`
- `tests/test_tokenization_common.py`

All passing (160 + 18 passed, 0 failures).


cc @itazap

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, byt5, dia, myt5, perceiver, qwen3_5