noamaanMulla-03/transformers-issues huggingface dataset:
DeformableDetrHungarianMatcher and LwDetrHungarianMatcher crash on non-finite cost matrices ("cost matrix is infeasible")
Author: MutugiD
Pull request: no
Labels: name: Code agent slop

### System Info

- `transformers` 5.14.0.dev0 (main)
- PyTorch 2.x
- Reproduces on CPU; originally hit on GPU with AMP

### Who can help?

@guarin @yonigozlan

### Description

Same failure mode as #47000 (RT-DETR), but in two more Hungarian matchers that are still unguarded on `main`. Under AMP/fp16 the sigmoid focal classification cost can overflow to `inf`/`NaN`; the resulting cost matrix is then passed unguarded to `scipy.optimize.linear_sum_assignment`, which raises `ValueError: cost matrix is infeasible`.

Affected (no guard on `main`):

- `DeformableDetrHungarianMatcher` — `src/transformers/loss/loss_deformable_detr.py`
- `LwDetrHungarianMatcher` — `src/transformers/loss/loss_lw_detr.py`

For context, RT-DETR was fixed in #47016, and DETR + Grounding DINO in #47048; these two matchers share the identical pattern.

### Reproduction

```python
import torch
from transformers.loss.loss_deformable_detr import DeformableDetrHungarianMatcher

matcher = DeformableDetrHungarianMatcher(class_cost=1, bbox_cost=5, giou_cost=2)

logits = torch.randn(1, 5, 4)
logits[0, 0, 0] = float("nan")  # e.g. produced by fp16 overflow under AMP
pred_boxes = torch.rand(1, 5, 4) * 0.5 + 0.25
targets = [{"class_labels": torch.arange(3), "boxes": torch.rand(3, 4) * 0.5 + 0.25}]

matcher({"logits": logits, "pred_boxes": pred_boxes}, targets)
# ValueError: cost matrix is infeasible  (scipy: "matrix contains invalid numeric entries")
```

The same happens for `LwDetrHungarianMatcher`.

### Expected behavior

The matcher should not crash on a non-finite cost matrix. Non-finite entries should be replaced with a large finite value before matching (as done for RT-DETR in #47016), so invalid entries are simply never selected:

```python
max_value = torch.finfo(cost_matrix.dtype).max
cost_matrix = torch.nan_to_num(cost_matrix, nan=max_value, posinf=max_value, neginf=max_value)
```

I have a fix ready (the same guard, with a regression test for each matcher) and will open a PR referencing this issue.


Comments:
Fix mllama cross-attention leak for fully masked-out text rows
Author: MutugiD
Pull request: yes
Labels: name: Code agent slop

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47103)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47103)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

Fixes #39379. In `MllamaCrossAttentionDecoderLayer.forward`, `full_text_row_masked_out_mask` was applied only to the MLP output, but not to the cross-attention output before its residual add. As a result, text rows that are fully masked out of cross-attention (tokens that precede their `
`) still received image information through the residual connection.

This matches the reference implementation, which applies the mask both after the attention and after the MLP. The added lines mirror the existing post-MLP masking:

```python
if full_text_row_masked_out_mask is not None:
    hidden_states = full_text_row_masked_out_mask[:, 0] * hidden_states  # type: ignore
hidden_states = residual + self.cross_attn_attn_gate.tanh() * hidden_states
```

As noted in the issue, the discrepancy only appears when the image is **not** the first token, which is why the existing tests (built around image-first prompts) didn't catch it.

## Tests

Adds `MllamaCrossAttentionMaskTest` (CPU-only): it drives a single `MllamaCrossAttentionDecoderLayer` with a fully masked-out text row and asserts that row is left unchanged by the layer (no cross-attention leakage), while a row that does attend to the image is modified. The gates are tanh-gated and zero-initialized, so the test opens them to make the residual contributions non-trivial. It fails on `main` (100% of the masked row's elements change) and passes with the fix.

```
pytest tests/models/mllama/test_modeling_mllama.py::MllamaCrossAttentionMaskTest
```

## Who can review?

@zucchini-nlp (thanks for the go-ahead on the issue) @ArthurZucker

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mllama

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47103)
**Latest run:** [28779603918](https://github.com/huggingface/transformers/actions/runs/28779603918)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->
docs: fix LED config max_decoder_position_embeddings default (16384 -> 1024)
Author: Kropiunig
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47106)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47106)
<!-- ci-dashboard-badge:end -->

The `max_decoder_position_embeddings` docstring documents `defaults to 16384`, but the actual default in `__init__` is `1024`. The `16384` value belongs to `max_encoder_position_embeddings`, matching the canonical LED architecture (e.g. allenai/led-base-16384: encoder 16384 / decoder 1024). This is a docstring-only fix to align the documented default with the code.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: led

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47106)
**Latest run:** [28793440471:2](https://github.com/huggingface/transformers/actions/runs/28793440471)
**Result:** `success` | **Jobs:** 2 | **Tests:** 24 | **Failures:** 0 | **Duration:** 2m 14s

<!-- ci-dashboard-recap:end -->

We're trying to cut down on these small docstring PRs right now because of the flood! We'll do an agent pass to fix them all at some point
Keep doc-builder pin comment dependabot-compatible
Author: mishig25
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47108)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47108)
<!-- ci-dashboard-badge:end -->

Follow-up to the doc-build workflow update: dependabot parses the comment after a SHA pin, so it must stay a plain ref name. See https://github.com/huggingface/peft/pull/3393#discussion_r3528292192

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47108)
**Latest run:** [28794814654](https://github.com/huggingface/transformers/actions/runs/28794814654)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47108). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Expected behavior of `compute_result` is hard to expect and inconsistent
Author: MilkClouds
Pull request: no

In trainer there exists a parameter `compute_result` given to `compute_metrics` when `batch_eval_metrics` is given to True.

https://github.com/huggingface/transformers/blob/1e0665a191f73f6b002209c3dfcda478baac6bac/src/transformers/trainer.py#L370-L375

I think there are several problems for `compute_result`,
1. User can't expect (1) what happen if `batch_eval_metrics` is given (2) what is given to `compute_result`  and when it change from True or False (3) what's HF's intention to implement `compute_metrics` with `compute_result`. since there are very few (only 3 line) instruction for this.
2. `compute_metrics` sometimes called with `compute_result` and sometimes not, EVEN WHEN `batch_eval_metrics` is present. See below lines. 

https://github.com/huggingface/transformers/blob/1e0665a191f73f6b002209c3dfcda478baac6bac/src/transformers/trainer.py#L4534-L4547

Creating this issue because I spend long time figuring out this.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

cc @SunMarc 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I think this issue should be reopened. The expected functionality of the `compute_result` argument is not clear: Does it require the compute_metrics function to be stateful and track all of the values of the metric of the current evaluation loop? And how is it supposed to know when a new evaluation loop has been triggered? 

A reference implementation would be most welcome -- and I suspect it may even be possible to provide a simple wrapper/mixin that requires users to only implement their custom metric logic.
[i18n-<languageCode>] Translating docs to <languageName>
Author: VIUK-XV
Pull request: no
Labels: name: WIP

Hi! I’m a native Japanese speaker and I’d like to help translate the Japanese documentation.

Can I work on docs/source/en/model_doc/distilbert.md?
I’ll add the Japanese version under docs/source/ja/model_doc/ and update docs/source/ja/_toctree.yml.

Comments:
cc @stevhliu 

hi, thanks for your interest. there's already an open issue (https://github.com/huggingface/transformers/issues/28301) for Japanese, so i'm going to close this one
Improve Japanese documentation wording
Author: VIUK-XV
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47078)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47078)
<!-- ci-dashboard-badge:end -->

This PR improves the wording of the Japanese documentation.

Changes:
- Made several Japanese sentences more natural
- Reduced overly literal translations
- Kept code blocks, links, and autodoc directives unchanged
- Preserved the original meaning of the English documentation

Related to #28301

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47078)
**Latest run:** [28722796219:2](https://github.com/huggingface/transformers/actions/runs/28722796219)
**Result:** `success` | **Jobs:** 2 | **Tests:** 15 | **Failures:** 0 | **Duration:** 53s

<!-- ci-dashboard-recap:end -->

cc @stevhliu
Add Japanese translation for DistilBERT docs
Author: VIUK-XV
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47077)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47077)
<!-- ci-dashboard-badge:end -->

This PR adds the Japanese translation for the DistilBERT documentation.

Changes:
- Added `docs/source/ja/model_doc/distilbert.md`
- Updated `docs/source/ja/_toctree.yml`

Related to #28301

cc @stevhliu @MKhalusova

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47077)
**Latest run:** [28722351066:2](https://github.com/huggingface/transformers/actions/runs/28722351066)
**Result:** `success` | **Jobs:** 2 | **Tests:** 9 | **Failures:** 0 | **Duration:** 37s

<!-- ci-dashboard-recap:end -->

cc @stevhliu
Fix CohereASR training loss double shift
Author: w-Jessamine
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46958)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46958)
<!-- ci-dashboard-badge:end -->

## Summary

Fixes #46894.
Supersedes #46947, which was closed after the fork branch was renamed.

CohereASR right-shifts `labels` into `decoder_input_ids`, then used `self.loss_function` (`ForCausalLMLoss`) to compute the training loss. That loss shifts labels internally again, so training targets were effectively `labels[..., 1:]`.

This switches CohereASR to a plain `CrossEntropyLoss`, matching the Moonshine fix and seq2seq models that already align decoder inputs from labels before computing loss.

## Tests

- `PYTHONPATH=src /tmp/transformers-pr-venv/bin/python -m pytest tests/models/cohere_asr/test_modeling_cohere_asr.py::CohereAsrModelTest::test_training_loss_no_double_shift -q`

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46958)
**Latest run:** [28370058252](https://github.com/huggingface/transformers/actions/runs/28370058252)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cohere_asr
[skills] model doc
Author: stevhliu
Pull request: yes

make model addition easier by asking your agent to complete the `model_doc/<model>.md` file

1. `transformers add-new-model-like` creates a template with `<PLACEHOLDER>` tokens
2. trigger skill with something like "finish the docs for this new model"
3. the /model-doc skill uses the existing /hf-papers, /hf-models, and /hf-collections skills to look up the model to add relevant links to the model on the Hub and summarize what's new and cool about the model
4. any remaining `<PLACEHOLDER>` tokens are manually completed

see #45612 for example outputs for Qwen3.5 and Perception Encoder

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45705). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Support loading glm4moe GGUF
Author: adonishong
Pull request: no
Labels: name: Good Second Issue

name: Feature request

### Feature request

Currently, GGUF versions of GLM-4.5 series MoE models raises "GGUF model with architecture glm4moe is not supported yet" error.

### Motivation

GLM-4.5 series MoE GGUF models will successfully run. That is a fantastic model ...

### Your contribution

nope ...

Comments:
Hi @adonishong ,
Could you please provide a code to reproduce this issue.

Try loading any glm4moe model, it fails.

```
model = AutoModelForCausalLM.from_pretrained(
    "unsloth/GLM-4.5-Air-GGUF",
    device_map="auto",
    gguf_file="GLM-4.5-Air-Q2_K.gguf"
)
```

Also while I have your attention, how do I load multipart GGUFs? The larger ones have multiple parts. Is it possible to load GGUF from disk instead?

I get the same error, running with the parameters specified by the [GLM 4.5 github page](https://github.com/zai-org/GLM-4.5) .
`vllm serve /mnt/llm_model_data/GLM-4.5-Air-UD-Q5_K_XL.gguf --tensor-parallel-size 4 --tool-call-parser glm45 --reasoning-parser glm45 --enable-auto-tool-choice --served-model-name glm-4.5-air`

cc @SunMarc 

We don't support GLM GGUF model in transformers yet. Feel free to add support to it ! 

Hi @SunMarc, Hi, I looked into adding GGUF support for the `glm4moe` architecture as suggested.

After investigating the codebase, it appears that the necessary base model implementation for `GLM4` (including `GLM4Config` and `GLM4ForCausalLM`) is not yet available in the library.

This makes adding GGUF support a much larger task that requires implementing the full model first. Given this prerequisite, the "Good Second Issue" label might need to be re-evaluated. Hope this information is helpful!


The `glm4_moe` architecture is now in the library `src/transformers/models/glm4_moe/`, so the missing piece is just the GGUF integration, not a full model port. I'd like to take this and add GGUF loading support for glm4moe. Will follow the existing arch mappings in integrations/ggml.py as a template and add a loading test.

I would love to work on this if it's still open!
[Model] Support `use_cache=False` for DeepSeek V4
Author: kylesayrs
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46965)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46965)
<!-- ci-dashboard-badge:end -->

## Purpose ##
* Support inferences with DSV4 without using kv cache

## Testing ##
Used these changes to calibrate and quantize https://huggingface.co/RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8

```python
import torch
from transformers import AutoConfig, AutoModelForCausalLM

def test_dsv4_cache_consistency():
    # Load the config from the pretrained model
    model_id = "inference-optimization/DeepSeek-V4-2.7B-tiny"
    config = AutoConfig.from_pretrained(model_id)

    # Set device
    device = "cuda" if torch.cuda.is_available() else "cpu"
    print(f"Using device: {device}")

    # Create model from config with random weights
    print("Creating model from config with random weights...")
    with torch.device(device):
        model = AutoModelForCausalLM.from_config(config, dtype=torch.bfloat16)
    model.eval()

    # Create random input
    batch_size = 2
    seq_length = 10
    vocab_size = config.vocab_size
    input_ids = torch.randint(0, vocab_size, (batch_size, seq_length)).to(device)

    print("Running inference with use_cache=True...")
    with torch.no_grad():
        outputs_with_cache = model(
            input_ids=input_ids,
            use_cache=True,
        )

    print("Running inference with use_cache=False...")
    with torch.no_grad():
        outputs_no_cache = model(
            input_ids=input_ids,
            use_cache=False,
        )

    assert not torch.all(outputs_with_cache.logits == 0)

    print("Comparing outputs...")
    # Compare the logits - they should be exactly the same
    torch.testing.assert_close(
        outputs_with_cache.logits,
        outputs_no_cache.logits,
        rtol=1e-3,
        atol=1e-3,
        msg="Outputs differ between use_cache=True and use_cache=False"
    )

    max_diff = torch.abs(outputs_with_cache.logits - outputs_no_cache.logits).max().item()
    print("✓ Test passed: Outputs are consistent between use_cache=True and use_cache=False")
    print(f"  Output shape: {outputs_with_cache.logits.shape}")
    print(f"  Max absolute difference: {max_diff:.2e}")
    print(f"  Mean of outputs: {outputs_with_cache.logits.mean().item():.2e}")


if __name__ == "__main__":
    test_dsv4_cache_consistency()
```

Comments:
cc @cyrilvallez for the cache maybe?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deepseek_v4

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46965)
**Latest run:** [28449366079:2](https://github.com/huggingface/transformers/actions/runs/28605584012)
**Result:** `success` | **Jobs:** 7 | **Tests:** 331 | **Failures:** 1 | **Duration:** 1m 14s

<!-- ci-dashboard-recap:end -->
(Draft) [CB] Change padding from interval to pow2
Author: remi-or
Pull request: yes

Draft until we have a full picture of the performance improvements.

<!-- 
# Summary

This PR makes some small changes to how CUDA graphs are bucketed in continuous batching. It used to be that they were bucketted based on intervals, now they are bucketed based on powers of 2. This means less buckets, less parameters in the config, and better performance.

Hinges on https://github.com/huggingface/transformers/pull/46490 to be merged.

## Performance

For Llama 8B

| label | main acc | with PR acc | main (tok/s) | with PR (tok/s) | diff |
|---|---|---|---|---|---|
| gsm8k_default | 0.822 | 0.825 | 2448.09 | 2431.56 | -0.7% |
| gsm8k_sampling | 0.796 | 0.781 | 1947.00 | 1946.53 | -0.0% |
| gsm8k_compile | 0.822 | 0.825 | 2454.80 | 2450.69 | -0.2% |
| gsm8k_no_fast_decode | 0.822 | 0.825 | 2349.59 | 2348.36 | -0.1% |
| gsm8k_bare_bones | 0.821 | 0.825 | 1862.95 | 1871.58 | +0.5% |
| ifeval_default | 0.447 | 0.457 | 8293.33 | 8940.09 | +7.8% |
| few_blocks | - | - | 691.24 | 682.46 | -1.3% |
| multi_return_seq | - | - | 1630.70 | 1630.93 | +0.0% |
| rollouts_1024 | - | - | 3285.90 | 3573.25 | +8.7% |
| rollouts_2048 | - | - | 3130.02 | 3413.55 | +9.1% |
| rollouts_4096 | - | - | 2789.79 | 2702.67 | -3.1% |
| rollouts_8192 | - | - | 2259.52 | 2393.42 | +5.9% |
| rollouts_16384 | - | - | 1397.03 | 1569.22 | +12.3% |

## Tests

TBD

## Review

Reviewed the PR using claude and compound engineering plugin.

-->

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46532). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46532)

This is outdated. Will re-open a new PR for new bucket style in the future.
[Fix] Make require_flash_attn3 match runtime
Author: remi-or
Pull request: yes

This PR changes the behavior of `require_flash_attn_3` so that it does not only check `is_flash_attn_3_available` (if the package if available) but also if the kernel can be loaded via `kernels`. 
Since using `kernels` is much more convenient than installing flash from source, it seems logical we enable users to run FA3-gated CI in these conditions.

Tagging @vasqu for review since it's related to our recent discussions around FA3 kernels integration

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46838)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46838). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Closing until FA3 tests are more stable