noamaanMulla-03/transformers-issues huggingface dataset:
Streamable chat parsing
Author: Rocketknight1
Pull request: yes

Work in progress PR for streamable chat parsing. This introduces a new improved kind of response schema, which I'm calling `response_template`. Response templates are a lot easier to write, since they avoid most of the messy regexes of the older system. More importantly, though, using a region-based system means that model authors can write templates that enable streaming parsing, emitting fields as they're ready, rather than needing the whole message to be ready at once.

TODO:

- [x] Merge/cleanup content types
- [x] Can we merge `coerce` into content types?
- [x] Correct handling of tokens/strings inputs, batched lists, etc.
- [x] Final docs cleanup, ensure examples match API
- [x] Lots and lots of general testing
- [x] Handling of multiple tool calls - do we need to emit arrays differently?
- [x] Explicit option for "multiple literals" as a match without needing a regex, avoids regex buffer
- [x] Properly document the weird content args

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45847). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This should now be ready for review! It's a lot, but thankfully I think you can skim a lot of it. It's also a little ugly right now, because we need to maintain legacy support for the older `parse_response()` method, but once this is in we can begin deprecating that very quickly. That cleanup, and support for batched inputs, will come in a follow-up PR (batched handling will also be needlessly messy because of the legacy code right now, but it'll be much cleaner to add once it's removed)

The main things to review are:
- The doc
- The public-facing changes in `response_parser.py`, `processing_utils.py` and `tokenizer_utils.py`
- The test coverage

Most other files are agent-written boilerplate changes to support the new API (the pipeline + Gemma conversion script changes), or agent-written internal mechanics that are mostly dense and should be well-covered by tests (`content_parsers.py`, `response_templates.py`  and the second half of `response_parser.py`)

I think we've gotten all the reviews we're going to get! Pinging core maintainers @arthurzucker @cyrilvallez @vasqu if any of you wants a veto before I merge this :sweat_smile: 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma4

I think most of the review comments have now been addressed, going to merge! There are a few nits in docs or performance that we can move to follow-up work but nothing that will break BC, so I think merging is safe at this point!
[Offloading] [Bugfix] Fix disk offloading of models with explicit tensor dtypes
Author: kylesayrs
Pull request: yes

## Purpose ##
* Fix bug where DSV4 would load `tid2eid` with the incorrect precision, leading to incorrect hash gate values
  * Casting from int64 to bfloat16 was causing `tid2eid` to round to out-of-bounds values, not to mention that the values were incorrect

## Changes ##
* Instead of using `model.dtype` to determine the load dtype for disk-offloaded weights, instead use the dtype of the associated meta tensor in the meta model. The meta tensor dtype is already the correct dtype, influenced by the `local_torch_dtype` load context
  * This `dtype` field is read by `OffloadedWeightsLoader` [here](https://github.com/huggingface/accelerate/blob/2f87576d329d12128c819b3f9e681fbe8c5d21de/src/accelerate/utils/offload.py#L177-L178)

## Testing ##
* Added `test_disk_onload_dtype`
```
RUN_SLOW=1 python3 -m pytest tests/utils/test_modeling_utils.py::ModelUtilsTest::test_disk_onload_dtype -sx
```

## Suggested Reviewers ##
@SunMarc @zucchini-nlp

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46849)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46849). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
add distributed config
Author: 3outeille
Pull request: yes


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46705). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46705&sha=799ac9

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46705)
[docs] require trust_remote_code for custom_generate
Author: stevhliu
Pull request: yes

adds docs for #46483 (fix outdated docstring and add warning)

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46677)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46677). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Rocketknight1, just a gentle reminder thanks!
[docs] distributed training
Author: stevhliu
Pull request: yes

- removes "Number of accelerators" section from "Accelerator selection" guide since this is probably pretty commonly known
- add a new "DDP" guide
- refactored "Accelerate" guide with a more focused overview of what it is and how to configure it (only covers `Trainer` now instead of raw PyTorch loop). this guide links out to more detailed FSDP/DDP guides for full config-specific settings. also documents some `AcceleratorConfig` settings
- refactored "FSDP" guide with cleaner table and `auto_wrap` explanation and detailed explanation of what the config settings are

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_44420). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

all the args should be updated to fsdpv2 now, lmk if there's anything else @SunMarc :)
Add Spanish translation of the torch.compile page
Author: delcenjo
Pull request: yes

This PR adds the Spanish (es) translation of the `torch.compile` page and lists it under the "Rendimiento y escalabilidad" section of the Spanish table of contents.

I am a native Spanish speaker. I kept the license header, the code block, the table and all links unchanged, translating only the prose.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46852)

cc @stevhliu

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46852). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Prevent auto-class from being modified for all models
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

as per title, when we re-assigned `cls` for current model, we didn't assign back the correct `cls` and all other models in MAPPING were using a wrong auto-class

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cosmos3_omni, ernie4_5_vl_moe, glm46v, glm4v, glm4v_moe, glm_image, glm_ocr, paddleocr_vl, qwen2_5_vl, qwen2_vl, qwen3_5, qwen3_5_moe, qwen3_vl, qwen3_vl_moe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46844). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46844)
Fix RT-DETR extra feature-level projection indexing (#46832)
Author: Aravind-11
Pull request: yes

## Summary

Fixes a latent crash in RT-DETR when `num_feature_levels` is set higher than the number of backbone output levels (e.g. 4 vs default 3).

### Bug (#46832)

In `RTDetrModel.forward`, the first extra decoder projection incorrectly indexed the **output** of `decoder_input_proj` with `[-1]` instead of selecting the last backbone feature map **before** the projection:

```python
# before (TypeError: conv2d received a list of tensors)
sources.append(self.decoder_input_proj[_len_sources](encoder_outputs.last_hidden_state)[-1])

# after (matches the correct pattern used in the next loop iteration)
sources.append(self.decoder_input_proj[_len_sources](encoder_outputs.last_hidden_state[-1]))
```

Default configs (`num_feature_levels` equal to backbone depth) are unaffected.

### Secondary fix (#46424)

Documented `generation_config.cache_config["max_cache_len"]` is now honored on the static-cache path (in addition to top-level `generation_config.max_cache_len`), so users can pin `StaticCache` size via either API and avoid realloc/`torch.compile` recompiles across generate calls.

## Test plan

- [x] Code review: extra-level branch now mirrors the already-correct `for i in range(...)` loop body
- [ ] Run RT-DETR repro from #46832:
  ```python
  import torch
  from transformers import RTDetrConfig, RTDetrForObjectDetection

  config = RTDetrConfig(num_feature_levels=4, num_labels=80, image_size=640)
  model = RTDetrForObjectDetection(config).eval()
  with torch.no_grad():
      outputs = model(pixel_values=torch.rand(1, 3, 640, 640))  # should not raise
  ```
- [ ] Confirm default `RTDetrConfig()` forward still works
- [ ] Optional: verify `generate(..., cache_implementation="static", cache_config={"max_cache_len": 2048})` sizes static cache to >= 2048

Fixes #46832
Related #46424

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: rt_detr

agent slop.
Implement a decode method in transformers.BasicTokenizer
Author: jiangwangyi
Pull request: no
Labels: name: Feature request

### Feature request

Transformers has provided a nice BasicTokenizer for basic tokenizing when we don't need BPE tokenizers. For data processing (like data format converting), it is better to offer a decode method for basic use.

### Motivation

When doing data format converting in some data processing problems, we usually meet the requirement to recover a list of tokens into continuous, readable text.

### Your contribution

None.

Comments:
@jiangwy99 The BasicTokenizer class will perform simple string processing e.g. splitting on white spaces. However it doesn't encode the tokens to ids, and so doesn't have a corresponding `decode` method. 

cc @ArthurZucker 

@amyeroberts BasicTokenizer has implemented a tokenize function, which converts text into a list of tokens. What I want is a de-tokenize, which converts a list of tokenized tokens into the original text, serving as a dual operation of BasicTokenizer.tokenize

@jiangwy99  The BasicTokenizer is just that, a very simple class used for doing basic preprocessing of strings: puncutation splitting, making lower case etc. Adding this functionality is beyond the scope of the current class. 

If it's something you're still interested in, you're welcome to make a fork of the repo with an implementation and share it here. The forum is [a good place](https://discuss.huggingface.co/) to discuss questions about implementation details.  Note: it won't be possible to always exactly recover the original input if `do_lower_case=True`, and special handling of spacing and capitalization with punctuation would be required.
Rework some old cache getters/properties
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Some of those old APIs for max sequence length and batch size were quite misleading and badly named. `get_max_cache_shape` and `max_cache_len` were more or less doing the same, but with different names -> `get_max_length` is a better API IMO, as it relates to the similar `get_seq_length` for max/current length respectively. `max_batch_size` is very misleading as well IMO, as the Cache will be changed anyway anytime the batch sizes changes. E.g. a StaticCache with batch size 8 will be changed to a StaticCache of batch size 2 if we use a new sequence of batch 8 -> 2, instead of using a larger cache that would be sliced (it will require re-compilation, but will save a lot of compute, so I still think this is overall the best approach)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bloom, dia, diffusion_gemma, voxtral_realtime

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46862). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46862)
Use meshgrid for brevity
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Realized we could let torch handle the `arange(N).repeat(x, y)` even for 3D inputs, accidentally came out in a discussion with Pablo 😄 

IMO this is more readable and less error-prone since we don't have to choose between `repeat_interleave(a * b)/repeat(a * y)/etc`

Comments:
run-slow: cosmos3_omni, ernie4_5_vl_moe, glm46v, glm4v, glm4v_moe, glm_image, glm_ocr, paddleocr_vl, qwen2_5_vl, qwen2_vl, qwen3_5, qwen3_5_moe, qwen3_vl, qwen3_vl_moe

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/28084539490)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/cosmos3_omni", "models/ernie4_5_vl_moe", "models/glm46v", "models/glm4v", "models/glm4v_moe", "models/glm_image", "models/glm_ocr", "models/paddleocr_vl", "models/qwen2_5_vl", "models/qwen2_vl", "models/qwen3_5", "models/qwen3_5_moe", "models/qwen3_vl", "models/qwen3_vl_moe"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46861). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cosmos3_omni, ernie4_5_vl_moe, glm46v, glm4v, glm4v_moe, glm_image, glm_ocr, paddleocr_vl, qwen2_5_vl, qwen2_vl, qwen3_5, qwen3_5_moe, qwen3_vl, qwen3_vl_moe

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46861)

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/28084539490)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [5985a80b](https://github.com/huggingface/transformers/commit/5985a80b3cf82b28087b402ad00dcc9b44c25526) | workflow commit (merge commit) |
| PR | [b18877c1](https://github.com/zucchini-nlp/transformers/commit/b18877c114ed0c6652cb12a4b14dba10ebb9dfa7) | branch commit (from PR) |
| main | [d87f670d](https://github.com/huggingface/transformers/commit/d87f670d5b23860b5937473a5fddbba9d675780a) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !


Not really, I suppose `torch.meshgrid` is doing some kind of optim internally tho I wasn't expecting it. Let me run check it as well in a while

Ha, indeed it is up to ~35% faster. I did a small bench with batch-size in (20, 40, 100) and random grids, offsets, merge sizes 😄 

beautiful!
blip 2 fix
Author: itazap
Pull request: yes

regression here! https://huggingface.co/datasets/hf-internal-testing/transformers_daily_ci/raw/59d114bc1c0d38d963fb5c66d32a267762a0e78a/2026-06-20/ci_results_run_models_gpu/new_failures_with_bad_commit_grouped_by_authors.json

instead of mapping all bliip-2 we should only map specific checkpoints

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46816). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46816)
nemotron_asr_streaming: set _supports_flex_attn to False
Author: kaixuanliu
Pull request: yes

It returns error AttributeError: 'BlockMask' object has no attribute 'dtype' when running test case `tests/models/nemotron_asr_streaming/test_modeling_nemotron_asr_streaming.py::NemotronAsrStreamingEncoderModelTest::test_flex_attention_with_grads`, it seems to be a similar case like https://github.com/huggingface/transformers/pull/42802: `_supports_flex_attn` should be set to False. Pls help review, thx! @vasqu 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: nemotron_asr_streaming

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46878)

Merging thank you :saluting_face: 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46878). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[RT-DETR] Fix misplaced [-1] index in decoder projection when num_feature_levels > backbone outputs
Author: saitejabandaru-in
Pull request: yes
Labels: name: Code agent slop

## What does this PR do?

Fixes a misplaced `[-1]` index in `RTDetrModel.forward` that causes a `TypeError` when `num_feature_levels` exceeds the number of backbone output feature maps.

Fixes #46832

## Root Cause

In the decoder input projection block, when `num_feature_levels > len(sources)`, the first extra feature level is constructed with:

```python
# BUG: [-1] indexes into the Conv2d output tensor, not the feature map list
sources.append(self.decoder_input_proj[_len_sources](encoder_outputs.last_hidden_state)[-1])
```

`encoder_outputs.last_hidden_state` is a **list** of feature-map tensors. The `[-1]` should select the last tensor from the list **before** passing it to the Conv2d projection — not **after**. The correct pattern already exists on the very next line:

```python
# CORRECT: [-1] selects from the list, then passes to Conv2d
sources.append(self.decoder_input_proj[i](encoder_outputs.last_hidden_state[-1]))
```

## Fix

Move the `[-1]` index inside the parentheses so it selects from the list before projection:

```diff
- sources.append(self.decoder_input_proj[_len_sources](encoder_outputs.last_hidden_state)[-1])
+ sources.append(self.decoder_input_proj[_len_sources](encoder_outputs.last_hidden_state[-1]))
```

Applied to both `modular_rt_detr.py` (source of truth) and `modeling_rt_detr.py` (generated).

## Reproduction

```python
import torch
from transformers import RTDetrConfig, RTDetrForObjectDetection

config = RTDetrConfig(num_feature_levels=4)  # > 3 backbone outputs
model = RTDetrForObjectDetection(config)
dummy = torch.randn(1, 3, 640, 640)

# Before fix: TypeError
# After fix: runs correctly
outputs = model(dummy)
```


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: rt_detr

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46874)
fix(mps): remove float64 casts in MusicFlamingo and UDOP that break Apple Silicon MPS
Author: Sriniketh24
Pull request: yes
Labels: name: Code agent slop

## What does this PR do?

Fixes #46723.

`torch.float64` is not supported on Apple Silicon MPS devices — any `.to(torch.float64)` call on an MPS tensor raises:
```
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
```

Two functions unconditionally upcasted to float64 before performing computation, causing all MusicFlamingo and UDOP inference to fail on MPS:

### `apply_rotary_time_emb` (MusicFlamingo)

```python
# Before — broken on MPS
hidden_states = hidden_states.to(torch.float64)
...
return torch.cat((rotated, passthrough), dim=-1).to(original_dtype)
```

The result was immediately cast back to `original_dtype`, so the float64 upcast gave no precision benefit for float32/bfloat16 models. Removed it entirely.  `modeling_musicflamingo.py` regenerated from the modular file with `utils/check_modular_conversion.py --fix`.

### `combine_image_text_embeddings` (UDOP)

```python
# Before — broken on MPS
bbox = bbox.to(torch.float64)
target_seg = (bbox.mean(-1) == 0.0) | (bbox.mean(-1) == 1.0)
```

The float64 was only needed for two exact comparisons against 0.0 and 1.0. Float32 is perfectly sufficient for these comparisons (bboxes that are exactly 0 or 1 are represented exactly in float32). Replaced with `.float()`.

## Tests

Added a unit test for each fix:
- `MusicFlamingoForConditionalGenerationModelTest::test_apply_rotary_time_emb_preserves_dtype` — verifies output dtype matches float32 input (no float64 upcast)
- `UdopModelTest::test_combine_image_text_embeddings_no_float64` — verifies the function runs without float64 and produces finite results

```
pytest tests/models/musicflamingo/test_modeling_musicflamingo.py::MusicFlamingoForConditionalGenerationModelTest::test_apply_rotary_time_emb_preserves_dtype tests/models/udop/test_modeling_udop.py::UdopModelTest::test_combine_image_text_embeddings_no_float64 -v
# 2 passed
```

## Coordination

- Addresses issue #46723 (opened by the community; no active PR existed — a commenter said "I'll be sending a PR momentarily" 10+ days ago but no PR appeared)
- Checked for duplicate PRs with `gh pr list --repo huggingface/transformers --state open --search "46723 in:body"` — none found
- AI assistance was used to draft this PR; changes reviewed and tested by the human submitter

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: musicflamingo, udop

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46879)
fix: redirect auto_docstring diagnostics from stdout to stderr (#46877)
Author: C1-BA-B1-F3
Pull request: yes
Labels: name: Code agent slop

Fixes #46877

## Problem
 uses  to emit diagnostic messages (missing types, undocumented parameters, missing checkpoints, etc.) directly to stdout. This pollutes stdout during normal module import — e.g.  produces  lines on stdout.

## Fix
Replace all 7  calls in  with  using Python's standard  module. This routes diagnostics through the logging system where they can be filtered, redirected, or suppressed as needed, instead of unconditionally printing to stdout.

Changes:
- Add  and 
- Replace  →  with -style formatting (7 call sites)
- Remove literal  prefixes (the log level conveys severity)

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46884)

> ⚠️ **Code quality check failed** — all test jobs were skipped. Fix the code quality issues and push again to run tests.