noamaanMulla-03/transformers-issues huggingface dataset:
Centralize multimodal token expansion via get_text_with_replacements
Author: ArthurZucker
Pull request: yes

Routes image/video/audio placeholder expansion through the shared `ProcessorMixin.get_text_with_replacements` helper (#45493) for ~25 processors that still rolled their own `.replace`/`re.sub`/sentinel logic. Each now defines `replace_image_token`/`replace_video_token`/`replace_audio_token` (or builds the replacement list inline where expansion is context-dependent). Output is byte-for-byte identical.

### Also
- **Perf:** `get_text_with_replacements` now skips building `batch_replacement_offsets` unless `return_text_replacement_offsets=True` (~40% faster on the text step; feature unchanged when requested).
- **Test:** new `ProcessorTesterMixin.test_processor_expands_tokens_via_get_text_with_replacements` enforces the contract; skip-listed for ColQwen2/ColModernVBert (text XOR images) and Mllama (one token/image + cross-attention).

### Notes
- Validated against each model's existing token-count tests; byte-equivalence harnesses where tests are gated (torchaudio / unreleased checkpoints). CI runs the rest.
- `gemma3n` deferred (its Hub template emits soft tokens → needs a soft-token-anchored migration, not boi/eoi).
- AI assistance was used; every migrated `replace_*_token` should be reviewed. Suggest squash-on-merge (history evolved in place).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46404). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cohere2_vision, colmodernvbert, colqwen2, csm, deepseek_ocr2, deepseek_vl, deepseek_vl_hybrid, emu3, ernie4_5_vl_moe, florence2, gemma3, glm_image, granite4_vision, granite_speech, higgs_audio_v2, internvl

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46404)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46404&sha=c2c78f

BTW, this PR can be overriden by https://github.com/huggingface/transformers/pull/46556. I am batch updating all VLMs and a few audio models

ty, closing then!
Allow tokenizers 0.23.1
Author: ArthurZucker
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46381)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46381)
<!-- ci-dashboard-badge:end -->

Closes #45736 — allow `tokenizers==0.23.1`.

Two 0.23.1 changes handled:
- `RobertaProcessing` args are now positional-only → fixed 5 call sites (convert_slow ×2, clip, layoutlmv3, roberta).
- Truncation early-exit (huggingface/tokenizers#1990) may drop the trailing overflow segment → test tolerates its absence; still fully checked on 0.22.x.

Tested: clip/roberta/layoutlmv3/bert/gpt2/t5 tokenizer suites pass on both 0.22.2 and 0.23.1. AI-assisted, human-reviewed.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46381). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46381&sha=6c8ef7

Oups doing this now 

> Oups doing this now

Did you push it to `master`? I just pulled the just-released 5.13.0 tag and it still complains about `tokenizers`.
Maybe I missed something.

Thanks a lot!

nope sorry


I was a bit busy with designing tokenizers v1

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bertweet, clip, herbert, layoutlmv2, layoutlmv3, layoutxlm, luke, markuplm, mluke, mpnet, pop2piano, roberta, roc_bert, tapas, udop

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46381)
**Latest run:** [29575704636:1](https://github.com/huggingface/transformers/actions/runs/29575704636)
**Result:** `success` | **Jobs:** 15 | **Tests:** 138,566 | **Failures:** 0 | **Duration:** 13h 34m

<!-- ci-dashboard-recap:end -->
Clarify device placement in pipelines
Author: LysandreJik
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47367)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47367)
<!-- ci-dashboard-badge:end -->

The docs were not aligned with the actual behavior.

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47367)
**Latest run:** [29500793415](https://github.com/huggingface/transformers/actions/runs/29500793415)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47367). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

yes, blunder
Olmo3 CI
Author: zucchini-nlp
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47175)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47175)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

as per title, long due, I broke it when fixing Olmo3 RoPE

Comments:
run-slow: olmo3

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/28953061525)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/olmo3"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47175). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/28953061525)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [c7b503d6](https://github.com/huggingface/transformers/commit/c7b503d60651867a80893a578570a22576efbe91) | workflow commit (merge commit) |
| PR | [9fd32f17](https://github.com/zucchini-nlp/transformers/commit/9fd32f17bc1a292e5d11d44fbb1315c303937b3b) | branch commit (from PR) |
| main | [d15c0361](https://github.com/huggingface/transformers/commit/d15c03619f0fefb48bc0569b815bb59964efef1f) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: olmo3

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47175)
**Latest run:** [28952916069:1](https://github.com/huggingface/transformers/actions/runs/29498174685)
**Result:** `success` | **Jobs:** 7 | **Tests:** 352 | **Failures:** 0 | **Duration:** 1m 24s

<!-- ci-dashboard-recap:end -->
LLaMA-MoE
Author: Spico197
Pull request: no
Labels: name: New model

### Model description

LLaMA-MoE is a series of token-choice based Mixture-of-Experts models on LLaMA2. It first partition LLaMA2's FFNs into multiple experts, then apply continual pre-training to recover its language abilities.

We believe LLaMA-MoE is a good start for MoE research under limited computing resources.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Our repo: https://github.com/pjlab-sys4nlp/llama-moe
HF models (currently set `trust_remote_code=True`): https://huggingface.co/llama-moe

Comments:
Hey @Spico197, apologies for the very late reply here, we're catching up on the new-model backlog.

The good news is that nothing is blocking you: as you noted in the issue, the LLaMA-MoE checkpoints already run in transformers today via `trust_remote_code=True`.

On a native port, we've decided not to schedule one, and I'm going to close this out. Two reasons:

- Model on the Hub first: our policy has shifted toward keeping new architectures on the Hub with remote code by default rather than merging them into the core library, which is where LLaMA-MoE already lives and works.
- Demand hasn't materialized. In the ~2.5 years this has been open it's picked up no comments or 👍, and the checkpoints see roughly 200 downloads/month across llama-moe/*. Every architecture we merge is something we maintain across refactors indefinitely, so we reserve that for models with sustained pull from users.

Thanks for the request
Add FlexiBERT
Author: shikhartuli
Pull request: no
Labels: name: New model

### Model description

FlexiBERT is a suite of *flexible* and *heterogeneous* models. The design space was proposed in this [paper](https://arxiv.org/abs/2205.11656) accepted for publication in the Journal of Artificial Intelligence Research.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The weights are available [here](https://huggingface.co/shikhartuli/flexibert-mini).

Comments:
The PR was closed by bot. Please reopen the PR and let me know if anything needs to be done to merge the PR. The PR would add the FlexiBERT suite of models to 🤗 Transformers.

Hey @shikhartuli, apologies for the very long silence here, we're catching up on a big backlog. Thanks for the paper and the earlier PR (#20356); FlexiBERT is neat work.

That said, we no longer add niche/research architectures directly into the core library; a full new architecture is an effectively permanent maintenance surface given our backward-compat bar, and the modern path is "Model on the Hub" with custom code loaded via `trust_remote_code=True`.
Your weights are already on the Hub at `shikhartuli/flexibert-mini`, but that repo currently ships only `config.json` + weights; if you add a `modeling_flexibert.py` and an `auto_map` to it, anyone can do `AutoModel.from_pretrained(..., trust_remote_code=True)` with no change to transformers.

Given that plus limited demand since 2022, I'm going to close this
[Kimi] add integration tests
Author: zucchini-nlp
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47383)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47383)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

Unblocks https://github.com/huggingface/transformers/pull/47096

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: kimi_k25

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47383)
**Latest run:** [29578875826](https://github.com/huggingface/transformers/actions/runs/29578875826)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47383). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: kimi_k25

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/29580279871)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/kimi_k25"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/29580279871)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [a5f61496](https://github.com/huggingface/transformers/commit/a5f61496f8e171b797cd484fcb1800d6f4bd4bfd) | workflow commit (merge commit) |
| PR | [a6327d06](https://github.com/zucchini-nlp/transformers/commit/a6327d06a03da6706d0c64d2daa2961955f691f8) | branch commit (from PR) |
| main | [13337f29](https://github.com/huggingface/transformers/commit/13337f2978a2016ea5e71ea6ae42201e760e927d) | base commit (on `main`) |

⚠️ **Model CI failed to report results**

The test failure analysis could not be completed. Please check the [workflow run](https://github.com/huggingface/transformers/actions/runs/29580279871) for details.
Fix important model failures on MI450
Author: Abdennacer-Badaoui
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47298)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47298)
<!-- ci-dashboard-badge:end -->

As per title.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47298). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: conditional_detr, deformable_detr, detr, gemma3, gemma3n, grounding_dino, internvl, llama, modernbert, qwen2, qwen2_5_omni, smolvlm, table_transformer, yolos

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47298)
**Latest run:** [29584804051:1](https://github.com/huggingface/transformers/actions/runs/29584804051)
**Result:** `failure` | **Jobs:** 2 | **Tests:** 22 | **Failures:** 1 | **Duration:** 3m 22s

> **Code quality check failed**: test jobs were skipped. Fix the code quality issues and push again to run tests.

<!-- ci-dashboard-recap:end -->
Fix `BlockMask` crash in CTRL flex-attention generation
Author: jiqing-feng
Pull request: yes

## Problem

Generating with `Salesforce/ctrl` under compiled flex-attention crashes:

```
AttributeError: 'BlockMask' object has no attribute 'ndim'
```

at `CTRLModel.forward` (`models/ctrl/modeling_ctrl.py`).

## Reproduction

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Salesforce/ctrl"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, attn_implementation="flex_attention", device_map="auto"
)
model.generation_config.cache_implementation = "static"  # compileable cache -> BlockMask path

inputs = tokenizer("Today the weather is", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=16)  # crashes before this fix
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Cause

`CTRLModel.forward` reshapes `attention_mask` with an unconditional `.ndim` check:

```python
if attention_mask is not None and attention_mask.ndim < 4:
    attention_mask = attention_mask.view(batch_size, -1)
```

In compiled flex-attention generation, `prepare_inputs_for_generation` pre-builds a
`BlockMask` (via `create_masks_for_generate`) and passes it as `attention_mask`.
`BlockMask` exposes `.shape` but not `.ndim`, so the check raises. The reshape is
only meant for 2D padding tensors; `create_causal_mask` already handles a 4D
`Tensor`/`BlockMask` downstream.

## Fix

Only reshape real tensors, letting `BlockMask` (and any 4D mask) pass through
unchanged. Mirrors the existing `doge` / `deepseek_v4` guard; `torch` is already
imported.

```python
if attention_mask is not None and isinstance(attention_mask, torch.Tensor) and attention_mask.ndim < 4:
    attention_mask = attention_mask.view(batch_size, -1)
```

## Validation

Benchmark suite on `Salesforce/ctrl` (causal-lm, level 1). Before: flex config
crashed (exit 1). After: all 4 configs pass (eager / FA2 / flex / FA2+CB), exit 0.

## Note

`models/gpt2/modeling_gpt2.py` has the identical latent bug; not touched here, can
be addressed in a follow-up.


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ctrl

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46854)

cc @arthurzucker since this is a text model attention bug I think!
wip step 3 7 flash
Author: itazap
Pull request: yes
Labels: name: New model

adding https://huggingface.co/stepfun-ai/Step-3.7-Flash

+ debug scripts to build mini model checkpoint (vibe-coded) and conversion

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=46658)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46658)
<!-- ci-dashboard-badge:end -->

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46658). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hmm isn't it same as https://github.com/huggingface/transformers/pull/46562 ? Not sure if this is somehow being discussed in internal slack channels though, and who is the contrib from the other PR (random user or StepFlash team)

thanks for flagging @zucchini ! I'll keep an eye on it as based on their modular file it doesn't seem very developed and im also using step_3_7_flash as a data point for modular_model_converter.py + a model adder skill 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, step_3_7_flash

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46658)
**Latest run:** [28519819027:1](https://github.com/huggingface/transformers/actions/runs/28587611782)
**Result:** `failure` | **Jobs:** 2 | **Tests:** 3 | **Failures:** 2 | **Duration:** 7s

> **Code quality check failed**: test jobs were skipped. Fix the code quality issues and push again to run tests.

<!-- ci-dashboard-recap:end -->

Coming with a detailed review next week :)

In the meantime, I am seeing unrelated files, configs, etc, lets delete them all and make it ready for merging
Use new `per_layer_config` for Gemma 4 so that heterogeneous attention config is explicit
Author: hmellor
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47384)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47384)
<!-- ci-dashboard-badge:end -->

Gemma 4 contains a different `head_dim` and `num_key_value_heads` for the full attention and sliding attention layers. Currently, which layers use which is implicit during model construction.

This PR:
- leverages the heterogeneous config support added in https://github.com/huggingface/transformers/pull/45333 to represent this at the config level
- deletes the old `*global*` keys and propagates the change to everywhere that used them

This will enable downstream libraries like vLLM to load Gemma 4 using the Transformers modelling backend more easily as this heterogeneity can be discovered via the config.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: diffusion_gemma, gemma4, gemma4_unified

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47384)
**Latest run:** [29589248735](https://github.com/huggingface/transformers/actions/runs/29589248735)
**Result:** `failure` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47384). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
ci: harden daily CI reporting against GitHub API rate limits
Author: tarekziade
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47382)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47382)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

The daily model CI report crashed in the "Send message to Slack" step with `KeyError: 'jobs'` (get_ci_error_statistics.get_jobs) and `KeyError: 'created_at'` (get_previous_daily_ci.get_last_daily_ci_run) whenever GitHub returned an error payload instead of run/job data.

The large matrices (Model/Torch-pipeline/Example/Quantization/Kernels) paginate over 1000+ jobs and exhaust the GitHub API rate limit, so their reports never post to Slack, making the run look like it never ran even though every test executed.

This harden this step with retry loops and a clear error in case it failed.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47382). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

slack step pushed again with this fix on a throw-away wf to verify https://github.com/huggingface/transformers/actions/runs/29575160394/job/87867850101

attempt 2 https://github.com/huggingface/transformers/actions/runs/29576459980/job/87871909474

(second commit will be removed before merging, it's only here to exercise the fix on yesterday's daily)

attempt 2 worked :) 

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47382)
**Latest run:** [29588877500:1](https://github.com/huggingface/transformers/actions/runs/29591646398)
**Result:** `success` | **Jobs:** 4 | **Tests:** 264 | **Failures:** 0 | **Duration:** 2m 15s

<!-- ci-dashboard-recap:end -->