noamaanMulla-03/transformers-issues huggingface dataset:
Optimize MoEs for decoding using batched_mm
Author: IlyasMoutawwakil
Pull request: yes

# What does this PR do?

batched_mm is more performant on "smaller inputs", especially when used with torch.compile.
smaller inputs: up to 2^6-2^9 tokens depending on the num experts and num topk

<img width="2000" height="2000" alt="crossover_plots" src="https://github.com/user-attachments/assets/21a9f333-b3d5-40ae-90a6-a060da98eb3b" />
<img width="1168" height="243" alt="generation benchmark" src="https://github.com/user-attachments/assets/54daeb58-e7dc-45ca-a29f-351f12c8b13d" />



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43126). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deepseek_v2, ernie4_5_moe, flex_olmo, qwen2_moe
[`fix:`]: initialize `ModernBertUnpaddedRotaryEmbedding` when loading from a checkpoint
Author: emapco
Pull request: yes

# What does this PR do?

 This fix improves accuracy and prevents existing ModernBert models from yielding NaN tensors. Previously, when a pretrained  modernbert model is loaded, the `ModernBertUnpaddedRotaryEmbedding`'s `inv_freq` non-persistent buffer is not initialized resulting in `NaN` loss and logits.

Without fix: 
- `ModernBertForSequenceClassification`  `ModernBertModelIntegrationTest.test_inference_sequence_classification_flash_attention_2` rarely returns NaN tensors but instead fails due to inaccurate logits
- `ModernBertModelIntegrationTest.test_inference_sequence_classification_flash_attention_2_modernbert_base` always fails due to NaN logits.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @Cyrilvallez

Comments:
I should add that I used the `flash-attn` package that I installed via `uv pip install flash-attn --no-build-isolation` with `torch==2.9` and `cuda12.8`.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: modernbert

Fixed in https://github.com/huggingface/transformers/pull/43115!
Use timm-side buffers initialization
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Clean-up timm buffers initialization that was temporarily added in https://github.com/huggingface/transformers/pull/42941. It has now been upstreamed directly in `timm`, so let's simplify on the Transformers side!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43124). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Cyrilvallez shouldn't this use `init_non_persistent_buffers()` instead of `_init_buffers()`?

A related FYI, there was a missed import error (in a deprecated path) due to a moved module location, still have to support those as lots of code uses it so will be doing a 1.0.24 release very shortly to fix it. 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3n, timm_backbone, timm_wrapper

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43124&sha=cbd492
Fixed failing `BioGPT` batch generation test
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
Fixes this failing [BioGPT-batch generation test](https://github.com/huggingface/transformers/actions/runs/20706072009/job/59437072400#step:14:1607). From this PR [Generation Config defaults are now None](https://github.com/huggingface/transformers/pull/42702/)

<img width="1321" height="171" alt="image" src="https://github.com/user-attachments/assets/ccb8f737-b33b-4c47-91e3-0f756ad013ee" />

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@Rocketknight1 @zucchini-nlp 

Comments:
run-slow: biogpt

This comment contains `run-slow`, running the specified jobs: 

models: ["models/biogpt"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20755255109)

✅ No failing test specific to this PR 🎉 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: biogpt

Thx, merging now 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43128). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix failing `DebertaV2ModelIntegrationTest`
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
Fixes this failing [DebertaV2ModelIntegrationTest](https://github.com/huggingface/transformers/actions/runs/20706072009/job/59437072475#step:14:1608).

<img width="1327" height="147" alt="image" src="https://github.com/user-attachments/assets/6cef8469-03ae-4e46-b533-ec94627f4dd9" />

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@Rocketknight1 @zucchini-nlp

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deberta_v2

run-slow: deberta_v2

This comment contains `run-slow`, running the specified jobs: 

models: ["models/deberta_v2"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20782862121)

✅ No failing test specific to this PR 🎉 !


Thx again, just did the sanity check with run-slow - merging :hugs: 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43136). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
use `TokenizersBackend`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes a bunch of issues (including https://github.com/huggingface/transformers/pull/42874)

With this PR, we now kinda enforce that **if there is a specific `python` path to be used** (meaning there is a `XXXXTokenizer` class that has python code whatever the backend) the tokenizer's saved `tokenizer_class` needs to match the class mapped in `tokenization_auto` to the `model_type`. 

- If there is no `config.json` we are just gonna use the serialized `tokenizer_class` but in most cases its gonna produce gibberish outputs because many many models on the hub use say `LlamaTokenizer` when the tokenizer they actually need is completely different.
- If there is a `config.json`, we extract the `model_type`, and check if `TOKENIZER_MAPPING[model_type]` matches the serialized `tokenizer_class`. 
    - if NO: we just use `TokenizersBackend`: we assume its not intended + its not a special python path + its a recent model -> we use `TokenizersBackend`. Falling back upon failure to convert to the `tokenizer_class`.
    - if YES: its safe to say this is intended, we have a good match / map we can enforce the `tokenizer` architecture.
- If it is remote code, `trust_remote_code=True` we always use the class people want


We cannot rely on the `tokenizer_class`, and we should not rely on it, but we rely on the model type. 

Also 90% of the tokenizers on the hub don't need special code, and are just supported OOB by `tokenizers` that the motivations behind this decision. 

We cannot fix ALL `tokenizers_config.json` on the hub + we actually don't want to. We want to just read the `tokenizer.json` in general using `TokenizersBackend`. 

## What matters, breaking changes 🔴 🔴 🔴 🔴 🔴 🔴 
1. `TokenizersBackend` no longer default to having `token_type_ids`. If you want them, set the flag.

2. 
Given:
```python 
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B")
tok.push_to_hub("ArthurZ/MyTokenizer") 
```
This
```python
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B")
assert tok.__class__ == Qwen2Tokenizer
```

still works, because if there is no `config.json` we do fallback to the serialized `tokenizer_class` in the `tokenizer_config.json`. But this class, for MOST of the models out there is WRONG! 😉 




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42894). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aria, auto, blenderbot, canine, chameleon, chinese_clip, code_llama, deepseek_vl, deepseek_vl_hybrid, ernie4_5_vl_moe, granite_speech, layoutlmv2, nougat, parakeet, pixtral

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42894&sha=0354e3
Update Japanese README to match English version
Author: lilin-1
Pull request: yes

### PR Title:
Docs: Update Japanese README to match main English version

### PR Content:
# What does this PR do?
This PR updates the Japanese version of the README ( README_ja.md ) to align with the latest changes in the main English README.md .

The Japanese documentation was significantly outdated compared to the English version. This update ensures that Japanese-speaking users have access to the most current information regarding:

- Installation : Added uv as a recommended installation method.
- Quickstart : Updated code snippets with modern models (Qwen2.5, Whisper v3, Llama 3.2).
- Example Models : Included newly added models across all modalities (ModernBERT, DepthPro, Moshi, etc.).
- New Sections : Added the "100 projects using Transformers" section.
- Structure : Reorganized the document to mirror the English version's flow.
No functional code changes are included; this is purely a documentation update for internationalization.

## Before submitting
- This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- Did you read the https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request , Pull Request section?
- Was this discussed/approved via a Github issue or the https://discuss.huggingface.co/ ? Please add a link to it if that's the case. (N/A)
- Did you make sure to update the documentation with your changes?
- Did you write any new necessary tests? (N/A for docs)
## Who can review?
Documentation: @stevhliu
Others: @ArthurZucker (for general maintenance)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43069). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[docs] Deploying
Author: stevhliu
Pull request: yes

This PR focuses on several improvements to the serving and deploying docs:

- consolidates integration guides and ExecuTorch/ONNX export guides to reduce navigation overhead and group related content together
- improve scope of the `serving.md` doc to only the `transformers serve` CLI. clearer intro on when to use it vs optimized inference engines, more complete endpoint descriptions (`v1/audio/transcriptions` and `v1/models`), MCP example with `GenerationConfig`, port forwarding, and better structure for optimizations

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42263). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I think the "serving" docs will tend to grow so keeping it in its section would be good imo; would it be possible to split this PR into two so I can review the serving docs independently and we can iterate on them? The deploying docs look good so we can split it in two to accelerate the merge

sounds good, i've split off the `transformers serve` docs in a separate PR!

this PR only focuses on consolidating executorch/onnx into a "exporting to production" doc in the models section which i think is a more natural fit

i've also reverted changes to the transformers as a modeling backend doc. i'll build that one out separately into its own community integrations section as discussed on slack 🙂
Fix nativ tok
Author: itazap
Pull request: yes

fixes https://github.com/huggingface/transformers/issues/42796

Fixes incorrect tokenization for Ministral-3 models by preserving and restoring the ByteLevel decoder from tokenizer.json when loading tokenizers. Also custom `init` methods were overwriting the ByteLevel decoder with a Metaspace decoder.

Happens because these models reference a `LlamaTokenizerFast` in their `tokenizer_config.json` on the hub. and LlamaTokenizer's init overrides tokenizer components


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42874). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: apertus, auto, bart, bigbird_pegasus, blenderbot, blenderbot_small, blip, blt, bridgetower, chameleon, clipseg, decision_transformer, dia, evolla, flaubert, gemma3n

Thanks @itazap, when do you think this could be merged?

Hello! Sorry actually this requires https://github.com/huggingface/transformers/pull/42894 as the fix! will be merged in the next couple days
Add chatterbox support
Author: manmay-nakhashi
Pull request: yes
Labels: name: New model

name: Audio


Comments:
This PR is very overwhelming, like a lot of code agent PRs. It's not clear to me how novel some of these architectures are, or whether we really need three whole new architectures in the codebase! :sweat_smile:

Ideally we could cut down the PR size a lot by using `modular` files and importing, and maybe treating some of the submodels as components or reusing existing architectures for those?

cc @ebezzam @eustlb since it's an audio model/pipeline

@Rocketknight1 t3 model is very similar to llama model but because this is a whole Text-to-Speech pipeline we have tokenizer(s3tokenizer different pr which compresses the audio to 25 tok/ sec we need this for conditioning), encoder(T3- mod-llama model (main changes are speech tokens and conditioning )), decoder(s3gen cfm based model from cozyvoice2) and hifinet(from cozyvoice2) which converts mel to wav. 
i checked we don't have this models in the hf currently

@eustlb @Rocketknight1 What is the best way to add BasicTransformerBlock in s3gen ?, i am currently importing from diffusers but it's not there in the transformers , can i add that as a dependency ?

@manmay-nakhashi thanks for your contributions 🤗 let's avoid importing from `diffusers`, especially since Transformer blocks are already implemented within `transformers`

I recommend taking a look at [CONTRIBUTING.md](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#do-you-want-to-implement-a-new-model), and in particular [this guide](https://huggingface.co/docs/transformers/en/modular_transformers) on adding new models with scripts called `modular_MODELNAME.py`.

It is very likely your models contain components (like Transformer blocks) that already exist in the library!

@eustlb 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, chatterbox, s3gen, s3tokenizer
[style] Rework ruff rules and update all files
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Time to update type annotations mostly!

Removed the `C` group that only matches C901 that was removed as well, added C4 for all unnecessary comprehensions.

Removed most exceptions we had, that should really not be excluded, especially:
- F402 -> Checks for import bindings that are shadowed by loop variables.
- UP004 -> Checks for classes that inherit from object.
- UP028 -> Checks for for loops that can be replaced with yield from expressions.
- UP045 -> Check for typing.Optional annotations that can be rewritten based on PEP 604 syntax.
- UP007 -> Check for type annotations that can be rewritten based on PEP 604 syntax.
- F823 -> Checks for undefined local variables.
- SIM300 -> Checks for conditions that position a constant on the left-hand side of the comparison operator, rather than the right-hand side.
- SIM212 ->  Checks for if expressions that check against a negated condition.
- UP009 -> Checks for unnecessary UTF-8 encoding declarations.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43144). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam3_video

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43144&sha=d27287
Remove duplicate TODO comment in audio_utils.py
Author: raimbekovm
Pull request: yes

## Summary
  Removes duplicate TODO comment that appeared twice in `audio_utils.py` (lines 51 and 169).

  ## Changes
  - Removed the duplicate from the `load_audio_as` function
  - Kept the TODO at the import statement (line 51) as it's more contextually appropriate

  This is a minor cleanup that reduces code duplication without changing functionality.


Comments:
Make projection LayerNorm configurable in LFM2-VL
Author: ankke
Pull request: yes

Make projection LayerNorm configurable in LFM2-VL models.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: lfm2_vl

Hey @ankke we usually never do update a model that was already shipped, v2 means a new modeling / modular file for us as the weights are differennt! We are most probably gonna revert this but happy to add the new model when it is out!
[CB] Minor fix in kwargs
Author: remi-or
Pull request: yes

This PR fixes an error in the `fork` method of the continuous batching API. The arguments name were lacking a `_`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43147). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> The tests run now but at least locally they seem at least flaky to me (on our dgx a100). On which GPU have you checked?

Checked on H100, what tests were flaky? 

> what tests were flaky

Sorry switched torch versions, torch 2.6 passed case 0 but not case 1 and torch 2.9 they both failed locally (`test_num_return_sequences`). I suspect a GPU diff here so our CI (A10) still might fail here then

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43147&sha=abf4b4
Fix typos: Remove duplicate duplicate words words
Author: efeecllk
Pull request: yes

- Fixed spelling: "wether" → "whether", "paramaters" → "parameters"
- Fixed grammar: "will allows" → "will allow"
- Removed duplicate words: "a a", "of of", "to to", "is is", "for for", "with with"

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: clap, dia, hunyuan_v1_moe, omdet_turbo, qwen2_audio, switch_transformers, xcodec

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43040). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

you're welcome :)