noamaanMulla-03/transformers-issues huggingface dataset:
Fix missing initialization of `FastSpeech2Conformer`
Author: bvantuan
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes missing initialization of `FastSpeech2Conformer`. For some reason, the CI pipeline in #39239 did not run the test `test_can_init_all_missing_weights` for `FastSpeech2Conformer`.

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@Cyrilvallez @ydshieh 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface, @SunMarc and @qgallouedec
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: fastspeech2_conformer

All right, thanks a lot @bvantuan! I just commited on top to reactivate the tests (they should not be skipped, which is the reason why this issue was not discovered before!).
Going to merge this one when tests are green!
[processors] add tests for helper fn
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title,  so no-one deleted it accidentally 😄 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39629). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The CI should turn green now, @qubvel can you re-review? I added tests per model file and fixed a few things in processors

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aria, aya_vision, chameleon, colpali, colqwen2, emu3, fuyu, gemma3, glm4v, got_ocr2, idefics3, internvl, llava_next, llava_next_video, paligemma, qwen2_vl
Add multi-candidate & tree search for assisted decoding (speculative decoding)
Author: transcend-0
Pull request: no
Labels: name: Feature request

### Feature request

Extend `GenerationMixin`'s `_assisted_decoding` to support multiple candidates and tree search.
Specifically, sample multiple draft tokens per decoding step, thereby improving the acceptance rate and speedup.


### Motivation

Current assisted decoding in `transformers` employs single-candidate drafting, which limits the acceptance rate.
Multi-candidate methods with tree search have shown notable performance in many studies [1-5].

[1] Yang et al. Multi-candidate speculative decoding. *arXiv preprint arXiv:2401.06706*.
[2] Cai et al. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads. In *ICML 2024*.
[3] Li et al. EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees. In *EMNLP 2024*.
[4] Hu et al. Towards Optimal Multi-draft Speculative Decoding. In *ICLR 2025*.
[5] Xia et al. Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding. In *ACL 2024*.

### Your contribution

To add this feature, we could modify the existing `_assisted_decoding` in `generation/utils.py`, or introduce a new decoding strategy (e.g., `_multi_candidates_assisted_decoding`).
Would the maintainers be open to such an addition? I’m happy to contribute to the implementation and documentation.
@zucchini-nlp  @gante 

Comments:
Hey @transcend-0 !

Sure, we could definitely add it as a new decoding method. Recently, we've enabled loading and using custom decoding methods from the hub (similar to how custom code for models work). It will allow the community to push their generation methods directly to the hub and make it available fir users, without going through a long review process. You can find the documentation [here](https://huggingface.co/docs/transformers/generation_strategies#custom-decoding-methods)

I think we should add tree-based speculative decoding on the hub first
[configuration] remove redundant `classmethod`
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, another tiny clean up that deletes a `from_text_vision_configs` for initializing configs from its sub-configs. That can perfectly be done by `cls.__init__` and this no need to maintain a separate method for that

I added a warning with a deprecation cycle of 7 minor releases, to give users time to adjust in case anyone was using it

Comments:
cc @Rocketknight1 if you have time to review

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38812). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aimv2, align, altclip, blip, bridgetower, chinese_clip, clap, clip, clipseg, efficientloftr, groupvit, owlv2, owlvit, pix2struct, siglip, siglip2
CRITICAL ISSUE REPORT! GEMMA 3 1B CANNOT RUN!
Author: yukiarimo
Pull request: no

How to reproduce:

Run this:

```
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the base model in FP16
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/gemma-3-1b-pt",
    low_cpu_mem_usage=True,
    return_dict=True,
    torch_dtype=torch.float16,
    device_map="mps",
)

# Load and configure the tokenizer
tokenizer = AutoTokenizer.from_pretrained("unsloth/gemma-3-1b-pt", trust_remote_code=True)

# Generate the text
prompt = "<bos>Once upon a time"
inputs = tokenizer(prompt, return_tensors="pt").to(base_model.device)
outputs = base_model.generate(inputs.input_ids, max_length=50)
# Decode the generated text
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)
```

Error:

```
(yuna) yuki@yuki AI % python gener.py
  k_out_updated = k_out_shifted.index_copy(2, update_position, key_states)
Traceback (most recent call last):
  File "/Users/yuki/Documents/AI/gener.py", line 19, in <module>
    outputs = base_model.generate(inputs.input_ids, max_length=50)
  File "/opt/anaconda3/envs/yuna/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/opt/anaconda3/envs/yuna/lib/python3.10/site-packages/transformers/generation/utils.py", line 2623, in generate
    result = self._sample(
  File "/opt/anaconda3/envs/yuna/lib/python3.10/site-packages/transformers/generation/utils.py", line 3649, in _sample
    next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
```

System: macOS Tahoe, MacBook Pro M1 with 16 GB of RAM

Comments:
Hi @yukiarimo, the issue is that you are setting `torch_dtype=torch.float16`, which makes the forward's output tensor all nans. If you remove that line, your script should work.

Fixed. Loaded in float32, worked. Anyway, for GGUF/MLX all formats work, it's just this

So, can you simple explain why it gives nans? Is float16 operations aren't available on MPS?

I got the same issues on Linux, so I wouldn't say it's a matter of backend.  It's surely related to precision but I haven't investigated. It would be nice to know if it happens only with these weights or also with others.

In general, many models fail with `float16`, and they are not designed to support it! Modern models generally are designed for `bfloat16`, which has the same range of values as `float32`, and so doesn't suffer from the underflows/overflows that cause `NaN`.
PATCH: add back n-dim device-mesh + fix tp trainer saving
Author: S1ro1
Pull request: yes
Labels: name: for patch

1. Fixes ndim check on device_mesh - This was merged in previously with #38949 but was by mistake reverted by #39501, we need this for upcoming accelerate/axolotl release. 
2. Makes sure we save properly on distributed rank if tp allowed in trainer



cc @ArthurZucker @SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39693). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llama

Fails unrelated, merging
Remove all expired deprecation cycles
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Time for cleanups!

cc @gante @qubvel @zucchini-nlp as well as you started some of those deprecation cycles!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39725). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: align, altclip, beit, bert, bert_generation, blip_2, bridgetower, bros, camembert, chinese_clip, clap, cohere2, data2vec, decision_transformer, dpt, electra
Update `QAPipelineTests::test_large_model_course` after #39193
Author: ydshieh
Pull request: yes

# What does this PR do?

Update `QAPipelineTests::test_large_model_course` after #39193

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39666). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
skip `Glm4MoeModelTest::test_torch_compile_for_training`
Author: ydshieh
Pull request: yes

# What does this PR do?

We have line

> token_indices, weight_indices = torch.where(mask)


which won't work with this test.

It's skipped also in dots1 and deepseek 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39670). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4_moe
Fix Layer device placement in Caches
Author: Cyrilvallez
Pull request: yes
Labels: name: for patch

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/39730!
If the Layer were initialized on the correct device in the beginning, this should not be needed. However, as explained in the comment and as was the case before, sometimes `generate` does not initialize them correctly (this should be fixable in the future and would be the preffered way, I will investigate this week)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39732). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix cache-related tests
Author: zucchini-nlp
Pull request: yes
Labels: name: for patch

# What does this PR do?

As per title, failing after latest [cache compatibility PR](https://github.com/huggingface/transformers/pull/38635) 

Comments:
run-slow: kyutai_speech_to_text, musicgen_melody, qwen2_5_omni, rag, roformer, superglue

This comment contains run-slow, running the specified jobs: 

models: ['models/kyutai_speech_to_text', 'models/musicgen_melody', 'models/qwen2_5_omni', 'models/rag', 'models/roformer', 'models/superglue']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39676). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: kyutai_speech_to_text, musicgen, musicgen_melody, rag, roformer, superglue

This comment contains run-slow, running the specified jobs: 

models: ['models/kyutai_speech_to_text', 'models/musicgen', 'models/musicgen_melody', 'models/rag', 'models/roformer', 'models/superglue']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: kyutai_speech_to_text, llava_next, llava_next_video, musicgen, musicgen_melody, qwen2_5_omni, qwen2_5_vl, qwen2_vl, rag, roformer, superglue
Disable static cache on certain MoE models
Author: ivarflakstad
Pull request: yes


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39108). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: deepseek_v3

This comment contains run-slow, running the specified jobs: 

models: ['models/deepseek_v3']
quantizations: [] ...

@ydshieh could you take a look at the results from the model tests I ran?
Doesn't look like this had the intended effect 

we should skip the integration test

> tests/models/deepseek_v3/test_modeling_deepseek_v3.py::DeepseekV3IntegrationTest::test_compile_static_cache

that one doesn't use the flag to control the test. It's integration test and usually not control by the flag.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deepseek_v3, dots1

run-slow: deepseek_v3, dots1

This comment contains run-slow, running the specified jobs: 

models: ['models/deepseek_v3', 'models/dots1']
quantizations: [] ...

run-slow: deepseek_v3, dots1

This comment contains run-slow, running the specified jobs: 

models: ['models/deepseek_v3', 'models/dots1']
quantizations: [] ...
Fix AMD dockerfile for audio models
Author: remi-or
Pull request: yes

This PR modifies the AMD dockerfile because its base container `rocm/pytorch:rocm6.4.1_ubuntu24.04_py3.12_pytorch_release_2.7.1` has been updated by AMD to include `torchaudio` . Since it also includes `torchvision`, I removed the line installing both those packages again.

I also added  the `audio` dependency to `transformers` and the `torchcodec` package, because it seems necessary on AMD, otherwise test fail when decoding audio. I have tested the Nvidia container on A100 and `torchcodec` does not seem to be needed there, which might be because either AMD has this additional dependency or because its dependecies are older.

Also pinned `parametrized` to 0.9 or above, because otherwise test parametrized this way:
```@parameterized.expand([256, 512, 768, 1024])```
(from `tests/models/gemma3n/test_processing_gemma3n.py` line 106) will fail, for instance with version 8.1 which was in the AMD container.

Comments:
Forgot to write it in the PR body, but I tested the new image on MI300 and there were no significant new failures there, and audio models are mostly working.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39669). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> for instance with version 8.1 which was in the AMD container.

you mean `pytest` version right?

(Nvidia daily CI have 8.4.1, maybe that is why we don't see the issue there)

No I think `parameterized` is a separate package, at least it is listed as a separate dependancy.

I mean, when you say `with version 8.1`, what package you mean ?

When I said with `version 8.1` I was talking about the `parameterized` package 

ah, ok, you mean 0.8.1 ...
Refactor (and fix) gpt_neox
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title, it refactors the model (it was in an in-between state, using attention functions but not the general ones). It also fixes the issue in flex attention (see https://github.com/huggingface/transformers/pull/35605)
It add supports to be used in tgi and vllm.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35610). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sorry to necropost, but this PR appears to break NeoX-like models with different `rotary_pct`s unless I'm missing something?
There's no clearly equivalent behavior in the new modular code of this slicing behavior from the old modelling code:
https://github.com/huggingface/transformers/blob/80dbbd103c217f422de91a3265bf6d8e8bc414f7/src/transformers/models/gpt_neox/modeling_gpt_neox.py#L427-L431


In the new version, we still calculate `rotary_ndims`:
https://github.com/huggingface/transformers/blob/a5923d4de7df2fbd1f373dfcfe983216b79b6937/src/transformers/models/gpt_neox/modeling_gpt_neox.py#L132

but it seems to only ever be used or passed when updating the cache of the prior state?  (IE, not being used when actually calculating attention scores on the current token)
https://github.com/huggingface/transformers/blob/a5923d4de7df2fbd1f373dfcfe983216b79b6937/src/transformers/models/gpt_neox/modeling_gpt_neox.py#L157-L168

It's now done directly in `apply_rotary_pos_emb`!

Hmmmm but the cos is never truncated tho? The `apply_rotary_pos_emb` just directly uses it, no slicing, so when it gets the shape it would just be the full dims

https://github.com/huggingface/transformers/blob/a5923d4de7df2fbd1f373dfcfe983216b79b6937/src/transformers/models/gpt_neox/modeling_gpt_neox.py#L273-L285

Please have a look here https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_rope_utils.py#L111-L119

Ahhhhh I see now, thank you! Odd that I was running into some issues with this arch then but I suppose that's a separate me-issue :p
add profiler to trainer
Author: re-imagined
Pull request: yes

# What does this PR do?

add profiler to trainer

related issue:
https://github.com/huggingface/transformers/issues/36360#issuecomment-2844195069



Comments:
cc @SunMarc 

> Thanks for getting it a go @re-imagined. I think that using the callback will be better. From now, it looks easier with this profiler but as we add different profilers, we don't want to bloat the training code. Also instead of enable_profiler, can you use `profiler = "pytorch"` and profiler is `None` by default instead as we might add more profiler in the future.

Thanks you for your review, I will re-write it with`callback`.


Hi @SunMarc 👋,

I've implemented the profiler integration in the callback as we discussed. Here's a quick overview:

add PyTorch Profiler to the callbacks if TrainingArgument's profiler is `pytorch`

init the callback with basic default configurations:
1. result table in log: sort by `self_cuda_time_total` by default
2. init profiler with `profile_memory=True, export_chrome_trace=True` by default, and a simple `schedule`

Enabled customization through TrainingArguments for:
1. Profiler parameters
2. Result table formatting:

Maintained compatibility while adding new profilers


This is currently a draft implementation that aligns with our previous discussions, but I'm working for further refinement (for example, the result table in log maybe still not reader friendly). Would you mind reviewing the approach again when you get a chance?

Specifically, I'd appreciate your feedback on:
1.The default configuration choices
2.The integration pattern with existing training arguments
3.Any additional parameters we might want to expose
4.Supporting different devices (like xpu)
Looking forward to your thoughts!

@SunMarc 👋  hi, would you mind take a look at this PR
🌐 [i18n-KO] Translated '<text-to-speech>.md' to Korean
Author: taemincode
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다! -->
# What does this PR do?

Translated the `<text-to-speech>.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에, 이 아래에 리뷰를 요청할 팀원들을 멘션해주세요! -->
May you please review this PR? 
@4N3MONE, @yijun-lee, @jungnerd , @harheem

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->

Comments:
Hi @taemincode 👋 Thanks for your translation.
I just to point out one thing, please check [our translation guide](https://mangrove-bridge-a13.notion.site/Hugging-Face-transformers-22b5e59aabcb804982ebd97f233d7ae2?source=copy_link) and make sure to follow it when working on your translations. 
It looks like the current PR may have been translated by LLM, but it's essential to have a human go through and refine the text to make it sound natural. So, we kindly ask you to review and revise the translation again. 🤗

If you have any trouble with the translation, always feel free to ask me! 
If needed, we can have an online meeting 😊

@jungnerd Thank you so much for the feedback and for guiding me through this process. I really appreceiate your support!

I’ll go through the translation again and revise it carefully to make it sound more human like and natural(following the translation guide you shared me).
I’m actually currently on a family trip, so it might take a bit of time for me to update the PR. I hope that’s okay! 🙏
If I run into trouble, I’ll definately reach out; and I’d be happy to join an online meeting if needed. Thanks again!