noamaanMulla-03/transformers-issues huggingface dataset:
Design question for integrating new model to Transformers?
Author: Manalelaidouni
Pull request: no

Hey, I’ve been working on adding the new Yue Model to Transformers, it’s a lyrics to song generation model that takes a lyrics prompt and an optional audio prompt. he audio prompt is passed to a sota audio tokenizer called X-codec that excels at capturing both semantic and accoustic information which significantly improves text-to-audio alignment. 

X-codec first encodes the audio using an acoustic model (DAC) and then extracts semantic information using a pretrained Hubert model, this semantic information is further refined with a semantic encoder (based on RepCodec). The combined features are then fed into a RVQ module (residual vector quantizer) that converts them into discrete tokens for the language model to predict

YuE itself uses a two stage language model based on Llama2, toped with a vocoder to go from the intermediate sound representation to the song waveform.

**My question is :**
Can I wrap the X‑Codec model functionality as a feature extractor so that it becomes part of the processor YuEProcessor for a multi-model input? or since it’s a relatively complex model should I instead add it as an optional part of the model architecture? I’ll be opening a PR soon, any pointers would be greatly appreciated!

<br>

<img width="452" alt="Image" src="https://github.com/user-attachments/assets/a1e6a5a2-4e82-41f7-991f-5aca093e7155" />

*The model was out for a while but the [paper](https://arxiv.org/abs/2503.08638) has just been released last week.*




Comments:
@Manalelaidouni hey! This is quite similar to VQ-VAE based vision models we have in transformers, for ex Chameleon and Emu3 (a few more PRs still in progress). 

The general idea of a processor is to be 1) lightweight and fast; 2) no dependency on specific frameworks like torch etc. So we usually add quantizer modules as part of the model. Feel free to check out how the above mentioned models are integrated

cc @eustlb for audio

Hey @Manalelaidouni, it's cool you're working on Yue Transformers integration !! 🤗

We want to have a separate X-Codec integration PR so that the modelling can live by itself, as for [mimi](https://huggingface.co/docs/transformers/en/model_doc/mimi) or [dac](https://huggingface.co/docs/transformers/en/model_doc/dac), all the more since it will enable other model integrations that would rely on it (e.g. Llasa 😊).

Then, the model can hold a reference to this codec model; see Moshi for example: 
https://github.com/huggingface/transformers/blob/19b9d8ae13c99554e5c857c8534f87b7951bf821/src/transformers/models/moshi/modeling_moshi.py#L1929

Thanks a lot for your pointers, very helpful! I'll be opening a PR for X-codec soon 😊
Refactor torchao docs 
Author: MekkCyber
Pull request: yes

# What does this PR do?
Refactoring torchao docs, with clear doc layout, better and exhaustive examples, benchmarks and more !


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37490). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @DerekLiu35 

Thanks a lot @MekkCyber! I was also planning to add more examples recently, your PR makes it much easier, the formatting is really nice.

one thing I'm thinking of is we can separate the example by target hardwares like (A100, H100, CPU etc.), will work on this now
Add code examples for creating & fine‑tuning EncoderDecoderModel (fixes #16135)
Author: HarshitaTechWizard
Pull request: yes

This PR implements the first two code‑example requests from Issue #16135:

1. Create & Save an EncoderDecoderModel
   - Shows how to load pre‑trained encoder & decoder, then save the combined model.

2. Fine‑Tune an EncoderDecoderModel
   - Demonstrates a one‑epoch training loop on a tiny sample dataset.

Both snippets are inserted under the appropriate bullets in `docs/source/model_doc/encoder-decoder.mdx` and have been verified by a local `make docs` build.  

Closes #16135.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @stevhliu 

I think this might already be fixed. The current version of the [docs](https://huggingface.co/docs/transformers/v4.51.3/en/model_doc/encoder-decoder#overview) already demonstrate initializing the model with `from_encoder_decoder_pretrained`, setting up the config, and includes a Colab notebook for training.
AssertionError when using bfloat16 with flash_attn on loading Qwen2.5 VL
Author: alexxchen
Pull request: no
Labels: name: bug

### System Info

transformers==4.49.0

### Who can help?

@li-plus 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

set gradient_checkpointing=True for trainer and will get x.dtype == cos.dtype AssertionError: Input and cos/sin must have the same dtype, got torch.float32 and torch.bfloat16

### Expected behavior

without error

Comments:
Duplicate of https://github.com/huggingface/transformers/issues/36187

Clsiong as resolved, https://github.com/huggingface/transformers/pull/36188 was merged! You can install transformers from `main` branch to get the fixed model: `!pip install --upgrade git+https://github.com/huggingface/transformers.git`



AssertionError: Input and cos/sin must have the same dtype, got torch.float32 and torch.bfloat16 .I had the same problem,can you help me 

@xdlzr the fix was merged a few days ago, can you install transformers from main branch and try again? LMK if that doesn't help with a small reproducer

Issue still exists in most recent version:

Temporary fix:
`pip install git+https://github.com/huggingface/transformers.git@8ee50537fe7613b87881cd043a85971c85e99519`

Issue solved with pip install -U transformers
(transformers==4.51.3)
Fix Quark quantization config
Author: MekkCyber
Pull request: yes

# What does this PR do?

Raises an `ImportException` in the quantization config if quark is not installed, since it's needed there

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37578). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix some GPU OOM after #37553
Author: ydshieh
Pull request: yes

# What does this PR do?

Can't say anything why #37553 gives some tests GPU OOM and the only way I can make those tests to pass is to do some cuda cleanup at some places.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37591). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Flag SpeechT5 flaky test
Author: molbap
Pull request: yes

# What does this PR do?

Flags a test as flaky - it's been blocking a few PRs. Not sure what is causing it to fail, cc @eustlb here maybe

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @Cyrilvallez  for core review! fewer loc than last one haha

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37587). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ydshieh too here, related to some failures I've seen here https://github.com/huggingface/transformers/pull/37391
🌐 [i18n-KO] Translated 'serving.md' to Korean
Author: olccihyeon
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `<your_file>.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [ ] Check for missing / redundant translations (번역 누락/중복 검사)
- [ ] Grammar Check (맞춤법 검사)
- [ ] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 본인의 조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
<!-- @cjfghk5697, @yijun-lee, @rlaalsrl0922 , @MinJu-Ha -->
<!-- @cjfghk5697, @yijun-lee, @devxaitist, @nsbg -->
<!-- @cjfghk5697, @yijun-lee, @Kim-Ju-won @junhkang @olccihyeon -->
<!-- @cjfghk5697, @yijun-lee, @garongkim @maximizemaxwell -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
cc @stevhliu 

Hi, thanks for your translation! There is already an open PR (#36757) for `serving.md`. Would you perhaps like to review the translation in that PR?
example with no trainer use accelerator.end_training() in a wrong way
Author: we1559
Pull request: no
Labels: name: bug

### System Info

the main branch could reproduce

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run the example with no trainer such as
`export TASK_NAME=mrpc

accelerate launch run_glue_no_trainer.py \
  --multi_gpu
  --num_processes 4
  --with_tracking
  --model_name_or_path google-bert/bert-base-cased \
  --task_name $TASK_NAME \
  --max_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --output_dir /tmp/$TASK_NAME/
`

it will fail here
https://github.com/huggingface/transformers/blob/9cda4265d61b0ecc276b705bd9b361a452106128/examples/pytorch/text-classification/run_glue_no_trainer.py#L641

the error message shows we don't init the process group


### Expected behavior

actually, I found all the example with no trainer use the accelerator.end_training() in a wrong way

there are 2 bugs here

for example
https://github.com/huggingface/transformers/blob/9cda4265d61b0ecc276b705bd9b361a452106128/examples/pytorch/text-classification/run_glue_no_trainer.py#L637-L646

the first bug:
it will end_training first and try to wait_for_everyone and then save the model
but we can see
https://github.com/huggingface/accelerate/blob/3b89987710e4edf1e55b23d5ac208525a06a70c4/src/accelerate/accelerator.py#L2982-L3001

the end_training do two things. 
1. finsh all the tracks, this is what the author want to do in example_no_trainer code because it will check if with_tracking before call it.
2. destroy the process group

after destroy the process group, we can't call wait_for_everyone or any other code related to the distributed module.
so we should call the accelerator.end_training() at the end of main() to make sure we will destroy the process group after all the possible distributed behaviour.

the second bug:
it would only call the end_training() when with_tracking. But we should always destroy the process group with or without tracking

Comments:
[generate] model defaults being inherited only happens for newer models
Author: gante
Pull request: yes

# What does this PR do?

#36684 activated the use of model-defaults (as opposed to global defaults) for unfilled parameters in the `generation_config` argument.

It may lead to unexpected behavior on models that saw no changes, especially if the model has defined flags like `max_length` or `max_new_tokens`. This PR adds a flag to control this behavior, and makes it the default only for models saved from v4.50 (next release) onwards.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36881). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I personally think the processing logic can be further improved.

For example, consider a user wanting to ensure greedy decoding by explicitly setting `my_config = GenerationConfig(do_sample=False)`. Since `do_sample=False` is also the default in the base `GenerationConfig`, the merging logic might interpret this as "not specified by the user." Consequently, if a model-specific configuration (like `gemma3_config`) defaults to `do_sample=True`, it could unintentionally override the user's explicit `do_sample=False` setting.

**The core issue is that the current approach seems to primarily track *deviations* from the default, rather than *explicit user intent*.**

**Perhaps the `GenerationConfig` object or the merging process could internally track which parameters were explicitly provided by the user during instantiation, even if the provided value matches the default.** This would allow the merging logic to prioritize any explicitly user-set value over model-specific defaults.

I'm not familiar enough with the codebase to propose a specific implementation, but hopefully, this description of the problem and the suggested conceptual approach is helpful.

@MTDickens yes, this will be improved :) Our path forward will consist of having per-model `GenerationConfig` classes, like we have per-model config classes. That way, when instantiating a new generation config, the starting point are the model-specific defaults, as opposed to generalist defaults + convoluted logic
Next batch of models with removed return_dict
Author: qubvel
Pull request: yes

# What does this PR do?

Continue removing the `return_dict` attribute from the modeling code.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37396). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Deprecate modeling_utils.py classes
Author: qubvel
Pull request: yes

# What does this PR do?

Classes such as `PoolerStartLogits`, `PoolerEndLogits`, `PoolerAnswerClass`, `SQuADHead`, and `SequenceSummary` are:

1. Defined in `modeling_utils.py`, which is somewhat against the main repository philosophy of having classes defined within dedicated modeling files
2. Used by a few older models and not used by newer ones

Following the above, I have transferred them to the corresponding modeling files and deprecated those in `modeling_utils.py`. This should unclutter `modeling_utils.py` and make it easier to remove those classes when models are going to be deprecated.

Comments:
run-slow: gpt2, xlm, xlnet, electra

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/gpt2', 'models/xlm', 'models/xlnet']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37298). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @Rocketknight1 when you have bandwidth

Change seems good but this is kind of a core library philosophy issue, so we'll need a core maintainer review! cc @Cyrilvallez
Fix gradient checkpointing bug Bart
Author: saswatmeher
Pull request: yes

# What does this PR do?
This PR fixes a bug that a user can encounter while using generate and models that use gradient_checkpointing.

Fixes issue #21737 for Bart.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. (#21737)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
cc @younesbelkada, @gante

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks a lot for your great work! 
Can you please run `make fix-copies` ? After that we should be good to merge
Add support for Moonlight 16B, add aux loss for Deepseek v3 model finetuning.
Author: Kimiko-AI
Pull request: yes
Labels: name: New model

# What does this PR do?
This add support for moonlight 16B which uses q_lora_rank = null, add sequence-wise auxiliary loss proposed in deepseek v3 paper and this [reimplemtation ](https://github.com/ScienceOne-AI/DeepSeek-671B-SFT-Guide/blob/main/model/DeepSeek-R1-BF16/modeling_deepseek.py)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] I read the contributor guideline, Pull Request section


@ArthurZucker




Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @Cyrilvallez 

Thanks for the new model and PR! 🚀 

> seq_aux name needs to be better + is this used or not by default for released checkpoint

In the paper they said they used it to prevent extreme imbalance within any single sequence (EQ 17-20)

> ce and single letter variables need better naming!
> branches for the loss should be like we do in Mixtral: we return the logits / topK indices to be computes outside, no in the module itself!
> Loss should not be in the model!

Yea, it is mostly a hack so I can get training working on axolotl. I will fix it as soon as possible

I am using modular to generate the modeling file but accidentally include that when push to github

Let me know when this is ready for review! 🤗
[Bugfix] Set default value for output_attentions parameter in Gemma3T…
Author: chenin-wang
Pull request: yes

# What does this PR do?

[Bugfix] Set default value for output_attentions parameter in Gemma3TextModel

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


@ArthurZucker @amyeroberts



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.


<img width="953" alt="image" src="https://github.com/user-attachments/assets/724319c1-b6c0-4562-9ae0-a60a2e745018" />



@chenin-wang can you update `gemma2/modular_gemma2.py` and then run `make fix-copies` to make CI happy?

thank you.I will do those and reopen.
Fix InternVL attention when using qk_norm (38B and 78B)
Author: yonigozlan
Pull request: yes

Fix an issue with InternVLVision Attention, where qk norm is not applied correctly

Cc @Cyrilvallez 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37620). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.