noamaanMulla-03/transformers-issues huggingface dataset:
Why it's easy to get nan for A_log in Qwen3Next
Author: npuichigo
Pull request: no
Labels: name: bug

### System Info

main branch to use Qwen3Next

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Tried many times for training, but it's really easy to get a nan value in A_log for some layer. Does anyone know the cause of this kinda instability?

```python
In [5]: for i, param in enumerate(a['shadow_params']):
   ...:     if param.isnan().any():
   ...:         print(i, param)
   ...: 
237 tensor([ 2.5156,  1.8125,  1.4062,  0.1504,  2.0000,  2.3594,  1.9375,  1.8438,
         1.3047,  0.8633,  2.5938,  2.2500,  2.5000,  0.4414,  1.8516, -0.3848,
         2.2812,  2.7500,     nan,  1.5078,  2.5469,  1.6406,  2.5000,  1.5391,
         2.4219,  1.6953,  1.4609,  1.0625,  0.4199,  2.5156,  2.0156,  2.4375],
       dtype=torch.bfloat16)

In [6]: a['parameter_names'][237]
Out[6]: 'transformer.model.layers.20.linear_attn.A_log'
```

### Expected behavior

Normal training

Comments:
Just a quick guess, but I assume that the initialization is not favorable atm, e.g. in the fla lib we have https://github.com/fla-org/flash-linear-attention/blob/c3bd56589033610264532b11f0972c69e4645f6e/fla/models/gated_deltanet/modeling_gated_deltanet.py#L131-L141

while in qwen3 next, we have https://github.com/huggingface/transformers/blob/374ded5ea40c9026cf36003c737e07c584dbd63d/src/transformers/models/qwen3_next/modeling_qwen3_next.py#L970-L972

This could easily lead to unstable first forwards / backwards. Are you also running mixed precision?

Yeah, with bf16. I have to clamp the gradient of A_log and after exp calculation to able to train smoothly, otherwise it's easy to get nan. With zamba like initialization ``torch.arange(1, self.num_heads + 1)`, I just got more nan.

I meant the `dt_bias` initialization as well (see the fla init), not sure how much it helps with the dynamics here as well. Thanks for the insights, though.

Not sure if we can do much from our side except tweaking the initialization 🤔 things like gradient clamping etc do sound like training recipes specific to the model architecture.

Yeah. clamp just works as workaround. Hope to get more stable training from the model side.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add Model Support for xLSTM
Author: stefan-it
Pull request: no
Labels: name: New model

### Model description

Inspired by [recent rumors](https://www.youtube.com/watch?v=hwIt7ezy6t8) about xLSTM - a hidden successor to LSTM - by Sepp Hochreiter, this issue tracks the open source implementation about adding xLSTM to Transformers library.

### Open source status

- [x] The model implementation is available [here](https://github.com/NX-AI/xlstm) and [here](https://github.com/NX-AI/xlstm-jax) as JAX implementation
- [x] Transformers integration is currently available on their [Transformers fork](https://github.com/NX-AI/transformers/tree/integrate_xlstm) 
- [x] The model weights are available [here](https://huggingface.co/NX-AI/xLSTM-7b) (7B model)

### Provide useful links for the implementation

- [x] Paper is available [here](https://arxiv.org/abs/2405.04517)

At the moment no implementation does exist.

Only rumors that xLSTM surpasses GPT-2 on various (small) downstream datasets.

Good overview is the [xLSTM Resources](https://github.com/AI-Guru/xlstm-resources) repository from @AI-Guru.

Comments:
Sounds like a money grab. If it is something useful, he should have chosen the academic path or at least filing patent.

This way of boldly claiming success via non-serious media channels is highly unprofessional. It smells like publicity is more relevant than results which further supports motivations like funding/personal gains/politics.

If I understood it correctly, a patent is on its way, and at least a paper about xLSTM will be published in less than 6 month.

I have some doubts if this is planned as an open source model. 

Paper is published now: https://arxiv.org/abs/2405.04517


Need code and checkpoint or it didn't happen. 

Official implementation is out now:

https://github.com/NX-AI/xlstm

Note that the official source code is AGPL-licensed.

December 11th update:

* The xLSTM JAX implementation was released [here](https://github.com/NX-AI/xlstm-jax)
* Transformers integration can be found in their [Transformers fork](https://github.com/NX-AI/transformers/tree/integrate_xlstm)
* 7B model has landed on the [model hub](https://huggingface.co/NX-AI/xLSTM-7b)

:partying_face: 

It was long ago integrated into Transformers, so I am closing here!
Revert "permissions worflows fix"
Author: ydshieh
Pull request: yes

Reverts huggingface/transformers#42080 for now, so @paulinebm could debug.

The merged PR cause (most, if not all) workflow runs fail:

https://github.com/huggingface/transformers/actions/runs/19194657983/job/54873901707
https://github.com/huggingface/transformers/actions/runs/19186763680/job/54854623041

Revert for now. 

Merge directly without requesting a review, as it's quite critical

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42110). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @paulinebm

Maybe you can create a new dummy workflow file, add this 

> - uses: GitHubSecurityLab/actions-permissions/monitor@v1

in that workflow, trigger a run for it. Once we know this job works, you can delete the dummy workflow, apply the changes to all other workflow files again.
[Qwen3-VL] The comment for this function may be incorrect.
Author: pengshuang
Pull request: no

https://github.com/huggingface/transformers/blob/c79040303967682a6351f3fc01a02b8ce5a13568/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py#L352

`[THTHWHTHW...TT]` here needs to be corrected to `[THWTHWTHW...TT]`

Is that right?

Comments:
I think you're right but cc @zucchini-nlp to confirm!
does DeepSeek-V3.2-Exp have update？
Author: Mally-cj
Pull request: no

cannot find DeepSeek-V3.2-Exp

Comments:
Hi can I work on this issue ?

PR for this is here: https://github.com/huggingface/transformers/pull/41251

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add Contribution Guidelines and License Section in README
Author: dimpal-yadav
Pull request: no

Description
Currently, the README.md does not have a Contribution section and the License section is either missing/not clearly visible.
Adding these two sections will make the project more professional, contributor-friendly, and compliant with open-source standards.

🎯 Acceptance Criteria
Contribution section added in README.md
License section added in README.md
Link to CONTRIBUTING.md and LICENSE file included

Comments:
Hi @dimpal-yadav, I'd like to contribute on solving this issue. Could you please assign it to me?
/assign

Hi @dimpal-yadav, I have opened up a PR for this issue as well as issue #41244. Please review them and suggest any changes required!

Hey @dimpal-yadav , I'd like to work on this issue. Could you please assign it to me?


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add hacktoberfest-2025 Banner to README
Author: dimpal-yadav
Pull request: no

The repository currently does not display any hacktoberfest-2025 banner in the README. Adding the banner will help highlight the project’s participation in hacktoberfest-2025 and increase visibility.

Comments:
Hi @dimpal-yadav , I'd like to contribute to solving this issue. Could you please assign it to me?
/assign

Hi @dimpal-yadav, I’d like to work on this. Could you please assign me?

Hi, @dimpal-yadav, I can work on this.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Correct type hint in config models
Author: jorisSchaller
Pull request: yes



# What does this PR do?
Most models are using a config with `rms_norm_eps: Optional[int] = 1e-6`: I modified it to an optional float.
This update configs that use a type of optional int instead of optional float for the epsilon value for their RMS norm.
You can clearly see that the epsilon has a default value of 10e-6 (float), the hint should therefore be adjusted.

FIx no issue.


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

Let me know if you want any changes,  @ArthurZucker @Cyrilvallez @stevhliu, I tried to find all occurence of it, but some might have slipped through.



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deepseek_v2, deepseek_v3, dots1, gemma, gemma2, gemma3, granite, granitemoe, granitemoehybrid, granitemoeshared, jetmoe, llama, mistral, qwen2, qwen2_5_omni, qwen2_moe

Looks good but please run `make fix-copies` to propagate modular changes! Ping me again whenever the CI is green and I'll merge it
Custom loss function for Qwen 2.5 VL 7b training
Author: akhila1729
Pull request: no
Labels: name: Vision

 /raid/akhila/conda_envs/mswift/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py

here there is self.loss_function under Qwen2_5_VLForConditionalGeneration class. I want to add a custom loss function which should be accessed from config. I am doing SFT training using mswift

Comments:
You can do it with `model.loss_function  = MyLossFn`. Note that the loss function has to accept the same input kwargs, in our case labels and logits. And it has to return a single loss value

@akhila1729 I would love to contribute in this could you assign it to me 


Hey `@akhila1729`, building on `zucchini-nlp`'s suggestion for a quick test, the most robust and idiomatic way to handle a custom loss within a training framework is to hook into the trainer itself, rather than the model. Since you're using **`mswift`**, it's highly recommended to check its documentation first, as it likely provides a dedicated argument in `SftArguments` or a `compute_loss` method in its trainer class designed for this purpose. For those using the standard `transformers.Trainer`, the best practice is to **subclass it and override the `compute_loss` method**. Both of these approaches keep the model architecture standard and cleanly separate the training logic from the model logic, which is much more maintainable for structured projects.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Seems to be resolved, feel free to re-open if you have any questions left
Fix return metadata checking logic
Author: Xqle
Pull request: yes

# What does this PR do?

 As suggested in https://github.com/huggingface/transformers/pull/42009#discussion_r2502673870, when `return_metadata` is in `kwargs` but set to `False`, metadata should be popped.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@zucchini-nlp @yonigozlan @molbap

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4v, qwen3_vl, smolvlm, video_llama_3

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42108). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Correctly handle unbatched audio inputs in Gemma3nAudioFeatureExtractor
Author: kho
Pull request: yes

# What does this PR do?

Correctly handle unbatched audio inputs in Gemma3nAudioFeatureExtractor

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
Thanks for sending this fix @kho!

cc @zucchini-nlp @Rocketknight1 for a review to help us land this for Gemma 3n.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3n

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42076). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Bugfix] fix qwen3vl expand generation with video
Author: JJJYmmm
Pull request: yes

# What does this PR do?

 fix qwen3vl expand generation with video

related issues: https://github.com/QwenLM/Qwen3-VL/issues/1769, https://github.com/QwenLM/Qwen3-VL/issues/1621

@zucchini-nlp 

test script:

```python
from transformers import AutoModelForImageTextToText, AutoProcessor

ckpt_path = "Qwen/Qwen3-VL-30B-A3B-Instruct"

# default: Load the model on the available device(s)
model = AutoModelForImageTextToText.from_pretrained(
    ckpt_path, dtype="bfloat16", device_map="auto",
    attn_implementation="flash_attention_2"
)

processor = AutoProcessor.from_pretrained(ckpt_path)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "video",
                "video": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2-VL/space_woaudio.mp4",
                "video": "space_woaudio.mp4",
            },
            {"type": "text", "text": "Describe this video."},
        ],
    }
]

# Preparation for inference
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt"
)
inputs = inputs.to(model.device)

# Inference: Generation of the output
num_return_sequences = 2
generated_ids = model.generate(**inputs, max_new_tokens=128, num_return_sequences=num_return_sequences)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip([inputs.input_ids] * num_return_sequences, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)
```
🫡 test pass with 
```bash
RUN_SLOW=True pytest tests/models/qwen3_vl_moe/test_modeling_qwen3_vl_moe.py::Qwen3VLMoeIntegrationTest
```

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_vl, qwen3_vl_moe

run-slow: qwen3_vl_moe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42089). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen3_vl_moe"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19171152958)

✅ No failing test specific to this PR 🎉 !


Slow tests are green, so let's merge
Add repro & preload-workaround scripts for torchcodec / transformers import-order issue
Author: Arcanixhades0
Pull request: yes

This PR adds small reproducible scripts and documentation demonstrating an import-order issue between torchcodec and transformers' audio modules and a minimal runtime workaround that preloads the system libavcodec before loading torchcodec's extension.\n\nFiles added:\n- scripts/torchcodec_preload/repro_fail_order.py — repro that imports torchcodec first and may fail\n- scripts/torchcodec_preload/repro_fix_ctypes_preload.py — preloads libavcodec via ctypes then loads torchcodec and transformers\n- scripts/torchcodec_preload/README.md — instructions to inspect libavcodec and run examples\n\nTesting notes:\n- In some environments (including the container used to create this PR), ffmpeg/libavcodec may not be installed;  may return nothing. Run  locally to find a candidate path.\n- Recommended local test sequence:\n  1) \n  2) Could not find libavcodec via ctypes.util.find_library('avcodec').
Run `ldconfig -p | grep avcodec` to locate a system lib, or set LD_PRELOAD. — should preload and succeed\n  3)  — demonstrates the original failing order\n\nRationale and suggested next steps:\n- This PR intentionally adds only examples and docs to help reproduce and triage loader/symbol-version mismatches without modifying library code.\n- If maintainers prefer, we can follow up with either a small runtime helper in  to optionally preload ffmpeg libs, or open a reproducible issue on the torchcodec repo and collaborate on a longer-term fix.\n\nIf you want me to also add a small helper function and unit tests in this PR, say so and I will prepare the changes.

Comments:
Fix base model prefix in VLMs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fiexes https://github.com/huggingface/transformers/issues/41814. The incorrect prefix is throwing errors when loadiing a base model from checkpoint

It was a bad copy from llava, where the prefix is empty string to support BC with old-style checkpoints

Update: just realized I had a PR already in https://github.com/huggingface/transformers/pull/41589 😅  which solves this issue + adds new functionality

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42059). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cohere2_vision, florence2, gemma3n, lfm2_vl, qwen3_vl, qwen3_vl_moe
📝 docs(smolvlm): fix variable name in batch inference example
Author: gorkachea
Pull request: yes

## Description

Fixed incorrect variable name in the SmolVLM documentation batch inference example.

## Issue

The example code on line 162 creates a list called `conversations` (plural) but incorrectly references `conversation` (singular) in the `apply_chat_template()` call. This would cause a `NameError` when users try to run the example.

## Changes Made

- Changed line 162 from `conversation` to `conversations` to match the variable defined on line 160

## Before
```python
conversations = [conversation1, conversation2, conversation3]
inputs = processor.apply_chat_template(
    conversation,  # ❌ NameError - undefined variable
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, dtype=torch.bfloat16)
```

## After
```python
conversations = [conversation1, conversation2, conversation3]
inputs = processor.apply_chat_template(
    conversations,  # ✅ Correct variable name
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, dtype=torch.bfloat16)
```

## Impact

This fix ensures the batch mixed media inference example runs without errors for users following the documentation.

## What does this PR do?

Fixes a bug in the SmolVLM documentation where the batch inference example uses an incorrect variable name that would cause a `NameError` when users try to run the code.

## Before submitting

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the contributor guideline, Pull Request section?
- [x] Was this discussed/approved via a Github issue or the forum? - Not required for obvious documentation bugs
- [x] Did you make sure to update the documentation with your changes? - This IS a documentation fix
- [x] Did you write any new necessary tests? - Not applicable for documentation fixes

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [x] Documentation update

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Documentation maintainers or anyone familiar with SmolVLM examples.



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42123). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.