noamaanMulla-03/transformers-issues huggingface dataset:
docs: update LLaVA-NeXT model card
Author: Bpriya42
Pull request: yes

# What does this PR do?
As mentioned in the issue https://github.com/huggingface/transformers/issues/36979 this PR updates the documentation of the Llava-Next model, which will now be aligned with the standardized format for all the docs.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@stevhliu, please let me know if any changes are needed.


Comments:
Hey @stevhliu, I have incorporated your comments. Please let me know if anything else is required. Thank you!

Hey @stevhliu, I have completed the changes mentioned. Please let me know if it is all okay :)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38894). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
DataCollatorForWholeWordMask does not return attention_mask
Author: Aktsvigun
Pull request: no

Hi,

** DataCollatorForWholeWordMask** does not output `attention_mask`. According to the `__call__` method:
`return {"input_ids": inputs, "labels": labels}`. 

Is there a peculiar motivation behind it or is a small bug? From where I see, when we do the pre-training, most instances will **not** be of the same length and applying _Attention_  for all the tokens (including the padding) may cause imprecise results.

Comments:
Not sure where you are seeing this. `DataCollatorWithPadding` can also pad attention masks, as it applies `tokenizer.pad()` on the inputs as can be seen [here](https://github.com/huggingface/transformers/blob/c7faf2ccc05a095870c3c905d232179fb323797d/src/transformers/data/data_collator.py#L118).

@NielsRogge am sorry for confusing, meant **DataCollatorForWholeWordMask**, will correct that now

There is also a bug with `pad_to_multiple_of` argument: it is not passed to `_collate_batch` inside `__call__`. Have made a pull request https://github.com/huggingface/transformers/pull/12999 to add its usage.

That class should never have been merged as it is, it was a mistake on our side. It contains multiple bugs and only works for BERT models. It need to be rewritten from scratch to be model agnostic.

For me it works perfectly for any arbitrary model (checked it is doing everything correct), except for `attention_mask` generation and `pad_to_multiple_of` usage (both can be corrected manually though, however, it is always better not to invent a bicycle).

It cant' give good results on a tokenizer that is not like BERT, since it relies on the "##" to detect if something is inside a word.

Is there any progress on this issue? It seems that this issue still exists.

Still exists

Would it be possible to get this issue reopened? I working on addressing it in a current fork.
Fix typo: langauge -> language
Author: tomaarsen
Pull request: yes

Hello!

## Pull Request overview
* Fix typo: langauge -> language

## Details
I noticed this typo on the [T5Gemma docs](https://huggingface.co/docs/transformers/model_doc/t5gemma), should speak for itself 🤗 

Documentation: @stevhliu

P.s. these don't seem to be working correctly: `<hfoptions id="usage"> <hfoption id="Pipeline">`
![image](https://github.com/user-attachments/assets/705bdeac-4204-43a1-822d-00f5bc436347)


- Tom Aarsen

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39317). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Efficient Expert Weight Fusion for Moe deepseek v3
Author: VassilyLombard
Pull request: yes

# What does this PR do?

This PR answers a call for contribution. It introduces a fully vectorized, efficient Mixture-of-Experts (MoE) implementation for the DeepseekV3 model, specifically in the class DeepseekV3Moe. The new approach eliminates the loop over experts, improving inference and training speed, especially for large numbers of experts (256 here!).

Key Changes
* Efficient MoE Forward Pass:
Instead of iterating over experts, all expert weights are stacked into tensors, and expert selection is performed using indexing and batched matrix multiplications.
* Expert Routing:
For each token, the top-k expert indices and routing weights are computed by the router. Inputs are repeated and routed to the selected experts in a single batched operation.
* Batched Expert Computation:
All expert computations (linear projections and activations) are performed in parallel for all tokens and their assigned experts.
* Aggregation:
The outputs from all routed experts are weighted and summed back to the original token positions, producing the final MoE output without explicit loop.
* Compatibility:
The implementation assumes all experts are instances of the same class and have the same architecture, enabling parameter stacking and efficient computation.

No additional dependencies are required. @ArthurZucker



Comments:
Hi! First-time contributor here, happy to make any changes. I intended to do the second commit only, sorry for the confusion. Please let me know if you need anything else from me!
Granite speech speedups
Author: avihu111
Pull request: yes

# What does this PR do?
This PR speeds up Granite speech by the following changes:
1. Register attention_dist to avoid cpu-to-gpu transfer every layer.
2. `pad_sequence` is much faster than per-sample-padding + concat.
3. Avoid returning audio to cpu when using a compute device.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @eustlb can you give that a look 🙏 
CC: @gsaon @alex-jw-brooks @avishaiElmakies 

Comments:
Could you give some numbers about the speedup? 

Sure! Thanks for looking into this! Commented on each part separately 👍 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: granite_speech

Thanks a lot, that's a great contribution for audio! 🤗
Fix broken SAM after #39120
Author: yonigozlan
Pull request: yes

Was going crazy with tests failing on Sam2 after merging with main, turns out there's a typo in SAM after https://github.com/huggingface/transformers/pull/39120 

Cc @ArthurZucker 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam, sam_hq

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39289). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Thanks, it is weirds that tests were not failing, can you check why none triggered an error for me?

I had failing slow integration tests for Sam on my end
fix xpu failures on PT 2.7 and 2.8 w/o IPEX and enable hqq cases on XPU
Author: yao-matrix
Pull request: yes


Comments:
@ydshieh , pls help review, thx very much. The ci failure case seems not relevant to my PR.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: chameleon, hqq
Delete deprecated stuff
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title. Removes

- deprecated legacy cache
- deprecation we has for new processor API
- `**rope_kwargs` from the RoPE API
- `_seen_tokens` in cache classes

First review @gante as most modifications are around cache/generation

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38838). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

nice, love those PRs! 🎉 

run-slow: align, aria, aya_vision, bart, bigbird_pegasus, blenderbot, blenderbot_small, blip_2, bloom, chameleon, codegen, colpali, colqwen2, csm, dbrx, falcon

This comment contains run-slow, running the specified jobs: 

models: ['models/align', 'models/aria', 'models/aya_vision', 'models/bart', 'models/bigbird_pegasus', 'models/blenderbot', 'models/blenderbot_small', 'models/blip_2', 'models/bloom', 'models/chameleon', 'models/codegen', 'models/colpali', 'models/colqwen2', 'models/csm', 'models/dbrx', 'models/falcon']
quantizations: [] ...

Failing slow tests look to be same as the failing test in `main` branch. @Cyrilvallez can you take one last look and then I'll merge?

run-slow: align, aria, aya_vision, bart, bigbird_pegasus, blenderbot, blenderbot_small, blip_2, bloom, chameleon, codegen, colpali, colqwen2, csm, dbrx, falcon

This comment contains run-slow, running the specified jobs: 

models: ['models/align', 'models/aria', 'models/aya_vision', 'models/bart', 'models/bigbird_pegasus', 'models/blenderbot', 'models/blenderbot_small', 'models/blip_2', 'models/bloom', 'models/chameleon', 'models/codegen', 'models/colpali', 'models/colqwen2', 'models/csm', 'models/dbrx', 'models/falcon']
quantizations: [] ...

Merging, looks like there are no more questions left. It will unblock mt for another clean-up PR 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: align, aria, aya_vision, bart, bigbird_pegasus, blenderbot, blenderbot_small, blip_2, bloom, chameleon, codegen, colpali, colqwen2, csm, dbrx, falcon
flash_attention_3 for Qwen2_5_VisionTransformerPretrainedModel
Author: aidando73
Pull request: no
Labels: name: Feature request

### Feature request

As per title

### Motivation

Would like better performance.

### Your contribution

Let me know if you need any more details

Comments:
Qwen2-VL currently supports all attention implementations. We recently refactored the API and loading the model with `attn_implementation="flash_attention_2"` will do the work

I see the `Supports_flash_attn_3` flag was removed from all models in https://github.com/huggingface/transformers/pull/39120, i guess accidentally. I will make a PR to bring back all the flags

Ah ok - so if I just install flash_attn v3 and set the flash_attention_2, then that'll be flash attention 3.

You need to explicitly set it to `attn_implementation="flash_attention_3"`, otherwise it will be dispatching FA2

Ah ok. I guess I have to install transformers from source then. Do you know when's the next release?

I'm I'm still getting the error even on source install:

ValueError: Specified `attn_implementation="flash_attention_3"` is not supported. The only possible arguments are `attn_implementation="eager"` (manual attention implementation), `"attn_implementation=flash_attention_2"` (implementation using flash attention 2), `"attn_implementation=sdpa"` (implementation using torch.nn.functional.scaled_dot_product_attention).

Which version of transformers you have installed? FA3 was added in 4.53.0 and seems like you have prev version installed 🤔 

Ah - sorry I installed transformers into the wrong environment. Not running into the error now.
fix `phi3` tests
Author: ydshieh
Pull request: yes

# What does this PR do?

These 2 tests are never passing after (#31436) , as we didn't update the expected outputs after

Rename Phi-3 rope scaling type (#31436)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: phi3

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39312). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Handle DAC conversion when using weight_norm with newer PyTorch versions
Author: edwko
Pull request: yes

# What does this PR do?
`nn.utils.weight_norm` has been moved to `nn.utils.parametrizations.weight_norm`. As a result, parametrized layers no longer expose `weight_g` or `weight_v`, but only `weight`. This causes an AttributeError when DAC conversion attempts to access weights:
```bash
AttributeError: 'ParametrizedConv1d' object has no attribute 'weight_g'. Did you mean: 'weight'?
```
This PR moves `apply_weight_norm` directly into the script, explicitly setting to use `weight_norm = nn.utils.weight_norm` to ensure compatibility.

Comments:
cc @kamilakesbi, who made the original conversions (do they still work with the existing conversion scripts?)

Maybe related to https://github.com/huggingface/transformers/pull/33275? cc @ylacombe 

> Maybe related to #33275? cc @ylacombe

Yes, that PR updates several models including [modeling_dac.py](https://github.com/huggingface/transformers/pull/33275/files#diff-c040b6b4632a1fb0f47fd589bb16a6ff9ba3d43b5dec03c42a69306e102d58fa) to use `nn.utils.parametrizations.weight_norm` for newer torch versions. However, the DAC script still expects the deprecated weight normalization, causing it to fail.

@bot /style

Style bot fixed some files and pushed the changes.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: dac

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36393). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
FlashAttention2 ImportError: undefined symbol with flash_attn_2_cuda when loading Phi-4-Multimodal
Author: tanuj-rai
Pull request: no
Labels: name: bug

### System Info

Transformers version: 4.53.1
PyTorch version: 2.6.0+cu124
Python version: 3.11.13
CUDA version: 12.4
GPU: Tesla T4


### Who can help?

```
ImportError                               Traceback (most recent call last)
[/tmp/ipython-input-9-1994488020.py](https://localhost:8080/#) in <cell line: 0>()
      2 
      3 import torch
----> 4 from transformers import AutoModelForCausalLM, AutoProcessor
      5 import soundfile as sf
      6 

10 frames
[/usr/local/lib/python3.11/dist-packages/flash_attn/flash_attn_interface.py](https://localhost:8080/#) in <module>
     13     from .flash_attn_triton_amd import interface_fa as flash_attn_gpu
     14 else:
---> 15     import flash_attn_2_cuda as flash_attn_gpu
     16 
     17 # isort: on

ImportError: /usr/local/lib/python3.11/dist-packages/flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
`

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
import torch
from transformers import AutoProcessor, AutoModelForCausalLM
import soundfile as sf


model_name = "microsoft/Phi-4-multimodal-instruct"
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16, trust_remote_code=True)


audio, sr = sf.read("your_audio.wav") 

messages = [{
    "role": "user",
    "content": [
        {"type": "audio", "array": audio, "sampling_rate": sr},
        {"type": "text", "text": "Transcribe the audio."}
    ],
}]

inputs = processor.apply_chat_template(messages, tokenize=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200)
response = processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]

print("Model response:", response)
```

### Expected behavior

The code should work without error.

Comments:
Hi @Tanuj-rai 

This is unlikely a `transformers` but  your python environment.

You can first try

https://github.com/Dao-AILab/flash-attention/issues/853

> Hi [@Tanuj-rai](https://github.com/Tanuj-rai)
> 
> This is unlikely a `transformers` but your python environment.
> 
> You can first try
> 
> [Dao-AILab/flash-attention#853](https://github.com/Dao-AILab/flash-attention/issues/853)

Thank you @ydshieh, for helping out. It works for me. I am closing this issue.
[server] add tests and fix passing a custom `generation_config`
Author: gante
Pull request: yes

# What does this PR do?

This PR:
- Adds basic usage tests for `transformers serve`, including tool use
- Fixes the `tiny-agents` demo (new exception, I think from the new huggingface_hub version?)
- Fixes the use of a custom `GenerationConfig` in `transformers serve`
- In both `transformers serve` and `transformers chat`, ensures we start building a `GenerationConfig` from the model's default, rather than the global default. This should enable better outcomes with default parameterization.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39230). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[tests] tag serve tests as slow 
Author: gante
Pull request: yes

# What does this PR do?

These tests are not slow and don't require GPUs, but they're being flaky on the CI setup. This PR tags them as slow to unblock CI.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39343). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add a default value for `position_ids` in masking_utils
Author: Cyrilvallez
Pull request: yes
Labels: name: for patch

# What does this PR do?

As per the title. See related discussion in https://github.com/huggingface/transformers/pull/39194

Also cc @BenjaminBossan, sorry again for the inconvenience on this one!


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39310). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Indeed, the added test is unrelated, I just forgot to add on the previous PR. There are no failed tests on Transformers, issue was highlighted on the `peft` side, so merging!
Update check_modular_conversion
Author: qubvel
Pull request: yes
Labels: name: Modular

# What does this PR do?

 - Adds multiprocessing for processing modular files (with and without fix_and_overwrite flag)
 - While checking, we always should overwrite files to be sure we did not miss any conversion
 
 
 cc @ydshieh 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37456). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qubvel Thank you for working on this.

I have admitted that I have to check the sort and dependency stuff here to get better understanding before I can judge correctly.

However I have a question: since we are running with multiple processes and each would modify some modeling files (i.e. generate them from modular files), it's not very clear to me that we are free of the race condition.

Maybe this won't happen from the way we (and you) handle the dependency and the control flow, but I find it's kind difficult to see the logic clearly.

I will leave some comments and questions in the PR changes so we can discuss in more specific positions.

Thanks for the review, @ydshieh. That's definitely a fair point. I will add more comments to clarify the algorithm!

cc @Cyrilvallez as well

Will review a bit later, in the meantime maybe those comments https://github.com/huggingface/transformers/pull/35562#discussion_r1907510904 are relevant to understand how it works and how we currently (that is without multiprocessing) ensure the order is sound @ydshieh 

Thanks @Cyrilvallez . Since he knows much better on this part, a review from his side is way better than what I can offer (at this moment).

The only thing I can suggest for the review is to make sure a file can't be modified by different processes, or modified by one process but potentially read by other process, unless we have some lock implemented.

@Cyrilvallez, just a friendly reminder. The number of modular models increases day to day, and CI/local modular conversion becomes too long

<img width="1609" height="595" alt="Screenshot 2025-07-10 at 16 57 40" src="https://github.com/user-attachments/assets/f92cede2-1790-4f45-af96-f3f54a2d4e20" />


For sure, I am becoming more and more frustrated by it as well! Sorry for the delay, reviewing now!
Update modeling_glm4v.py
Author: ddeellttaa
Pull request: yes

Fix example: resolve input, image, and processor usage issues
- Changed input from list to tensor to match model requirements.
- Added image to model input to enable correct multimodal inference.
- Replaced tokenizer with processor for proper multimodal preprocessing.




Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4v
[Fix] Make EoMT compatible with pipeline
Author: yaswanth19
Pull request: yes
Labels: name: Vision

Renames arg `original_image_sizes` -> `target_sizes` and few more changes so that the model is compatible with Image segmentation Pipeline

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39122). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yonigozlan Please review the PR. 

@yonigozlan  Done with the changes, GTM 🚀 

run-slow: eomt

This comment contains run-slow, running the specified jobs: 

models: ['models/eomt']
quantizations: [] ...

Test fails on CI:

```
FAILED tests/models/eomt/test_modeling_eomt.py::EomtForUniversalSegmentationTest::test_disk_offload_bin - AssertionError: ValueError not raised
FAILED tests/models/eomt/test_modeling_eomt.py::EomtForUniversalSegmentationIntegrationTest::test_segmentation_pipeline - TypeError: list indices must be integers or slices, not str
```

Yup, also can see some failure due to device mismatch. Will try to fix it as there is a 7B model for which multi-gpu might be relevant


run-slow: eomt

This comment contains run-slow, running the specified jobs: 

models: ['models/eomt']
quantizations: [] ...

run-slow: eomt

This comment contains run-slow, running the specified jobs: 

models: ['models/eomt']
quantizations: [] ...

@qubvel Can you look into the issue. CI failed due to SHA mismatch 🤔 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: eomt

run-slow: eomt

This comment contains run-slow, running the specified jobs: 

models: ['models/eomt']
quantizations: [] ...
surpport for google/medgemma-27b-it
Author: moshilangzi
Pull request: no
Labels: name: New model

### Model description

surpport for google/medgemma-27b-it

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

surpport for google/medgemma-27b-it

Comments:
It should already be supported as it uses the gemma3 architecture, they also provide a snippet with transformers (pipeline) for generation at https://huggingface.co/google/medgemma-27b-it#how-to-use

Let me know if I'm missing something here