noamaanMulla-03/transformers-issues huggingface dataset:
Hotfix: Flash Attention 2 support in Pixtral
Author: uminaty
Pull request: yes

## Context

Pixtral support for `ALL_ATTENTION_FUNCTIONS` was added in [this PR](https://github.com/huggingface/transformers/pull/37960), but a subsequent [rebase](https://github.com/huggingface/transformers/pull/37576) unintentionally modified a line that sets `attention_mask` to `None` when using Flash Attention 2. 

Currently, without this condition, using Flash Attention 2 with Pixtral raises the following error:

`RuntimeError: cu_seqlens_q must have shape (batch_size + 1)`

Setting `attention_mask` to `None` resolves the issue. It also appears that the current tests doesn’t catch this case.

cc: @zucchini-nlp, @ArthurZucker


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

run-slow: pixtral

This comment contains run-slow, running the specified jobs: 

models: ['models/pixtral']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38146). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[torch-xla 2.7] Change xm.xrt_world_size() to xr.world_size(). xm.get_ordinal() to xr.global_ordinal()
Author: jeffhataws
Pull request: no
Labels: name: bug

### System Info

In torch-xla version 2.7, xm.xrt_world_size() is replaced by xr.world_size() and  xm.get_ordinal() is replaced by xr.global_ordinal() (xr is "import torch_xla.runtime as xr")

https://github.com/search?q=repo%3Ahuggingface%2Ftransformers+xrt_world_size&type=code
https://github.com/search?q=repo%3Ahuggingface%2Ftransformers+get_ordinal&type=code

Currently these show up as warnings in torch-xla 2.5/2.6:

```
WARNING:root:torch_xla.core.xla_model.xrt_world_size() will be removed in release 2.7. is deprecated. Use torch_xla.runtime.world_size instead.            
WARNING:root:torch_xla.core.xla_model.xla_model.get_ordinal() will be removed in release 2.7. is deprecated. Use torch_xla.runtime.global_ordinal instead. 
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Run any finetuning/training test on Neuron or TPU
2. Look for warnings like below: WARNING:root:torch_xla.core.xla_model.xrt_world_size() will be removed in release 2.7. is deprecated. Use torch_xla.runtime.world_size instead.            
WARNING:root:torch_xla.core.xla_model.xla_model.get_ordinal() will be removed in release 2.7. is deprecated. Use torch_xla.runtime.global_ordinal instead. 

### Expected behavior

No warnings.

With torch-xla 2.7, you will see errors.

Comments:
This is in trainer so cc @muellerzr @SunMarc - I think we can just search-replace with the new methods?

Yeah, we are replacing them in accelerate. I'll ping the author to do the same in transformers

2.7 is being released. Any updates on this topic?


See the above PR ! 

 @muellerzr @SunMarc you mentioned that accelerate is being worked on for this issue. Can you provide the PR for accelerate also?

Found it https://github.com/huggingface/accelerate/pull/3484 

Hi @muellerzr @SunMarc  do you know when will the next release with this (and accelerate) change happen?

There will be a new transformers release very soon ! As for accelerate, we released today !
How to check the number of tokens processed or the load of each expert in the Qwen3 MoE model during inference?
Author: wumaotegan
Pull request: no


Comments:
Hi @wumaotegan! This isn't really a bug, but we're launching a new community discussion space for questions like this. Try posting it there and members of the community and the Hugging Face team should hopefully be able to help! https://huggingface.co/spaces/transformers-community/support
Omit creation of positional IDs within ESM if applicable
Author: simonlevine
Pull request: yes


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @Rocketknight1 , I think all tests are good though I see `build / build_pr_documentation` is still "Expected". Let me know if there's anything you need on my end. Thanks!

@simonlevine yep, that one requires maintainer approval. I'll run it now and merge once it passes!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38089). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Multiple processor classes have input side-effects
Author: nph4rd
Pull request: no
Labels: name: bug

Multiple processor classes mutate their `text` input when it's a list.

Example:

https://github.com/huggingface/transformers/blob/42c489f2ae738a3b690bb90aab274f02ff024795/src/transformers/models/qwen2_5_vl/processing_qwen2_5_vl.py#L156C21-L156C25

This results in unwanted downstream behaviour. For example, see [this comment](https://github.com/huggingface/trl/pull/3072#issuecomment-2741246702).

This behaviour shouldn't be handled neither by TRL nor vLLM, in my opinion.

### Who can help?

@ArthurZucker @qubvel 


### Reproduction

The case of Qwen2.5-VL can be tested [here](https://gist.github.com/nph4rd/f003323ac4c8940f779f44a24b815ff7).


### Expected behavior

Ideally the function should have no input side-effects.

Comments:
Opened https://github.com/huggingface/transformers/pull/36866 

I believe it addresses all processor classes with that pattern.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@molbap / @ArthurZucker / @qubvel - any thoughts?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Fixed in https://github.com/huggingface/transformers/pull/37342
[Examples] Add advanced GPT2 text generation with temperature and top_k
Author: sakthitharani
Pull request: yes

## Summary
This Pull Request adds a runnable example to the `examples/pytorch/text-generation/` directory demonstrating how to control text generation behavior in GPT2 using:
- `temperature` for randomness control
- `top_k` for limiting token choices
- `num_return_sequences` for generating multiple outputs

## Why This Is Useful
Many beginners may not fully understand how these parameters impact the quality and variety of generated text. This example shows:
- How **low temperature** results in more **predictable, repetitive outputs**.
- How **higher temperature** leads to **more creative, diverse outputs**.
- How **top_k sampling** controls the token selection range.

## How to Run
```bash
python examples/pytorch/text-generation/gpt2_advanced_usage.py


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Thanks for the PR, but we generally don't add very small examples like this! This kind of thing is covered by docstrings.
[WIP] Add flex attention for gpt2
Author: mayankagarwals
Pull request: yes

# What does this PR do?

Adding flex_attention for Gpt2 model following https://github.com/huggingface/transformers/issues/34809

## Who can review?




Comments:
cc @arthurzucker for review, but in the meantime @mayankagarwals you can make the tests pass by doing `pip install transformers[quality]` and then `make fixup` in the repo directory

Hi @Rocketknight1 

Thanks a ton, got it. 

Waiting for discussions to close here https://github.com/huggingface/transformers/pull/34896. Thinking that PR should follow similar guidelines to keep one standard design of implementation of generic attention block. 

@vasqu Let me know once major design bits are closed! I'm following your PR for gpt neox and aligned with everything there. 





@mayankagarwals Sure I'll let you know when stuff clears up!

Having a look at your PR @vasqu, I think quite a few things were fixed since so will check! 

@mayankagarwals #34896 is now merged, so you can follow conventions made there 

Closing this as https://github.com/huggingface/transformers/pull/35235/commits/5639b81a5d22e5907cee4179ac72233798c3a271 solves this issue by `ALL_ATTENTION_FUNCTIONS` refactor. 

Kudos folks!
enable trainer test cases on xpu
Author: yao-matrix
Pull request: yes


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@ydshieh @IlyasMoutawwakil , pls help review, thx.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38138). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
enable csm integration cases on xpu, all passed
Author: yao-matrix
Pull request: yes

@ydshieh @IlyasMoutawwakil , pls help review, thx.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
[don't merge]
Author: ydshieh
Pull request: yes

# What does this PR do?

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38169). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[don't merge]
Author: ydshieh2
Pull request: yes

# What does this PR do?

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Fix ModuleNotFoundError torchao.prototype.low_bit_optim since torchao  > 0.11.0
Author: baptxste
Pull request: yes


# What does this PR do?

Update import path of AdamW4bit and AdamW8bit for torchao > 0.11.0


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Include output embedding as well with `include_embedding` flag
Author: jerryzh168
Pull request: yes

Summary:
att

Test Plan:
python tests/quantization/torchao_integration/test_torchao.py -k test_include_input_output_embeddings

Reviewers:

Subscribers:

Tasks:

Tags:


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @MekkCyber @SunMarc please take a look, just a small change to include_embedding

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37935). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@MekkCyber @SunMarc can you merge this

Done ! sorry for the delay
Allow qwen/emu3 to process low res images
Author: qubvel
Pull request: yes

# What does this PR do?

Allow to process images with low resolution, found this limitation while evaluating model on the MMMU-pro benchmark

Reroduction:
```python
from PIL import Image
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

model = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct", torch_dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

image = Image.new("RGB", (25, 100), color="red")
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": image,
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=image, return_tensors="pt").to("cuda")

generated_ids = model.generate(**inputs, max_new_tokens=128)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text)
```

Gives the following error on main:
```
...
 line 432, in preprocess
    patches, image_grid_thw = self._preprocess(
  File "/home/ubuntu/projects/transformers/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 251, in _preprocess
    resized_height, resized_width = smart_resize(
  File "/home/ubuntu/projects/transformers/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 68, in smart_resize
    raise ValueError(f"height:{height} and width:{width} must be larger than factor:{factor}")
ValueError: height:100 and width:25 must be larger than factor:28
```



Comments:
cc @zucchini-nlp 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38150). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[VLMs] add helpers to get multimodal encodings
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, makes it available in all VLMs where it was not added. This PR is another small part of vLLM integration for multimodality


Would be nice to update a few audio LLMs we have, but qwen2-audio is a mess and we can update it only after cleaning up deprecations. Granite Speech is already updated, cool

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37743). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker ready for review. Rebased main after recent major changes

Merging this so I can get to work on processors and finalize it by today, or at least by Monday
Bart: new cache format
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, enables new cache format in Bart ans several models copied from Bart. Since there are too many models copying attention from Bart, I decided to not touch the audio ones and changed their "Copied from" statements

TODO:
- [x] Run all tests for models with new cache + slow ✅ 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35314). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @BenjaminBossan I am running slow tests from transformers side and the current state of PR should be almost ready for review. So we might need to run PEFT tests now 

> I am running slow tests from transformers side and the current state of PR should be almost ready for review. So we might need to run PEFT tests now

Thanks for the ping. I skimmed the PR and if I'm not mistaken, of all the models that were changed, Bart is the only one that is covered in the PEFT test suite. Therefore, running tests with `-k bart` should be sufficient. LMK if you run the tests, otherwise I can get to it later.

Cool, the code owners tagged all relevant people. Ready for review!

Slow tests for text models that now support cache class are passing on my end 

@zucchini-nlp  
I am curious about the status of this PR. Could you update it?

@ducviet00 sorry but the PR is blocked by another one.These got stale a bit since we had some higher priority releases recently. I will be back on Bart from next week and get it merged, thanks

Hi @zucchini-nlp 
Can we have bart with new cache format, it will boost performance much when using with `torch.compile` 

It is currently blocked by another PR (https://github.com/huggingface/transformers/pull/35786). cc @ArthurZucker can you review it again please?

awesome @zucchini-nlp  thank you so much 


cc @BenjaminBossan, totally forgot to ping you. Bart is basically same as T5, hope it won't cause red CI on peft 😅
Disable `Trigger CircleCI by ready for review`
Author: ydshieh
Pull request: yes

# What does this PR do?

Disable `Trigger CircleCI by ready for review`

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38171). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
When I  using pipeline to inference the valid dataset, I got the different results with evaluation result during training!
Author: kingman1980
Pull request: no

I am working with three classes image classification task.
I got the confusion matrix at 30th epoch during training like this confusion matrix: 
[[230, 0, 1], [1, 28, 2], [1, 10, 3727]]
And I used the pipeline with the model's checkpoint at 30th epoch to infer the same test dataset, I got the confusion matrix like this:
[[230, 1, 0], [0, 31, 0], [8, 166, 3564]]

the model is timm/mobilenetv3_small_100.lamb_in1k
Anybody knows why?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @kingman1980, this probably isn't a bug, but we have a new space where you can ask questions like this and get support from other users and the Transformers team at Hugging Face! https://huggingface.co/spaces/transformers-community/support
Disable `convert to draft` workflow
Author: ydshieh
Pull request: yes

# What does this PR do?

It causes too much issues and confusion. There is no working solution for now. I will contact CircleCI.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38177). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
remove some commands from `fetch_tests` CircleCI job
Author: ydshieh
Pull request: yes

# What does this PR do?

`git log -n 1` sometimes get 

> Too long with no output (exceeded 10m0s): context deadline exceeded

Those commands are only to show some information when working on some changes, we don't really need them.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38176). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Feat: add warnings for unused keys and rules in tensor parallel
Author: S1ro1
Pull request: yes

# What does this PR do?

Implements extra warnings when tensor parallelism was applied on the model. To be exact it
prints out unused rules in the tp_plan and layers that were not sharded by tp.
First from the serie of PRs making TP more user friendly.


cc @ArthurZucker 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?





Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37893). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Should be good to re-review and merge now.