noamaanMulla-03/transformers-issues huggingface dataset:
special_image_mask handling can get hit by accidental same embedding value at certain dims
Author: lancercat
Pull request: no

https://github.com/huggingface/transformers/blob/5c47d08b0d6835b8d8fc1c06d9a1bc71f6e78ace/src/transformers/models/gemma3/modeling_gemma3.py#L1262

FWIS should be testing if the [whole vector] is same enough to  the special token embedding,
instead of testing same at each dim,
which should be something like 

```
            if input_ids is None:
                special_image_mask =((inputs_embeds - self.get_input_embeddings()(
                    torch.tensor(self.config.image_token_index, dtype=torch.long, device=inputs_embeds.device)
                )).abs().sum(-1)<0.009).unsqueeze(-1);
            else:
                special_image_mask = (input_ids == self.config.image_token_index).unsqueeze(-1)
            special_image_mask = special_image_mask.expand_as(inputs_embeds).to(inputs_embeds.device)

```

Comments:
@lancercat It should not be an issue if embeds are obtained using the same model with the same `dtype`. Is it failing for you at inference time?

No. 
Bcs inference takes id and thus does not compare embedding.
Doing some prefix finetuning and it suddenly became angry bcs the random embedding hits some one or two dims :)


Ah, oke, so IIUC the issue, when doing prefix tuning some virtual inputs embeds get assigned an image token idx. I still dont think this needs a fix, because comparing the diff to an arbitrary `<0.009` will lead to more issues if vocabulary has embeddings nearly identical to the image token

If you are doing prefix tuning with PEFT, we can try to fix it on PEFT side (though I think PEFT doesn't expand embeds but rather cache). If it was a custom script for tuning, I suggest to initialize virtual embeddings to be non-equal to the image token

I am already meddling with the PEFT. 
The problem is bigger than i thought due to Gemma's 4d attention mask and its right alignment magick :)
I am currently writing a dedicated PEFT class for it... 

Anyway, if the embedding comparison behaviour is not intended, maybe remove the logic? 
as the current comparison  does not seem to do what it seems to do either...

The feature works when users pass `embeds = get_input_embeddings(input_ids)` to `forward` along with the pixels. It should not be removed
Fix wording in `torchscript.md`
Author: tosroka
Pull request: yes

# What does this PR do?

This PR improves documentation by fixing a small typo.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. (doesn't apply)
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation). 
- [ ] Did you write any new necessary tests? (doesn't apply)


## Who can review?

Anybody, this is a very minor change.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38004). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Error during processing: MllamaForCausalLM does not support Flash Attention 2.0 yet.
Author: sangramddreg
Pull request: no
Labels: name: Feature request

Error during processing: MllamaForCausalLM does not support Flash Attention 2.0 yet. Please request to add support where the model is hosted, on its model hub page: https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct/discussions/new or in the Transformers GitHub repo: https://github.com/huggingface/transformers/issues/new



I wanted to use vision model for multiple pdf analysis for generate the pdf report, but model is not supporting although we can go with other model and doing same but it will be helpful if you guys will resolve this issue at your end, if possible.

Comments:
Duplicate of https://github.com/huggingface/transformers/issues/35634

Yes, would be nice to add FA2, but it is a lower priority task currently. If you have time, feel free to submit a PR :)

Hi @zucchini-nlp, I have some time, could potentially pick this up. Do you have a reference PR so I can see what exactly needs to be done?

Also is there any other recent multimodal model that supports Flash Attention 2 that you know of?

@shngt great, thanks! Your help would be very much appreciated. 

Since we are adding FA2, I'd suggest to apply new attention API which will pick up all new attention implementations by itself. For an example of new API, we can look at llama. 

The main changes are these lines and some tiny changes in the decoder layer. For cross-attention and vision attention it will basically be the same. LMK if you need more guidance and feel free to tag me in a draft PR

https://github.com/huggingface/transformers/blob/beb9b5b02246b9b7ee81ddf938f93f44cfeaad19/src/transformers/models/llama/modeling_llama.py#L289-L308

Thank you @zucchini-nlp! Will take a look and get back to you.
Flash attention 2 support for PaliGemma model
Author: Ram81
Pull request: no
Labels: name: Feature request

name: Flash Attention

### Feature request

Hi,
 
Is it possible to enable flash attention for PaliGemma models?

### Motivation

This feature is required to speed up inference using PaliGemma VLMs

### Your contribution

If someone can point me to the steps required to do this I 'll be happy to help. Is it as simple as enabling [this flag?](https://github.com/huggingface/transformers/blob/main/src/transformers/models/paligemma/modeling_paligemma.py#L195)

Comments:
I think it is already supported! Might not be properly passed to the underlying model, but Gemma2 (used by pali gemma if I am not mistaken) supports it! 🔥 

Hi @Ram81! We would appreciate your help in enabling FA2 for PaliGemma, you also have to modify 
https://github.com/huggingface/transformers/blob/e782e95e3465b66a377c0a7fe95f8f10cd87459d/src/transformers/models/paligemma/modeling_paligemma.py#L309 

Similar to 
```python
self.vision_tower = AutoModel._from_config(config.vision_config, attn_implementation=config._attn_implementation)
```

Let me work on this

Thanks!
make `test_speculative_decoding_non_distil` device-agnostic
Author: faaany
Pull request: yes

## What does this PR do?
This test is not a cuda-specific test and can work on other GPUs, e.g. on XPU it can pass. 

cc @ydshieh @yao-matrix 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38010). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add ALL_ATTENTION_FUNCTIONS compatibility for Pixtral model
Author: uminaty
Pull request: yes

### What does this PR do?

This PR adds support for `ALL_ATTENTION_FUNCTIONS` to the Pixtral model’s attention mechanism. I added and verified compatibility with `sdpa`, `flash_attention_2`, and `flex_attention`. Since Pixtral also serves as the vision tower in `Mistral 3.1`, users can now set the entire model to use `flash_attention_2`.

I tried to follow the implementation pattern of other models using this interface. For `flash_attention_2`, I reused `position_ids` because the existing attention mask shape isn’t supported. Since Pixtral uses sequence packing and already generates `position_ids`, we leverage [`prepare_fa2_from_position_ids`](https://github.com/huggingface/transformers/blob/afbc293e2b8523bfc92232d8a8d50e29bdce9f41/src/transformers/modeling_flash_attention_utils.py#L184) instead of a mask.

I tested these changes in training and inference: losses match very closely and we observe a 10–25 % throughput improvement depending on the setup.

### Who can review?
- @amyeroberts  
- @qubvel  
- @ArthurZucker


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Thanks @qubvel for the review 🙏! I made the changes you suggested, let me know if anything else is needed.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37960). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks everyone for your reviews! Let me know if anything else is needed before merging 😊

Thanks for the contrib!
Update Loss Functions to Accept Tensor num_items_in_batch
Author: iamsernine
Pull request: yes

This PR updates the ForCausalLMLoss and ForMaskedLMLoss functions to handle num_items_in_batch as a torch.Tensor instead of an int, aligning with how the parameter is actually passed during training.

Changes:

Updated num_items_in_batch: Optional[int] → Optional[torch.Tensor] in both functions.

Applied .item() conversion when computing loss to retain compatibility with fixed_cross_entropy.

Preserved all existing comments and logic for clarity and maintainability.



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
enable mamba2 integration cases on xpu
Author: yao-matrix
Pull request: yes

all 4 cases passed.

@ydshieh , pls help review, thx.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @IlyasMoutawwakil 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38006). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix: Use deep copy for bbox_embed layers when decoder_bbox_embed_share is False
Author: byteakp
Pull request: yes

# What does this PR do?

This PR fixes a bug in the `GroundingDino` model implementation related to how the `bbox_embed` layers are instantiated when `decoder_bbox_embed_share=False`.

In the previous code, the same `GroundingDinoMLPPredictionHead` instance was reused across decoder layers, resulting in a shallow copy. This behavior is unintended when `decoder_bbox_embed_share` is set to `False`, as each decoder layer should have its own unique prediction head.

The fix ensures a deep copy by creating a new instance of the `GroundingDinoMLPPredictionHead` for each decoder layer.

Fixes #37333

## Motivation

This bug prevents proper support for models like `LLMDet`, which rely on separate bbox prediction heads per decoder layer. Fixing this improves extensibility and aligns with expected behavior.

## Before submitting

- [x] I have read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request).
- [x] This PR was discussed in Issue #37333.
- [x] I’ve made the necessary code changes in `modeling_grounding_dino.py`.
- [x] This fix improves model compatibility with `LLMDet` variants.

## Who can review?

Vision models: @qubvel, @NielsRogge


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
qwen3-moe attention module is defined repeatedly.
Author: Jintao-Huang
Pull request: no


Comments:
<img width="685" alt="Image" src="https://github.com/user-attachments/assets/e5b4b8e4-fd1b-4457-9a59-a7b05646efc9" />

Thank you, @Jintao-Huang, for reporting this issue. 
Self.mlp = Qwen3MoeMLP(config) is also reassigned in the if-else block.

https://github.com/huggingface/transformers/pull/37848
enable utils test cases on XPU
Author: yao-matrix
Pull request: yes

@ydshieh @IlyasMoutawwakil , pls help review, thx.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38005). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Swinv2Model reports an error when using the parameter use_obsolute_embeddings
Author: SCP-KAKA
Pull request: no

I want to use `position_imbeddings` when creating the swinv2 model, so I set `use_absolute_embeddings` to True in the config and randomly input a data into the model to report an error:
```
File "/home/software/anaconda2019/envs/TrOCR/lib/python3.8/site-packages/transformers/models/swinv2/modeling_swinv2.py", line 362, in forward
    embeddings = embeddings + self.position_embeddings
RuntimeError: The size of tensor a (9216) must match the size of tensor b (9217) at non-singleton dimension 1
```

The reason is that in line 291 of the `modeling_Swinv2.py` file, the dimension used to create `position_imbeddings` is `num_ patches+1`, which is different from the dimension of `patch_imbeddings`. There was an error when performing the`+`operation on both(at line 362).

Is this a bug? If not, how should I correctly use position_imbeddings in the Swinv2 model.
Here is my code:
```
swinv2_config = {
    "architectures": ["Swinv2TinyLayers3"],
    "attention_probs_dropout_prob": 0.0,
    "drop_rate": 0.0,
    "depths": [2, 2, 6],
    "drop_path_rate": 0.1,
    "embed_dim": 96,
    "encoder_stride": 32,
    "hidden_act": "gelu",
    "hidden_dropout_prob": 0.0,
    "hidden_size": 384,
    "image_size": 384,
    "initializer_range": 0.02,
    "layer_norm_eps": 1e-05,
    "mlp_ratio": 4.0,
    "model_type": "swinv2",
    "num_channels": 3,
    "num_heads": [3, 6, 12],
    "num_layers": 3,
    "patch_size": 4,
    "qkv_bias": True,
    "use_absolute_embeddings": True,
    "window_size": 16,
    "patch_norm": True,
    "use_checkpoint": False,
    "pretrained_window_sizes": [0, 0, 0]
}

class CustomSwinv2Encoder(Swinv2Model):
    def __init__(self, config):
        super().__init__(config)

    def forward(self, pixel_values):
        outputs = super().forward(pixel_values)
        return outputs.last_hidden_state

swinv2_config = Swinv2Config.from_dict(swinv2_config)
encoder = CustomSwinv2Encoder(swinv2_config)

dummy_input = torch.randn(2, 3, 384, 384)
out = encoder(dummy_input)
print(out.shape)
```




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Docs: fix docstrings for Gemma3 modeling
Author: kiddj
Pull request: yes

# What does this PR do?

Fixes the example in the gemma3 docstring.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Can you fix the repo consistency? 🤗
Add support for manually setting `head_dim` in Qwen2 MoE
Author: LuHC409
Pull request: yes

# Add support for manually setting `head_dim` in Qwen2 MoE

## Problem Description

Currently, in the Qwen2 MoE model, the `head_dim` is strictly set to `hidden_size // num_attention_heads`. This PR adds support for manually setting the `head_dim`, similar to the implementation in the Llama, Mistral, and Mixtral models.

## Changes

- Added a `head_dim` parameter to `Qwen2MoeConfig`
- Modified the `Qwen2MoeAttention` class to support manually defined `head_dim`
- Added corresponding test cases to verify the functionality

## Tests

The following test cases have been added:
- `test_head_dim_manual_setting`: Tests the functionality of manually setting `head_dim`
- `test_head_dim_auto_calculation`: Tests the default automatic calculation of `head_dim`
- `test_head_dim_validation`: Tests validation logic for `head_dim`

## Related Issue

Fixes #36659

## Checklist

- [x] This PR fixes a functional issue
- [x] I have read the contribution guidelines
- [x] The feature was discussed and approved in the issue
- [x] Documentation has been updated accordingly
- [x] Necessary test cases have been added

## Reviewers

@Rocketknight1 @ArthurZucker

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
[Temporary] Log some information in some pytest/pluggy internal places
Author: ydshieh
Pull request: yes

# What does this PR do?

We get 

> Too long with no output (exceeded 10m0s): context deadline exceeded

from time to time. It's very unclear how could this happen. It might be pytest, pytest's plugin, our code, or even the `tee` command.

This time, let's monitor some progresses in pytest and see if we can get more information when this happens again.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37996). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Trigger CircleCI via GitHub Actions when `ready for review`
Author: ydshieh
Pull request: yes

# What does this PR do?

So far we are using another approach that will prevent PRs from forked repositories not trigger CircleCI when `ready for review`. This PRs try to use a GitHub Actions workflow with `pull_request_review: ready_for_review` that will trigger CircleCI pipeline.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37885). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Disable `Trigger CircleCI via GitHub Actions when `ready for review`
Author: ydshieh
Pull request: yes

# What does this PR do?

In #37885, we use a Github Action workflow file to trigger CircleCI via API. We use `pull_requesttarget` event to allow the Github Actions workflow being triggered without an approval, but this leads to the triggered CircleCI checkout to the `main` branch, which is the wrong target to test against.

This PR disable that workflow for now so we don't get wrong and confusing CI results. I will think if we have other options.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38038). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.