noamaanMulla-03/transformers-issues huggingface dataset:
Add style bot
Author: SunMarc
Pull request: yes

# What does this PR do ? 

This PR adds the style bot from `huggingface_hub` (same as in accelerate / diffusers)
To run, in a PR call @bot /style

cc @hanouticelina @Wauplin @ydshieh @ArthurZucker 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38102). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

As you already now, the CI won't be triggered if the push is from a bot. This is true if the style job is a github actions.

I never check with circleci (which is where style job runs). If it works, LGTM.

@bot /style

ah, it won't be triggered unless PR is merged ...

Actually it works quite well.

As mentioned offline, we might want to run the check under the same docker env. as used by our CircleCI. But it should not be a major issue for now.

Let's merge and update the necessary component if any.
Fix description and formatting errors in code docs
Author: bilibili12433014
Pull request: yes

# What does this PR do?
Fix description and formatting errors:

1. Fix: Two `True` exists in one conditional statement.
2. Removed line breaks to comply with the doc guideline: 'No need to indent further for the elements building the return.'

Origin:
```
    Return:
        `torch.BoolTensor`. (`torch.BoolTensor` of shape `(batch_size, 1)`), where `True` indicates we stop generation
            for a particular row, `True` indicates we should continue.
```
Fix:
```
    Return:
        `torch.BoolTensor`. (`torch.BoolTensor` of shape `(batch_size, 1)`), where `True` indicates we stop generation for a particular row, `False` indicates we should continue.
```


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@stevhliu 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

This commit appears under the generation module, but it only includes documentation updates. @stevhliu, would you mind reviewing this?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38074). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@grant I’ve pushed a new commit to keep the formatting consistent with the rest of the files. Is this correct?

Thanks a lot for the quick review and approval, really appreciate it!
while using trainer to train mnist model,  'ValueError: Found input variables with inconsistent numbers of samples: [10000, 8750]'
Author: HaoyaWHL
Pull request: no
Labels: name: bug

It was the problem with the data processing during the training process that led to the inconsistent shapes.

Comments:
hey ! in the traceback, it shows that y-pred and y-true are not the same size. You should check what you get from accuracy_score
`output_hidden_states` only return part of hidden_state when setting `device_map="auto"`
Author: sangyx
Pull request: no
Labels: name: bug

### System Info

* transformers version:  4.49.0

When I set `device_map="auto"` and let the model return hidden_state, the returned hidden_state only has value for the first n layers; for the rest, it's all 0s.

My GPUs:

<img width="952" alt="Image" src="https://github.com/user-attachments/assets/ba11d554-9716-4c9a-9f5a-cebb57ca609a" />

### Who can help?

@SunMarc 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
MODEL_PATH = "meta-llama/Llama-3.1-8B-Instruct"
llama_model = transformers.AutoModelForCausalLM.from_pretrained(
    MODEL_PATH, device_map="auto"
)
llama_tokenizer = transformers.AutoTokenizer.from_pretrained(
    MODEL_PATH, device_map="auto"
)

sentence = "xxxxx"
input_tokens = llama_tokenizer(sentence, return_tensors="pt")

# skip samples with more than 300 tokens, otherwise GPU runs out of memory
if len(input_tokens.input_ids[0]) > 300:
    continue
gen_text = llama_model.forward(
    input_tokens.input_ids, output_hidden_states=True, return_dict=True
)
hidden_states = []

# iterating over all layers and storing activations of the last token
for layer in gen_text["hidden_states"]:
    hidden_states.append(layer[0][-1].detach().cpu().numpy())

print(hidden_state)
```

### Expected behavior

I set the device to CPU, and all values can be obtained from the hidden state.

Comments:
hi @sangyx, can you confirm:

1) What hardware are you using for this?
2) Does the bug go away when you don't use `device_map="auto"`?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@Rocketknight1 I use a 4 x RTX A5000 server. I can only set the device to "CPU" if I don't use `device_mpa = "auto"` as I don't have enough single GPU memory to run the model.

Hi @sangyx I can't reproduce the bug, unfortunately! I think this may be an environment issue or a problem in the code somewhere. You can try:

1) Reducing memory with `torch_dtype=torch.bfloat16` when initializing the model and placing it on a single GPU only
2) Calling `llama_model()` directly instead of `llama_model.forward()`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[video processor] fix tests
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes a few tests that started failing after https://github.com/huggingface/transformers/pull/35206

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38104). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Potential bug in Qwen 2/2.5 VL Image Preprocessor
Author: ritwickchaudhry
Pull request: no

https://github.com/huggingface/transformers/blob/5c47d08b0d6835b8d8fc1c06d9a1bc71f6e78ace/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py#L278

The `temporal_patch_size` is used to group consecutive video frames. However, if the number of frames are not divisible, then the last frame is repeated. The current number of repetitions is `temporal_patch_size - 1`. While this will work for `temporal_patch_size = 2` but it wouldn't work for larger patch sizes.

In my opinion, the code should be modified to:
```
repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size - (patches.shape[0] % temporal_patch_size), axis=0)
```

Comments:
@ritwickchaudhry correct! There was another issue with PR somewhere (https://github.com/huggingface/transformers/pull/37350), probably got stale. I have forgot about it due to low priority. Would you like to open a PR for this?

LMK if you can't contribute, I can finalize and merge the existing PR later next week :)

Thanks @zucchini-nlp ! Sure, let me create a PR soon!

Hi @ritwickchaudhry and @zucchini-nlp,

I've also encountered this issue and have implemented the fix based on the discussion here.
@ritwickchaudhry, I saw you mentioned you'd be creating a PR. Are you still planning to, or would you mind if I submit one with the changes? 


@anshulsc I'll be releasing the PR very soon, as I finished most of it. Thanks for the offer though!

@ritwickchaudhry  great !!


Done actually! @zucchini-nlp can you please review the PR: https://github.com/huggingface/transformers/pull/38076
a logic error in _preprocess  function of Qwen2VLImageProcessor Class
Author: InsaneGe
Pull request: no
Labels: name: bug

### System Info

in the _preprocess  function of Qwen2VLImageProcessor Class(https://github.com/huggingface/transformers/blob/348f3285c5114159d2ff4933b4b8ae36866d01a7/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py#L278), it writes down as follows:
```python
if patches.shape[0] % temporal_patch_size != 0:
    repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size - 1, axis=0)
    patches = np.concatenate([patches, repeats], axis=0)
grid_t = patches.shape[0] // temporal_patch_size
```
it should repeat `temporal_patch_size - (patches.shape[0] % temporal_patch_size)` instead of temporal_patch_size - 1, to make sure patches.shape[0] can be divisible by temporal_patch_size.
```python
if patches.shape[0] % temporal_patch_size != 0:
    repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size- (patches.shape[0] % temporal_patch_size), axis=0)
    patches = np.concatenate([patches, repeats], axis=0)
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

nothing

### Expected behavior

nothing

Comments:
@InsaneGe hey, I agree it is not very generalizable and seems that the prev PR was merged to fix only certain cases for `num_frames`. Feel free to make a PR :)



Hi, I've updated the code as suggested:
```python
if patches.shape[0] % temporal_patch_size != 0:
    pad_len = temporal_patch_size - (patches.shape[0] % temporal_patch_size)
    repeats = np.repeat(patches[-1][np.newaxis], pad_len, axis=0)
    patches = np.concatenate([patches, repeats], axis=0)
```
I tested the updated code locally and confirmed that it passes all tests. The issue with incorrect padding is now resolved. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Fixed by https://github.com/huggingface/transformers/pull/38076
Fix auto batch size finder test
Author: ivarflakstad
Pull request: yes

`--auto_find_batch_size 0` has to be the last test argument because later the test does `testargs[-1] = "1"` :)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38125). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Inconsistency in installation instructions for `venv` and `uv`
Author: arjunaskykok
Pull request: no

In the README, we have installation instructions for both `venv` and `uv`.

![Image](https://github.com/user-attachments/assets/944fcbea-107e-402e-816a-1e40a145c40b)

The last step should be:

```
# venv
pip install .
```

and

```
# uv
uv pip install .
```

No?

It's a minor issue (so if you close this issue, that's fine), but the inconsistency annoys me.

Comments:
Yup, it's an inconsistency (leaving out uv in the last from source version). Would you like to open a PR for that? 

Okay, I'll do it.
[docs] add uv installation instructions for source builds
Author: arjunaskykok
Pull request: yes

# What does this PR do?

Fixes #37956

## Who can review?

@vasqu 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @Rocketknight1 since it's only a simple doc change. Wdyt?

 #37640 updated the readme as well, can you merge/rebase and apply your changes on top? @arjunaskykok Sorry about that

@vasqu , no worry. I've updated the PR.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37968). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks @arjunaskykok :hugs:
Add my username to `run_slow` whitelist
Author: manueldeprada
Pull request: yes

Per @ydshieh’s instructions, requesting approval from @gante to run slow tests :)








Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38126). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix incorrect attention mask truncate in WhisperFlashAttention2
Author: OliBomby
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes incorrect attention calculation when training Whisper with Flash Attention 2 and passing `decoder_attention_mask` with some values set to `False`.

This error might've been made when copying the same truncating code from the other attention implementations. The problem is that in `WhisperFlashAttention2` the dimensions have been transposed.

cc @sanchit-gandhi @ylacombe 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

cc @eustlb

I think this is still a pretty big bug since it highly affects the attention computations in fa2 and addresses #36585 (which has been closed under the assumption of this fix)

cc @eustlb 

It would be great to merge this fix! cc @eustlb 

Also affected by the issue, thanks for the fix @OliBomby. 

@eustlb would be really great to get this merged 🙏🏼 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36477). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
enable finegrained_fp8 and granite_speech cases on XPU
Author: yao-matrix
Pull request: yes

@ydshieh @IlyasMoutawwakil , pls help review, thx

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@ydshieh , resolved comments, thx

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38036). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Qwen3] Qwen3 MoE add tp plan for expert mlps
Author: hgt312
Pull request: yes

# What does this PR do?
Qwen3 MoE config: add tp plan for expert mlps

tested via vllm

@ArthurZucker


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38135). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Llama4TextExperts module implementation
Author: Godofnothing
Pull request: no
Labels: name: Usage

name: bug

### System Info

Llama4 model family adopts `MoE` layer implementation for better efficiency. 

However, in the current [implementation](https://github.com/huggingface/transformers/blob/d1b92369ca193da49f9f7ecd01b08ece45c2c9aa/src/transformers/models/llama4/modeling_llama4.py#L85) MoE layer in fact performs an ordinary dense FFN forward pass with all experts being involved in the computation. One can see, that `gate_up_proj` matrix has the same shape as if all `num_experts` are active.

<img width="726" alt="Image" src="https://github.com/user-attachments/assets/9cf64546-43f2-45d7-b2d1-f847f065a136" />

I guess the intent was to perform computation only for the experts **selected by router**.

### Who can help?

@ArthurZucker

### Reproduction

Any usage of the model

### Expected behavior

Only experts chosen by the router are involved in computation

Comments:
Hey! there are different formulations to MoE, we went with the one that requires a bit more memory but is more time efficient: we use `tensor parallel` to alleviate some of the problems, and we expand the inputs so that all expert view all inputs, its not highly optimized but we'll add a better implementation soon! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
transformers has no attribute TFFlorence2ForConditionalGeneration
Author: wuchaotao
Pull request: no
Labels: name: TensorFlow

How can I solve the issue: transformers has no attribute TFFlorence2ForConditionalGeneration

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
🔴 Remove head mask in generative models
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Unblocks me on https://github.com/huggingface/transformers/pull/35314#discussion_r1916314929. This PR raises error whenever one wants to use `head_mask` with `SDPA/FA2`, instead of raising a warning and redirecting to `eager`. 

Note that we didn't change vision models like ViT in this PR, those models will simply fallback to eager and have no problems because they are not generative (no cache/no attn mask)

See the linked comment for reasons why we need this

Comments:
@ArthurZucker yeah, just removing also works since no-one seems to use it anyways. I updated each model's docs with small note about head-mask and the models now silently ignore the head-mask

Maybe we can simply remove it from valid args in v5.0?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35786). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[VLMs]  support attention backends
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, another step closer to vLLM + transformers

What was done:
- Support attention API for VLM related models if not yet done
- Pass `kwargs` so vLLM can forward its attention instances
- Replace all loss computations to `self.loss_fn` (see https://github.com/huggingface/transformers/pull/36044#issuecomment-2746657112)
- Minor clean up so new models can copy prettified version, update return block with `can_return_tuple`

Fixes https://github.com/huggingface/transformers/issues/36557, https://github.com/huggingface/transformers/issues/35634, https://github.com/huggingface/transformers/issues/36904 and fixes https://github.com/huggingface/transformers/issues/33963



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37576). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qubvel if you have time to give it an initial review, while Arthur is off :)

@qubvel comments addressed. Skipped Kosmos test can be run now (discovered your `additional_model_inputs` for each tester), but it has larger diff than expected sometimes. Skipping it for now and I will investigate the source of flakiness before merging

EDIT: kosmos apparently doesn't support padding, this test is skipped in new kosmos as well

@qubvel @ArthurZucker updated the PR after the latest big refactor on VLMs. Can you review when you have time?

Yeah, agreed. We can do modular in a separate PR to not bloat up this one

completely!