noamaanMulla-03/transformers-issues huggingface dataset:
Fix rope_deltas corruption in Qwen2.5VL during CFG generation
Author: notkisk
Pull request: yes

## Summary

  Fixes #39749 
 

  This PR addresses a critical issue where the `rope_deltas` attribute in Qwen2.5VL models gets corrupted during Classifier-Free Guidance (CFG) generation due to shared mutable state between forward passes.

  **Problem**: During CFG generation, the model performs two forward passes (conditional and unconditional). The `rope_deltas` state was being modified during the first pass and incorrectly reused in the second pass, leading to position embedding corruption and incorrect generation
  results.

  **Solution**: Implemented proper state management by:
  - Adding `_update_model_kwargs_for_generation` method to preserve `rope_deltas` in model kwargs
  - Modifying `prepare_inputs_for_generation` to restore `rope_deltas` from kwargs when available
  - Ensuring each forward pass gets the correct `rope_deltas` state

  **Impact**: Fixes CFG generation for Qwen2.5VL models, ensuring correct position calculations and generation quality.

  ## Changes Made

  - Modified `src/transformers/models/qwen2_5_vl/modular_qwen2_5_vl.py` with the fix
  - Regenerated `src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py` from the modular source
  - Added comprehensive state management for `rope_deltas` during generation

  ## Test Plan

  - [x] Verified the fix addresses the original issue reproduction case
  - [x] Tested CFG generation with multiple forward passes
  - [x] Confirmed `rope_deltas` state is properly preserved and restored
  - [x] Ran quality checks and style formatting
  - [x] Ensured backward compatibility with existing functionality

  The fix is ready and properly references issue #39749 as requested. The branch contains all the necessary changes to resolve the rope_deltas corruption issue during CFG generation.

Comments:
@Rocketknight1 @gante 

@sgugger @thomwolf 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, deepseek_vl, deepseek_vl_hybrid, efficientloftr, glm4v, janus, lightglue
Set `torch.backends.cudnn.allow_tf32 = False` for CI
Author: ydshieh
Pull request: yes

# What does this PR do?

See https://github.com/pytorch/pytorch/issues/157274#issuecomment-3090791615

(if we don't do this, we have to increase the thresholds in many tests when torch 2.8 is released) 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39885). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Better return type hint for `AutoModelForCausalLM` and `AutoModelForImageTextToText`
Author: qubvel
Pull request: yes
Labels: name: typing

# What does this PR do?

Defines a better return type hint for `AutoModelForCausalLM` and `AutoModelForImageTextToText`, from_pretrained method returns `_BaseModelWithGenerate` so anyone can see `generate` signature (see images)

#### On main
<img width="757" height="204" alt="Screenshot 2025-08-04 at 12 12 35" src="https://github.com/user-attachments/assets/1cfb7abb-9a82-4521-843f-0a7bec15d8ef" />

#### On branch
<img width="738" height="456" alt="Screenshot 2025-08-04 at 12 12 10" src="https://github.com/user-attachments/assets/8e8f9138-7dd9-4333-995d-88baceb671a1" />




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39881). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@molbap, thanks for looking into it! The `... | None` is out of scope for this PR because the PR simply exposes `GenerationMixin` methods. In my opinion, removing `None` from the signature could be messy, as this might mislead users into thinking that arguments are required, when in fact, they are not and can be omitted while calling generate.

P.S. for readability, I would advise IDE devs to allow another signature formatting, e.g. like in Python: one argument per line 😄 

Updating the branch with last main should solve the checks!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto
[modular] small fixes
Author: MHRDYN7
Pull request: yes

Some small fixes:

1. If there are no imports from the modeling files, an exception is raised https://github.com/huggingface/transformers/blob/ff76b6a07fb0d16ee9b31b5f1cd7ddb5f02b6774/utils/create_dependency_mapping.py#L47-L58 Otherwise, the users currently see the assert message only, which is not helpful https://github.com/huggingface/transformers/blob/f90de364c2484c7c325bbe05befdcf487bd75b63/utils/modular_model_converter.py#L1774

2. The cli command for converting the modular files should have the name of the file not the path as per the code in modular_model_converter.py . 

Pinging @Cyrilvallez, for being the last editor of these parts of code. 

Comments:
@gante @Cyrilvallez 

Hey @MHRDYN7! Could you expand on why this is needed? It should never happen, and even if it happens it should not be an issue 🤗

@Cyrilvallez for change 1. I was trying contribute a new model using the modular file and for that I wanted to check the complete code initially without inheriting any of the classes from existing models. That's when I kept running into that little assert message "Some files will not be converted".  Well, then I learned the hard way that having any random import from modeling files is good enough for me to start using the new modular file.  The exception message added should help contributors who are new to the modular format.

For change 2. I think I ran into an error by passing the relative path of the new modular file as the arg following the modular docs. The code does seem to expect the name only (if I'm not wrong)
https://github.com/huggingface/transformers/blob/ff76b6a07fb0d16ee9b31b5f1cd7ddb5f02b6774/utils/modular_model_converter.py#L1760-L1763

Hey! Not sure we need the change as modular file without inheriting is simply full code, i.e. a modeling file!
As for 2., you can use either one, so it's not wrong either!

Does that make sense?

alright, I checked 2. again and the code is fine for both relative path and model name. Probably the issue earlier was due to 1. or a typo somewhere.  Anyway closing the pr. 1. is surely optional, but you may certainly consider making the change in the doc for 2. so that new users use the model name instead of the path which is inconvenient.

Indeed we could change it! Thanks!
[DOCS] : Improved mimi model card
Author: rohitthewanderer
Pull request: yes
Labels: name: Audio

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs 
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. https://github.com/huggingface/transformers/issues/36979
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ebezzam 

Comments:
Thanks @rohitthewanderer ! Requesting review from @stevhliu as he's taking care of this push to standardize docs!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39824). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update cohere2 vision test
Author: ydshieh
Pull request: yes

# What does this PR do?

Just use dummy model (4 LLM layers), otherwise it won't fit into A10 runner.
The output is still deterministic, just some checkpoint weights are not used in the dummy model (all of its weights are loaded from checkpoint)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39888). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: cohere2_vision
send some feedback when manually building doc via comment
Author: ydshieh
Pull request: yes

# What does this PR do?

Something like this

https://github.com/ydshieh/transformers/pull/15#issuecomment-3151095765

The changes are mostly copied from some jobs defined in

.github/workflows/self-comment-ci.yml

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39889). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
chore: update DETR model card
Author: arpon-kapuria
Pull request: yes

# What does this PR do?

This PR updates the model card for DETR, following the template outlined in the issue.


## Before submitting
- [x] This PR improves the docs.


## Who can review?
@stevhliu 


Comments:
@stevhliu Changes made as suggested. Let me know if anything else is needed.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39822). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Served models handle with nested content
Author: jakeret
Pull request: yes

# What does this PR do?
This PR fixes an issue in the `transformers serve` functionality where the server could not handle chat messages with nested content (e.g., when the `content` field in a message was a list instead of a string or dictionary). The unhandled `TypeError` caused the server to crash instead of processing the request or returning a meaningful error message.

<!-- Remove if not applicable -->

Fixes  https://github.com/huggingface/transformers/issues/39791

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed.


Comments:
@jakeret thank you for opening the PR (and sorry for not noticing it before opening #39829 !)

I think #39829 cover more cases (a list with multiple items in `content`), but let me know if you think it misses some functionality :)

@gante no worries. In my tests with gradio I didn't encounter the case where the list contained more than one item. Thats why I only fetched the first item.
Could it be that #39829 now misses one of the original cases where `content=message["content"]["text"]` ?

@jakeret that case should be covered (and has tests 👍 )

@gante I think it's sill missing the case where `message["content"]` contains a dict.

I've pushed a new commit to this PR, that handles and tests. all cases
[`Exaone4`] Fixes the attn implementation! 
Author: ArthurZucker
Pull request: yes

# What does this PR do?
There was a typo in the config!
Should superseed #39698

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: exaone4

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39906). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
facebook/dinov2-with-registers-giant does not appear to have a file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt or flax_model.msgpack.
Author: MengHao666
Pull request: no
Labels: name: bug

### System Info

linux, latest transformer version: transformers-4.53.0

### Who can help?

raceback (most recent call last):
  File "/home/momiao.mh/Projects/debug.py", line 47, in <module>
    model = AutoModel.from_pretrained(model_name)
  File "/home/momiao.mh/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 600, in from_pretrained
    return model_class.from_pretrained(
  File "/home/momiao.mh/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 311, in _wrapper
    return func(*args, **kwargs)
  File "/home/momiao.mh/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4674, in from_pretrained
    checkpoint_files, sharded_metadata = _get_resolved_checkpoint_files(
  File "/home/momiao.mh/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1243, in _get_resolved_checkpoint_files
    raise OSError(
OSError: facebook/dinov2-with-registers-giant does not appear to have a file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt or flax_model.msgpack.

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoImageProcessor, AutoModel
from PIL import Image
import requests

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)
print(image.size)

model_name = 'facebook/dinov2-with-registers-giant'
processor = AutoImageProcessor.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

inputs = processor(images=image, return_tensors="pt")
print(inputs["pixel_values"].shape)
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
print(last_hidden_states.shape)

### Expected behavior

none

Comments:
Hi @MengHao666, that code works for me, and the [model repo](https://huggingface.co/facebook/dinov2-with-registers-giant/tree/main) has a `model.safetensors` file! I'm not sure why you're getting that error.

Is it possible you have a local directory or an out-of-date cache somehow?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Inefficient memory resharding in attention layer
Author: null-pointer-access
Pull request: no

While analyzing the self-attention implementation in HuggingFace Transformers and comparing it to vLLM, I noticed that after the KQV projection step, the model performs a memory resharding process involving three separate Tensor::contiguous() calls to align with the memory layout of the attention kernel. This introduces significant overhead especially under small batch sizes, where kernel launch and memory movement become relatively more expensive.

In contrast, replacing the multiple contiguous() calls with a single fused operator could significantly reduce latency and improve runtime efficiency. This behavior is easy to reproduce by inspecting the self-attention forward pass on a small batch size (e.g., batch=1, seqlen=128, GPT2 model).

```python
# models/gpt2/modeling_gpt2.py:L298
query_states = query_states.view(shape_q).transpose(1, 2)
key_states = key_states.view(shape_kv).transpose(1, 2)
value_states = value_states.view(shape_kv).transpose(1, 2)
```

Comments:
Hey! thanks for reporting, I am super interested if you have a solution that does not involve making weight changes! 
This is something we are aware (fused vs unfused). 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Refactor StableLM using modular
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Refactor StableLM to remove all the copied-from and use modular instead.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34013). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Refactor roberta with modular
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Let's use modular more and more and get rid of the copied from.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34565). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove unnecessary CUDA sync in qwen2_5_vl
Author: cyyever
Pull request: yes

# What does this PR do?
Remove a sync point in qwen2.5 vl training. See its overhead collected in my SFT using py-spy (other lines are omitted):
```
 %Own   %Total  OwnTime  TotalTime  Function (filename:line)
 27.00%  27.00%   16.44s    16.44s   forward (transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py:248)
```
 

Comments:
run-slow: glm4v, qwen2_5_vl

This comment contains run-slow, running the specified jobs: 

models: ['models/glm4v', 'models/qwen2_5_vl']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39870). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qubval Unrelated CI failures.

run-slow: glm4v, qwen2_5_omni, qwen2_5_vl, qwen2_vl

This comment contains run-slow, running the specified jobs: 

models: ['models/glm4v', 'models/qwen2_5_omni', 'models/qwen2_5_vl', 'models/qwen2_vl']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4v, qwen2_5_omni, qwen2_5_vl, qwen2_vl

run-slow: glm4v, qwen2_5_omni, qwen2_5_vl, qwen2_vl
Avoid aliasing in cond's branches for torch 2.8
Author: ydwu4
Pull request: yes

torch 2.8 enforces a stricter check for the aliasing relationship for correctness reasons. Fixes https://github.com/pytorch/pytorch/issues/158375 by cloning the outputs that aliasing inputs.


cc @ydshieh


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39488). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove super call from EncoderDecoderCache init
Author: manueldeprada
Pull request: yes

`EncoderDecoderCache` is a wrapper around two caches. Thus, it should not call super() to initialize `self.layers` or `self.layer_classes`.

This can create bugs since other parts of the code might try to access those attributes.

The call was added on https://github.com/huggingface/transformers/pull/39590

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39786). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Already part of https://github.com/huggingface/transformers/pull/39797! 🤗
enable GraniteMoeHybridIntegrationTest in UT
Author: yao-matrix
Pull request: no
Labels: name: bug

### System Info

N/A

### Who can help?

@ydshieh 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Currently [GraniteMoeHybridIntegrationTest](https://github.com/huggingface/transformers/blob/main/tests/models/granitemoehybrid/test_modeling_granitemoehybrid.py#L107) is skipped, yet seems there is a ["ibm-granite/granite-4.0-tiny-preview"](https://huggingface.co/ibm-granite/granite-4.0-tiny-preview) rlsed, do we wait for "ibm-granite/granite-4.0-tiny" or go ahead w/ "ibm-granite/granite-4.0-tiny-preview"?

### Expected behavior

enabled

Comments:
Thank you @yao-matrix ! Well well, the best person for this question is @alex-jw-brooks I guess!

kindly ping  @@alex-jw-brooks again

Hi @ydshieh @yao-matrix thanks for reminder, and sorry for the late follow-up. Yes, we can use the tiny preview model for this - I'll open a PR!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix CI: Tests failing on CPU due to `torch.device('cpu').index` being None
Author: manueldeprada
Pull request: yes

`torch.device('cpu').index` is None, hence `int(routing_weights.device.index)` fails on tests being run on CPU, see example [here](https://app.circleci.com/pipelines/github/huggingface/transformers/140894/workflows/8e8620a9-fc6b-45b9-b5bd-0eb23296b587/jobs/1865476)



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: granitemoe, granitemoehybrid, granitemoeshared, jamba, jetmoe, olmoe, phimoe, qwen2_moe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39933). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Merging it to help CI!
Add pytest marker: `torch_compile_test` and `torch_export_test`
Author: ydshieh
Pull request: yes

# What does this PR do?

The torch team considers to run some transformers tests on their side, especially the compile and export tests. They request us to provide a way to easily and reliably running those related tests. 

Therefore this PR adds 2 new pytest marker and use them to the relevant tests.

A run using `torch_compile_test` is

https://github.com/huggingface/transformers/actions/runs/16751561312/job/47423145415


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: albert, aria, beit, bert, clip, cohere2, colpali, colqwen2, data2vec, deepseek_v2, depth_anything, depth_pro, distilbert, dots1, dpt, exaone4
circleci: pin torch 2.7.1 until `torchcodec` is updated
Author: ydshieh
Pull request: yes

# What does this PR do?

to make CircleCI ✅ 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39951). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.