noamaanMulla-03/transformers-issues huggingface dataset:
No repeat kv
Author: Rocketknight1
Pull request: yes

Draft PR for now, please don't merge!

The goal of this PR is to experiment with an `einsum` approach for `eager_attention_forward` that avoids a lot of reshapes and transposes that introduce intermediate tensors and unnecessary copies. In notebook benchmarks it seems to be 25-50% faster, but I'm making a proper PR so I can test more thoroughly.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39402). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Fyi, I think #36218 also tried to not materialize the kv repetitions; you could take a look at https://github.com/mseeger/transformers/blob/b49651192f55ce86981c5851aaf9afc209ea3407/src/transformers/models/llama/modeling_llama.py#L138-L192 for reference / comparison w/o using einsum.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: apertus, arcee, aria, bamba, bitnet, blt, chameleon, cohere, cohere2, csm, cwm, dbrx, deepseek_v2, deepseek_v3, dia, doge

Admitting defeat on this one - on the latest version of `torch` the compiler can recognize our existing code as SDPA and automatically lowers it to `torch.ops.aten._scaled_dot_product_flash_attention.default`, which makes it much faster + more memory efficient. However, it can't recognize the version with `einsum`, and so we take a big penalty. Although the code in this PR is better in the eager case or when we need to return attentions, the `torch.compile()` case takes precedence!
Fix validation checks order in benchmark_v2
Author: Abdennacer-Badaoui
Pull request: yes

- Reorder the validation checks in the benchmarker to avoid TypeError when required arguments are missing.
- Fix `adapt_configs()` preserving old benchmark names (e.g., s128_n128) when parameters are changed via command-line arguments. Remove the name field from the config dict so it gets re-inferred with the updated sequence length and token generation values. 

----
@remi-or 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42280). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update torchcodec to match torchaudio version
Author: remi-or
Pull request: yes

There is an issue in the latest AMD container: along with ROCm, `torch` was updated, so the `torchcodec` version does not match anymore. This PR fixes this, which in turn fixes `whisper` tests, and I imagine other audio model's tests.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42288). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> (Just FYI: if you want to update to torch 2.9.1 )

No ROCm container for that 2.9.1 for now sadly :( we can try a manual update though
Use `torch.get_autocast_dtype` instead of `torch.get_autocast_gpu_dtype`
Author: qgallouedec
Pull request: yes

Fixes the warning 

```
  /fsx/qgallouedec/miniconda3/envs/trl/lib/python3.12/site-packages/transformers/integrations/flash_attention.py:54: DeprecationWarning: torch.get_autocast_dtype('cuda') is deprecated. Please use torch.get_autocast_dtype('cuda') instead. (Triggered internally at /pytorch/torch/csrc/autograd/init.cpp:852.)
    target_dtype = torch.get_autocast_dtype('cuda')
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42055). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: diffllama, esm, falcon, gpt_neo, gptj, kyutai_speech_to_text, mimi, moshi
Fix device_map computation part 2
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Follow up to https://github.com/huggingface/transformers/pull/42264.
Since https://github.com/huggingface/transformers/pull/42043, `compute_module_sizes` is also used in `_init_infer_auto_device_map`. However, there it expects to have all the modules inside, but also the leaves parameters and buffers, which was not the case. Because we return `defaultdict` instance, we would not have any `KeyError` but instead very nasty silent bugs because the values found would always be 0 (the default of the defaultdicts.

This PR fixes it by adding the option for adding them in the dict. Doing `compute_module_sizes(model, only_modules=False)` is now equivalent to `accelerate.compute_module_sizes(model)`, which is not the case without the flag (leaves are not in the dict)

cc @SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42290). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fixed the docstring for `WhisperFeatureExtractor`
Author: TopCoder2K
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

Fixes https://github.com/huggingface/transformers/issues/42180

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu @Rocketknight1

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: whisper

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42286). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
The `padding` argument is missing from the docstring for `WhisperFeatureExtractor`
Author: TopCoder2K
Pull request: no

Link: [https://huggingface.co/docs/transformers/model_doc/whisper?usage=AutoModel#transformers.WhisperFeatureExtractor.__call__](https://huggingface.co/docs/transformers/model_doc/whisper?usage=AutoModel#transformers.WhisperFeatureExtractor.__call__)
It took me some time to figure out that there was a `padding` argument because I only looked at the docstring. Also, the order of the descriptions doesn’t correspond to that of the arguments.

Comments:
Theoretically, I can update it myself. Would something like "the padding strategy (see [PaddingStrategy](https://github.com/huggingface/transformers/blob/6408d3b01a4b5446655c222daa0b947bddf7db82/src/transformers/utils/generic.py#L412))" be appropriate?

Yes, or you could copy the description of that arg from tokenizers and just tweak it a bit!

@Rocketknight1, good, I'll try it on the weekend. I'll keep the issue open until the corresponding PR is merged.

UPD from 11-17-2025:
I've already started. I'm planning to finish this week.

@Rocketknight1, I also noticed that [this](https://github.com/huggingface/transformers/blob/8cb5963cc22174954e7dca2c0a3320b7dc2f4edc/src/transformers/models/whisper/generation_whisper.py#L426) link doesn't work:
<img width="1512" height="66" alt="Image" src="https://github.com/user-attachments/assets/eb5f6551-2f80-4d70-bac5-df5d8972387d" />
after clicking:
<img width="845" height="408" alt="Image" src="https://github.com/user-attachments/assets/c995d139-f00b-42bd-93cd-c83b87e756b9" />
How can I also update this?

@TopCoder2K thanks for spotting it! If you have time, you can make a separate PR to Transformers to update that docstring link too

Actually, @omkar-334 is handling the docstring PR at #42252!
SpeechT5 RelativePositionalEncoding can create empty tensors
Author: ppadjinTT
Pull request: no

## Overview

This is the definition of the `SpeechT5RelativePositionalEncoding` class in `transformers/src/transformers/models/speecht5/modeling_speecht5.py` file:

```
class SpeechT5RelativePositionalEncoding(torch.nn.Module):
    def __init__(self, dim, max_length=1000):
        super().__init__()
        self.dim = dim
        self.max_length = max_length
        self.pe_k = torch.nn.Embedding(2 * max_length, dim)

    def forward(self, hidden_states):
        seq_len = hidden_states.shape[1]
        pos_seq = torch.arange(0, seq_len).to(device=hidden_states.device, dtype=torch.long)
        pos_seq = pos_seq[:, None] - pos_seq[None, :]

        pos_seq[pos_seq < -self.max_length] = -self.max_length
        pos_seq[pos_seq >= self.max_length] = self.max_length - 1
        pos_seq = pos_seq + self.max_length

        return self.pe_k(pos_seq)
```

In the forward pass, lines that can create empty tensors are the ones using advanced conditional indexing:
```
        pos_seq[pos_seq < -self.max_length] = -self.max_length
        pos_seq[pos_seq >= self.max_length] = self.max_length - 1
```

While this works with `torch` with 'cuda', this can be problematic for other AI accelerators like Tenstorrent AI accelerator chips.

## Proposed solution

This issue can be easily avoided by using the equivalent `torch.where` syntax that yields the same results, but doesn't result in empty tensors. The above problematic lines can be switched with:

```
        pos_seq = torch.where(pos_seq < -self.max_length, -self.max_length, pos_seq)
        pos_seq = torch.where(pos_seq >= self.max_length, self.max_length - 1, pos_seq)
```

Comments:
fix bug for `_load_pretrained_model` on windows
Author: kaixuanliu
Pull request: yes

When I use transformers built from source on windows, I meet several errors:
1. Existing code use / (forward slash) for splitting paths with rsplit("/", 1), but on Windows, the paths might use \ (backslash). When the path contains backslashes, this causes incorrect path joining
2. On Windows with symbolic links, checkpoint_files[0] might be a symlink, and when you get its dirname and join it with another file, it will create malformed paths
3. we need to load shards sequentially for limited gpu memory, and also, on windows, the virtual address for memory mapping is limited, current implementation will casue error `OSError: The paging file is too small for this operation to complete`
This PR tries to solve the 3 errors mentioned above.
@CyrilVallez pls help review, thx!

Comments:
@Cyrilvallez it's your code but I know the core maintainers are busy, let me know if you want me to take this one instead!

Hey! I removed this code this week, so the issue should not happen anymore on `main`! Let me know if you still encounter it though! In which case feel free to reopen the PR or create a new one!
ENH: Add support for LoRA hotswapping
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

LoRA hotswapping has been available in PEFT since 0.15.0. There is already a diffusers
integration (https://github.com/huggingface/diffusers/pull/9453), but the transformers integration was still missing this feature. This PR remedies this. It sticks closely to the diffusers PR, both implementation-wise and API-wise. 

Hotswapping allows to swap different LoRA adapters in-place instead of loading multiple adapters and switching between them. Not only can this be advantageous to safe memory and potentially for quicker loading, the biggest advantage is that if the model is compiled, we can hotswap without triggering recompilation (loading a separate adapter would require recompilation).

There are some caveats to using this feature, most notably that only LoRA is supported. This was fine for diffusers, as it only works with LoRA, but the transformers integration works with other PEFT methods too. However, LoRA should be by far the most common method, so this should be fine for now. This and other caveats have been documented.

Note that testing is not super deep, there could be more edge cases being tested. However, as this is mainly about calling PEFT functionality, which is extensively tested in PEFT, I focused the tests mostly on the integration, with only a couple of tests for the functionality itself (namely the tests that call `_check_model_hotswap`).

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41297). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Let's maybe update diffusers to uplift our current constraint that this feature isn't available for text encoder LoRAs once this is merged.

Yes, that would be the next step once this PR is merged.

@ydshieh How to proceed with this PR? Sayak approved but I guess we need a transformers dev's approval too?

sorry cc @Cyrilvallez can you reviiew?
Add chatterbox
Author: eustlb
Pull request: yes

# What does this PR do?

⚠️ This PR depends on and shouldn't be undrafted until both are merged:
https://github.com/huggingface/transformers/pull/38757
https://github.com/huggingface/transformers/pull/38756

Should be regularly rebased from the temporary aggregated branch [add-chatterbox-base](https://github.com/eustlb/transformers/tree/add-chatterbox-base) following the workflow:
```bash
git checkout main
git pull
git branch -D add-chatterbox-base
git checkout -b add-chatterbox-base
git merge add-hiftnet
git merge add-s3tokenizer

git checkout add-chatterbox
git rebase add-chatterbox-base
git push --force-with-lease
```



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38759). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@bot /style

Style fixes have been applied. [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/15606203085).

Added S3 Tokenizer support here
#42298
Add llama 4 scaling
Author: juliendenize
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Add llama 4 scaling for long context to Mistral models.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten @ArthurZucker @zucchini-nlp


Comments:
Look good, let's just address one comment and fix CI so it's ✅ 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mistral, mistral3
Remove error string test that was failing
Author: Rocketknight1
Pull request: yes

`test_use_safetensors` was failing in the CI after #41580 because different error messages are returned in some cases, and this test inspected specific error message text.

This PR keeps the rest of the test, but removes the specific text check.

cc @ArthurZucker @ydshieh 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42301). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I think it's fine - I don't think we really care about the exact message contents!
Transformers 4.57.1 not working with torch < 2.3
Author: iglaweb
Pull request: no
Labels: name: bug

### System Info

In the Readme it is said

> Transformers works with Python 3.9+, and [PyTorch](https://pytorch.org/get-started/locally/) 2.1+.

I use torch 2.2.2 on Mac Sonoma 14.7 (Intel) and I get the following error:

 ```bash
   inputs = processor.apply_chat_template(
  File "/home/venv310/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
    return func(*args, **kwargs)
  File "/home/venv310/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
    return func(*args, **kwargs)
  File "/home/venv310/lib/python3.10/site-packages/transformers/processing_utils.py", line 1705, in apply_chat_template
    out = self(
  File "/home/venv310/lib/python3.10/site-packages/transformers/models/internvl/processing_internvl.py", line 229, in __call__
    video_inputs = self.video_processor(videos=videos, **video_kwargs)
  File "/home/venv310/lib/python3.10/site-packages/transformers/video_processing_utils.py", line 206, in __call__
    return self.preprocess(videos, **kwargs)
  File "/home/venv310/lib/python3.10/site-packages/transformers/video_processing_utils.py", line 387, in preprocess
    preprocessed_videos = self._preprocess(videos=videos, **kwargs)
  File "/home/venv310/lib/python3.10/site-packages/transformers/models/internvl/video_processing_internvl.py", line 126, in _preprocess
    stacked_videos = self.resize(stacked_videos, size=size, interpolation=interpolation)
  File "/home/venv310/lib/python3.10/site-packages/transformers/image_processing_utils_fast.py", line 361, in resize
    if torch.compiler.is_compiling() and is_rocm_platform():
AttributeError: module 'torch.compiler' has no attribute 'is_compiling'
```

'is_compiling' NOT exist in <= 2.2.2
https://github.com/pytorch/pytorch/blob/v2.2.2/torch/compiler/__init__.py

### Who can help?

@zucchini-nlp @gante @stevhliu 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Steps to reproduce:
1. Install latest transformers version and torch 2.2.2
2. Run video/image inference with InternVL3-1B model
3. 

### Expected behavior

Run w/o errors with Torch 2.2.2

Comments:
Hi, this is outdated, and our minimum version now is `2.3` I believe. Torch 2.2.2 is quite old by now!
Properly protect the is_compiling checks
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title! The checks need to be guarded depending on torch versions
Fixes https://github.com/huggingface/transformers/issues/41960

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: csm, falcon_mamba, lfm2, lfm2_moe, mamba

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42304). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
docs: clarify recommended usage of max_new_tokens in generate()
Author: hawon223
Pull request: yes

# What does this PR do?

This PR updates the documentation for the `generate()` method by clarifying that `max_new_tokens` is now the recommended argument for controlling output length, while `max_length` remains for backward compatibility.

## Before submitting
- [x] This PR fixes a typo or improves the docs

## Who can review?
@stevhliu

Comments:
Remove outdated methods in modeling_utils.py
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. Do some cleanup. They are very out-dated and not useful anymore.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42302). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bark