noamaanMulla-03/transformers-issues huggingface dataset:
Fix cache update!
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. https://github.com/huggingface/transformers/pull/37873 broke the cache update when going beyond the sliding window, see my comment [here](https://github.com/huggingface/transformers/pull/37873#discussion_r2081837193).
This PR fixes it. 
This also incorporates the issue mentioned in https://github.com/huggingface/transformers/issues/37574! TLDR, the order of operations here is important as we check strict inequality!!
Correcteness can be verified with

```python
model_id = "google/gemma-2-9b-it"
device = 0


tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map=device)



chat1 = [  # This size + the new tokens is > than sliding window
  {"role": "user", "content": "This is a nice place. " * 675 + "\n\nForget about the previous text, and tell me who you are?"},
]
prompt1 = tokenizer.apply_chat_template(chat1, tokenize=False, add_generation_prompt=True)
chat2 = [
  {"role": "user", "content": "create a list of at least 10 colors please"},
]
prompt2 = tokenizer.apply_chat_template(chat2, tokenize=False, add_generation_prompt=True)

inputs = tokenizer([prompt1, prompt2], padding=True, return_tensors="pt").to(0 if device == "auto" else device)

print(f"Sliding window: {getattr(model.config, 'sliding_window', None)}")
print(f"Input size: {inputs.input_ids.shape}")
# print(inputs.keys())





cache = "hybrid"

compile_config = CompileConfig(fullgraph=False)
out = model.generate(**inputs, do_sample=False, max_new_tokens=100, cache_implementation=cache, compile_config=compile_config)

text = tokenizer.batch_decode(out[:, inputs.input_ids.shape[-1] :], skip_special_tokens=False)
print("\n\n")
for seq in text:
    print("NEW SEQ:")
    print(seq)
```

It use to generate correctly for both the sequence > sliding window and the padded sequence, and now generates very badly. This fixes it once and for all.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @gante 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38046). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Cyrilvallez indeed, my previous PR fixed a small (tested) issue but created a large (untested) issue 🙈
Gemma2 crash on load with torch <= 2.4
Author: francois-rozet
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.52.4
- Platform: Linux-6.1.134.1.fi-x86_64-with-glibc2.28
- Python version: 3.12.5
- Huggingface_hub version: 0.33.0
- Safetensors version: 0.5.3
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.4.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Gemma2 crashes on load with "older" torch version (2.4). It does not crash with torch 2.5 and 2.7.

```python
>>> from transformers import AutoModelForCausalLM
>>> AutoModelForCausalLM.from_pretrained("google/gemma-2-2b-it")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/home/frozet/.venvs/transformers/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 571, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/frozet/.venvs/transformers/lib/python3.12/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/frozet/.venvs/transformers/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4508, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/frozet/.venvs/transformers/lib/python3.12/site-packages/transformers/models/gemma2/modeling_gemma2.py", line 659, in __init__
    self.model = Gemma2Model(config)
                 ^^^^^^^^^^^^^^^^^^^
  File "/mnt/home/frozet/.venvs/transformers/lib/python3.12/site-packages/transformers/models/gemma2/modeling_gemma2.py", line 429, in __init__
    self.post_init()
  File "/mnt/home/frozet/.venvs/transformers/lib/python3.12/site-packages/transformers/modeling_utils.py", line 1969, in post_init
    if v not in ALL_PARALLEL_STYLES:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable
```

This issue affects the `diffusers` library (see https://github.com/huggingface/diffusers/issues/11721)

### Expected behavior

No crash.

Comments:
cc @cyrilvallez since this seems related to the TP refactor!

Hey! I'm pretty sure this has been fixed already! Could you check again on `main`? 🤗

I just confirmed that this is fixed in the `main` branch. I'll close the issue!
[Whisper] fix shape mismatch in tests
Author: eustlb
Pull request: yes

# What does this PR do?

The switch to `torch.testing.assert_close` entailed shape mismatches, forgot to update 😅


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39074). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`WIP`][`Bert`] Attention Refactor for Bert-based Models
Author: vasqu
Pull request: yes

Follow up to #38108 (should not be merged before that PR)

Comments:
Will reopen to a later point :D included some stuff I didn't want to

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38265). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix `Gemma3nProcessorTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

Use  [a new repository](https://huggingface.co/hf-internal-testing/namespace-google-repo_name-gemma-3n-E4B-it/tree/main) to run processor tests on CircleCI.

or if @RyanMullins could check these maybe?

There are still 2 failures, could you take a look on these 2 please?

> FAILED tests/models/gemma3n/test_processing_gemma3n.py::Gemma3nProcessorTest::test_save_load_pretrained_additional_features - AssertionError: '{\n [107 chars]  "disable_grouping": null,\n  "dither": 5.0,\[1044 chars]n}\n' != '{\n [107 chars]  "dither": 5.0,\n  "do_center_crop": null,\n [1015 chars]n}\n'

> FAILED tests/models/gemma3n/test_processing_gemma3n.py::Gemma3nProcessorTest::test_save_load_pretrained_default - AssertionError: '{\n [107 chars]  "disable_grouping": null,\n  "dither": 0.0,\[1044 chars]n}\n' != '{\n [107 chars]  "dither": 0.0,\n  "do_center_crop": null,\n [1015 chars]n}\n'


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39068). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Uninstallling Flash attention from quantization docker
Author: MekkCyber
Pull request: yes

# What does this PR do?

Flash attention (dependency of autoawq) is failling with some missing symbol error, it causes all the other quantization tests to fail. Uninstalling it for now to allow the tests to run.

Failling Job : https://github.com/huggingface/transformers/actions/runs/15915442841/job/44892146131
New docker job : https://github.com/huggingface/transformers/actions/runs/15924428551/job/44918352936#step:5:3812

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39078). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Cleanup Attention class for Siglip and dependent models
Author: yaswanth19
Pull request: yes

Observed in another PR by Arthur where we error out early if output_attn is True for SDPA. This PR is inline with that for Siglip and it's dependent Models.

Comments:
CC: @ArthurZucker for review.

Regarding the CI there seems to a circular dependency for T5gemma, `make fix-copies` is adding a line whereas `make style` is removing that 🤔 

@yaswanth19 those kinds of circular issues can be caused by outdated style tools - try `pip install -e .[quality]`!

@Rocketknight1 Installed again (already up to date) and tried multiple times and in diff combinations between fix-copies, fixup and style but the issue did not get resolved 😢 . The issue maybe coz of `auto_docstring` 🤔 . If possible can you check on your local machine whether this persists or use the style bot here to see if it resolves it or not.

@yaswanth19 This might be less than ideal, but a practical solution might be to delete the modeling code of `t5_gemma` and recreate it via modular, e.g. `python utils/modular_model_converter.py ...` 

Can you check if that works?

Most of the changes in `t5gemma` modular file are for consistency reasons and to make CI green. Weird issue but with changes in last commit CI is fixed. 

@ArthurZucker This PR should help in #36625 as we are using siglip attention in there. Main issue was since we were using refactored mask utils - we need to also use the updated attention classes where we don't fall back to eager implicitly. 

Yep thanks, started reviewing but forgot about it 🤣
Add EoMT
Author: NielsRogge
Pull request: no
Labels: name: New model

name: Good Second Issue

### Model description

It'd be great to make the EoMT model available in the Transformers library, because it does image segmentation (instance, semantic and panoptic) in a much simpler way that [Mask2Former](https://huggingface.co/docs/transformers/main/en/model_doc/maskformer) and [OneFormer](https://huggingface.co/docs/transformers/en/model_doc/oneformer). The latter consist of multiple building blocks (vision backbone, pixel decoder, transformer decoder), making them overall pretty complex and hard to implement.

Simplicity allows to inherit the optimizations already done for ViT/BERT to be applied to segmentation as well. 

Initial thread: https://github.com/tue-mps/eomt/issues/1

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

Paper: https://huggingface.co/papers/2503.19108

Code: https://github.com/tue-mps/eomt

Comments:
Hi. I would like to contribute. I am new to open source but I have been using transformers for years. I am a senior data scientist. I would like to implement it :) . 

I just saw that the authors of the model itself will implement the model. I'll look for another issue then. 

cc @NiccoloCavagnero @tommiekerssies regarding the integration, please let us know if you have the bandwidth to proceed with it

Should it just be the model for inference, or also the code for training?

It would be super nice to have loss function implemented as well, the finetuning routine is usually provided as a separate example script / notebook and can be done on our side or by the community if you have a lack of bandwidth

I'll start with the model for inference, which is the easiest (simply a ViT with some extra query tokens, a class head for the queries, a mask head for the queries, and an upscale head for the patch tokens). Afterwards, we can implement the model for training (which adds masked attention and mask annealing). For the loss, we are actually already using the loss from the Transformers implementation of Mask2Former.

Hey! If anyone from the community is down to continue contributing the model, we would appreciate your help! You can start your work from the linked above PR 🤗 

cc @NahieliV

HI @qubvel , I can pick this up. Since I already have experience in contributing a model, it would be quicker and easier for me to iterate 🤗 .

@yaswanth19 sounds good 👍

Thanks, @yaswanth19! Let us know if you need anything or have any questions along the way. Please also see: https://github.com/huggingface/transformers/pull/37392#issuecomment-2798771866
Pipeline: fix unnecessary warnings
Author: eustlb
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/33498 https://github.com/huggingface/transformers/issues/35524

Handles two unnecessary warnings for the ASR pipeline:
- attention mask related: need to set `return_attention_mask=True` in `chunk_iter`. Its done anyway when not using `chunk_length_s` parameter ([here](https://github.com/huggingface/transformers/blob/099d93d2e9a1a5000f43f12fe63d66951f09a248/src/transformers/pipelines/automatic_speech_recognition.py#L445C12-L470C22))
- warning triggered when using deprecated input name for Whisper.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35753). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Integrating Kimi-Audio
Author: SeungyounShin
Pull request: yes

# What does this PR do?

Integrating [Kimi-Audio](https://huggingface.co/moonshotai/Kimi-Audio-7B-Instruct) to transformers

TODO

- [x] update `KimiAudioConfig`, `KimiAudioForCausalLM`
    + just copied from [official code](https://huggingface.co/moonshotai/Kimi-Audio-7B-Instruct/blob/main/modeling_moonshot_kimia.py)
- [ ] `KimiAudioProcessor` (whisper + glm4_tokenizer)
- [ ] `KimiAudioForConditionalGeneration` (processor -> model -> detokenizer)

TEST CODE :

```python

from transformers.models.kimi_audio import KimiAudioConfig, KimiAudioForCausalLM, KimiAudioProcessor

model_id = "Seungyoun/Kimi-Audio-7B-Instruct"
config = KimiAudioConfig.from_pretrained(model_id)
# model = KimiAudioForCausalLM.from_pretrained(model_id)  
processor = KimiAudioProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": "/home/robin/Kimi-Audio/test_audios/output.wav"},
        ]
    }
]

```

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@eustlb

Comments:
Hey Hey @SeungyounShin,
Love you started working on this, thanks a lot! 🤗
What's the status of this PR?
Remove 'as_target_processor' from docstring of Wav2Vec2Processor* methods
Author: renet10
Pull request: no
Labels: name: bug

### System Info

Companion report to #38609 to modify the docstring within the code for Wav2Vec2Processor and Wav2Vec2ProcessorWithLM

### Who can help?

@eustlb 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

None.  Refer to #38609 for details.

### Expected behavior

Docstring should not refer to as_target_processor so users will migrate to new method.

Comments:
Opened by mistake, misunderstood a comment in #38609
fix `mistral3` tests
Author: ydshieh
Pull request: yes

# What does this PR do?

Mistral3's processor will return different prompt on different days, something like

```
Your knowledge base was last updated on 2023-10-01. The current date is 2025-06-24.
```

see the date at the end.

This PR patched the `processor.chat_templat` to always have ` 2025-06-20` and update the expect values for it.

I remove the expected values for T4 (we are going to switch to A10 anyway)

Comments:
run-slow: mistral3

This comment contains run-slow, running the specified jobs: 

models: ['models/mistral3']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38989). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I hope I can sleep this weekend 🙏 

FYI: @yao-matrix Maybe you will need to update the expected values for xpu too
Add kwargs support in WhisperForConditionalGeneration
Author: tanuj-rai
Pull request: yes

# What does this PR do?

This PR update the forward method to support kwargs for flexible loss customization.

Fixes #36119 


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface, @SunMarc and @qgallouedec
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
cc @eustlb

> thanks, can you make sure the ci is green? 🤗

Thank you @ArthurZucker for reviewing it. I will make sure that all ci should be green.
fixed typo for docstring in prepare_inputs method
Author: JINO-ROHIT
Pull request: yes

fixed typo for docstring in prepare_inputs method

- [x] This PR fixes a typo or improves the docs .


## Who can review?

Documentation: @stevhliu



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39071). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
TST PEFT integration tests with pipeline generate
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

Some PEFT integration tests involving text generation pipelines were failing since #38129 because the base model is too small to generate longer sequences. Setting max_new_tokens fixes this.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.  <= internally on slack
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests? <= already there


## Who can review?

Comments:
ping @ydshieh @gante 

I think that the PEFT integration tests are not run as they are marked as slow.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39086). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> works for me as long as you locally verify it works :-)

Yeah, tests passed locally.
add fast image processor nougat
Author: NahieliV
Pull request: yes

# What does this PR do?

Adds fast image processor for Nougat model. 

https://github.com/huggingface/transformers/issues/36978


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? 
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@yonigozlan 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@yonigozlan , changes are done. There are small differences due to the difference in the implementation of  interpolation.BICUBIC  from PIL and PyTorch. For the test `test_slow_fast_equivalence`, there are 16 pixels with a difference over 1e-1. The max pixel difference is 0.18. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37661). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.