noamaanMulla-03/transformers-issues huggingface dataset:
RWKV: fix mask warning typo
Author: RobinKa
Pull request: yes

`RwKvModel.forward` is supposed to warn if `attention_mask` is passed. Right now the check is `attention_mask is None` which I assume was supposed to be `attention_mask is not None`, which this PR fixes. @gante

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Remove deprecated code
Author: cyyever
Pull request: yes

Remove code deprecated before current version.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Since this touches various bits of code, cc @qubvel @zucchini-nlp @mekkcyber @sunmarc, are you okay with these deprecated functions being removed?

The test failure is unrelated.
[tests] remove cuda-only test marker 
Author: faaany
Pull request: yes

## What does this PR do?
This PR enables this case on XPU: https://github.com/huggingface/transformers/pull/36656. So we don't need this test marker anymore. Furthermore, `AwqConfigTest` already has the `require_torch_accelerator` test marker. 

```bash
================================================================================= short test summary info =================================================================================
PASSED tests/quantization/autoawq/test_awq.py::AwqConfigTest::test_wrong_backend
==================================================================================== 1 passed in 0.10s ====================================================================================
```
@ydshieh 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Thanks!

cc @SunMarc for a second look 🙏
Add long vita
Author: NieCha
Pull request: yes

# What does this PR do?

Add the awesome Long_Vita model.

The referenced project links are as follows
https://huggingface.co/VITA-MLLM/Long-VITA-16K_HF
https://github.com/VITA-MLLM/Long-VITA


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

> Hey! THanks for the PR 🤗 before I review this, can you make sur your modeling code is as close as possible to `Llama` for example? meaning there are no single letter variable, you use our standard API for flash attention, make sure the layers are also similar in terms of the way we write the forwards etc! 🤗

Yes, we referred to the implementations of LLaMA and Qwen2_VL for developing our model and will continue to conduct checks.
Additionally, after submitting our request, we encountered two issues related to automated checks: 1) An error reporting 'import torch' in the modeling files; and 2) Numerous errors unrelated to our submitted code. 

It would be very helpful if you could explain the reasons behind these issues.🤗

Hey! I think the simplest way to share the model first is to put it on the hub! You won't have to deal with all these issues 🤗 
otherwise the process is a bit more involved, you should use `modular` https://huggingface.co/docs/transformers/main/en/modular_transformers to make sure it's easier to review and more aligned!

> Hey! I think the simplest way to share the model first is to put it on the hub! You won't have to deal with all these issues 🤗 otherwise the process is a bit more involved, you should use `modular` https://huggingface.co/docs/transformers/main/en/modular_transformers to make sure it's easier to review and more aligned!

Thank you for your reply😊😊. The model has already been uploaded to the Hugging Face hub, and the project address is https://huggingface.co/VITA-MLLM/Long-VITA-16K_HF.

Also, how can the code be merged without addressing these issues~

In order to get the model into transformers, one needs to make sure it is as close to our standard API as possible! So as I mentionned, make sure to use modular for classes that are inherited / very similar to other models. This makes the review a lot easier! 
The modeling code needs to be cleanup, and differences isolated to better understand what is this model! 
🤗 then once you think this is ready for review you can ping us 🤗
Fix processor kwargs qwen2 vl
Author: yonigozlan
Pull request: yes
Labels: name: for patch

# What does this PR do?

- Add support for using custom images kwargs in Qwen2VL and Qwen2.5VL processors
- Fix issue where `self.size` dict is mutated by a processor call in qwen2vl image processors

- Add tests for both these issues

Cc @hmellor @ArthurZucker 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36890). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ah great, I was banging my head with this !

This changes result in error in newest vllm 0.8.2
when loading UI-TARS

docker run --runtime nvidia --gpus all -d --name vllm-UI-Tars-7b --restart unless-stopped -v ~/.cache/huggingface:/root/.cache/huggingface -e VLLM_USE_V1=0 -p 8000:8000 vllm/vllm-openai:v0.8.2 --model bytedance-research/UI-TARS-7B-DPO --served-model-name llm  --trust-remote-code

error is:
File "/usr/local/lib/python3.12/dist-packages/transformers/models/qwen2_vl/image_processing_qwen2_vl.py", line 144, in __init__
ERROR 03-28 00:22:29 [engine.py:448]     raise ValueError("size must contain 'shortest_edge' and 'longest_edge' keys.")

Arf, happy to fix cc @hmellor ! 

[vLLM 0.8.2 uses Transformers 4.48.2](https://github.com/vllm-project/vllm/blob/v0.8.2/requirements/test.txt#L644), this PR is not present in that release.

Your command doesn't indicate that you're using a custom image, so I'm not sure how you're seeing an error caused by this PR @ErykCh?
:rotating_light: :rotating_light: :rotating_light: No more pointing at remote repos
Author: Rocketknight1
Pull request: yes

This is a very very draft PR, where we test disallowing one repo from loading another by using the `repo_id--classname` syntax, as this permits supply chain attacks. This is probably going to break all kinds of things until I chase down the code that was depending on it.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37047). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Closed because this is used by a lot of finetuned repos as well and we'll need some way to handle them properly before we could consider merging it
[docs] add return_timestamps=True for Whisper long-form transcription
Author: ADAning
Pull request: yes

In the [example of WhisperForConditionalGeneration.generate](https://huggingface.co/docs/transformers/model_doc/whisper#transformers.WhisperForConditionalGeneration.generate.example), `return_timestamps=True` should be set, otherwise it will trigger the ValueError [here](https://github.com/huggingface/transformers/blob/15bd3e61f8d3680ca472c9314ad07584d20f7b81/src/transformers/models/whisper/generation_whisper.py#L1306):

```shell
in _set_return_timestamps
    raise ValueError(
ValueError: You have passed more than 3000 mel input features (> 30 seconds) which automatically enables long-form generation which requires the model to predict timestamp tokens. Please either pass `return_timestamps=True` or make sure to 
pass no more than 3000 mel input features.
```

After adjusting to `generated_ids = model.generate(**inputs, return_timestamps=True)`, it works correctly:

It seems like it was just forgotten to be added :)

Comments:
cc @eustlb
TFAutoModelForSequenceClassification support for ModernBertConfig
Author: nkaccounting
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.50.3
- Platform: Linux-5.15.0-91-generic-x86_64-with-glibc2.35
- Python version: 3.11.10
- Huggingface_hub version: 0.26.2
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: MULTI_GPU
        - mixed_precision: no
        - use_cpu: False
        - debug: False
        - num_processes: 4
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: False
        - main_training_function: main
        - enable_cpu_affinity: False
        - downcast_bf16: False
        - tpu_use_cluster: False
        - tpu_use_sudo: False
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): 2.18.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100-SXM4-80GB

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I want to convert modern BERT from PyTorch version to TensorFlow version.
My code is below:
from transformers import TFAutoModelForSequenceClassification
import tensorflow as tf

in_path = ""
out_path = ""

with tf.device('/GPU:0'):
    tf_model = TFAutoModelForSequenceClassification.from_pretrained(in_path, from_pt=True)
    tf.keras.models.save_model(tf_model, out_path)
I get the result :
ValueError: Unrecognized configuration class <class 'transformers.models.modernbert.configuration_modernbert.ModernBertConfig'> for this kind of AutoModel: TFAutoModelForSequenceClassification.
Model type should be one of AlbertConfig, BartConfig, BertConfig, CamembertConfig, ConvBertConfig, CTRLConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, EsmConfig, FlaubertConfig, FunnelConfig, GPT2Config, GPT2Config, GPTJConfig, LayoutLMConfig, LayoutLMv3Config, LongformerConfig, MistralConfig, MobileBertConfig, MPNetConfig, OpenAIGPTConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoFormerConfig, TapasConfig, TransfoXLConfig, XLMConfig, XLMRobertaConfig, XLNetConfig.

### Expected behavior

All PyTorch model weights were used when initializing TFAutoModelForSequenceClassification.

All the weights of TFAutoModelForSequenceClassification were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.

Comments:
Hi @nkaccounting, we don't have a TF port of ModernBert at this time! You'll need to use PyTorch unless a user opens a PR to add TF support.
Add Fast Image Processors for mobileViT
Author: MinJu-Ha
Pull request: yes

related to #36978 
cc: @yonigozlan 

I added Fast image processor for mobileViT and I noticed a noticeable difference between the outputs after preprocessing.

Here’s the code I used to compare them:
```
diff = (encoding_slow.pixel_values - encoding_fast.pixel_values).abs()
print(f"\n📊 Difference statistics:")
print(f"  Max difference: {diff.max().item():.10f}")
print(f"  Mean difference: {diff.mean().item():.10f}")
print(f"  Slow min/max: {encoding_slow.pixel_values.min().item():.10f} ~ {encoding_slow.pixel_values.max().item():.10f}")
print(f"  Fast min/max: {encoding_fast.pixel_values.min().item():.10f} ~ {encoding_fast.pixel_values.max().item():.10f}")
print(f"Slow implementation dtype: {encoding_slow.pixel_values.dtype}")
print(f"Fast implementation dtype: {encoding_fast.pixel_values.dtype}")
```

results:
```
📊 Difference statistics: 
  Max difference: 0.3411765397
  Mean difference: 0.1117687449
  Slow min/max: 0.0313725509 ~ 0.9764705896
  Fast min/max: 0.0313725509 ~ 0.9764706492
Slow implementation dtype: torch.float32
Fast implementation dtype: torch.float32
```

Even though the size configs look the same ({'shortest_edge': 20}), and both use torch.float32, the output difference seems quite significant for a slow/fast equivalence test.

Comments:
[don't merge] debug audio pipeline
Author: ydshieh
Pull request: yes

# What does this PR do?

debug

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37138). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
skip
Author: ydshieh
Pull request: yes

# What does this PR do?

skippppppppppppppp for now

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37141). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Can not use flash-attention and flash-varlen-attention on Ascend NPU
Author: FightingZhen
Pull request: no

Transformers integrate flash-attention and flash-varlen-attention with package `flash-attn`, but this package is not supported on Ascend NPU, **even can not be installed**. In that case, we can not use flash-attention or flash-varlen-attention with transformers naturally.

Comments:
Refactor `return_dict` logic to remove complicated if/else paths
Author: qubvel
Pull request: yes
Labels: name: Core: Modeling

# What does this PR do?

As for the PR name, refactor `return_dict` logic to get rid of multiple if/else statements that overcomplicate the code.

Applied to the following base models and dependencies:
 - CLIP
 - SigLIP
     - SigLIP2
 - SAM
     - GOT-OCR2
 - Llama
     - Aria
     - Bamba
     - DiffLlama
     - Emu3
     - Gemma
     - Gemma2
     - GPT NeoX
     - Paligemma
     - Gemma3
     - GLM
     - Helium
     - JetMoe
     - Jamba
     - Mistral
     - Mistral
     - Mixtral
     - Nemotron
     - Olmo
     - Olmo2
     - Persimmon
     - Phi
     - Phi3
     - PhiMoe
     - Qwen2
     - Qwen2_moe
     - Qwen3
     - Qwen3_moe
     - StableLM
     - Starcoder2

## Who can review?

cc @yonigozlan @ArthurZucker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36794). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Here is the first batch of models, test failure seems to be unrelated, all Aria's tests pass locally
<img width="1245" alt="Screenshot 2025-03-19 at 10 24 47" src="https://github.com/user-attachments/assets/5be8e45e-29a0-4ac7-8d8d-a0eb1a32284c" />


Another error this time (test_processors), probably flaky, or another set of tests was triggered. I will trigger it once again

PR is updated to use `@can_return_tuple` decorator to move all `return_dict` logic out of modeling code.

P.S. Here is the same PR, but applied only to one model for easier initial review
 - https://github.com/huggingface/transformers/pull/36838
Allow quantizers to work with a state dict on meta device
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Following https://github.com/huggingface/transformers/pull/35926 and https://github.com/huggingface/transformers/pull/37086, make the quantizers work with a state dict on meta device.
See https://github.com/huggingface/transformers/pull/37086 for the issue at hand

Currently, will lead to inconsistencies and params initialized with random weights

cc @SunMarc @MekkCyber is there any way to replace params with only `param_value` instead of doing it for a whole layer at once with the `state_dict` for hqq and bnb 4bits? I believe this should be possible, but would require to know a bit more about internals of each lib for me.
Otherwise if truly not possible we can add an exception for these 2 types of quantizers, but it would be so much simpler if we can simply standardize and just replace one param after the other, instead of doing a full layer at once

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37133). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run slow: bnb 

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: []
quantizations: ['quantization/bnb'] ...
Remove deprecation cycle for `logits_to_keep`
Author: Cyrilvallez
Pull request: yes

# What does this PR do?



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
[New Model Addition] Baichuan M1
Author: Vaibhavs10
Pull request: yes

# What does this PR do?

Adds support for Baichaun M1

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Move `return_dict` logic into `can_return_tuple` decorator
Author: qubvel
Pull request: yes

# What does this PR do?

Move all `return_dict` logic from the modeling code to the `@can_return_tuple` decorator.

The default behaviour is to return the `ModelOutput` object but we still have the ability to get `tuple` output as follows:

```python

# returns `ModelOutput` object
outputs = model(**inputs)

# returns `tuple` outputs
outputs = model(**inputs, return_dict=False)

# returns `tuple` outputs
model.config.return_dict = False
outputs = model(**inputs)
```


Comments:
run-slow: llama

This comment contains run-slow, running the specified jobs: This comment contains run-slow, running the specified jobs:

models: ['models/llama']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36838). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Merged in https://github.com/huggingface/transformers/pull/36794
add gpt2 test on XPU
Author: jiqing-feng
Pull request: yes

Hi @SunMarc . We used to skip the gpt2 bnb tests due to the precision error, but now it has been fixed. We enable gpt2 bnb model test on the test script, and it can pass!

Also, the mix-precision 8-bit training on CPU and XPU has been fixed.

cc @Titus-von-Koeller @matthewdouglas 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37028). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Nice ! please fix quality issue

Hi @SunMarc . I have fixed the quality issue by updating branch.
Use public export API on torch 2.5 and future
Author: guangy10
Pull request: yes
Labels: name: torch export

# What does this PR do?

This is a follow-up of a TODO work. Due to issue https://github.com/pytorch/pytorch/issues/128394, we're using an internal API to export models if torch version is less than 2.5. Since the fix is included in 2.5, switch to use public export API for 2.5+.

Test
Tested the models exported using this integration points with torch 2.5, 2.6 and 2.7.0.dev

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker @amyeroberts @qubvel


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

Addressed comments

cc: @tugsbayasgalan before merging

Updated

@qubvel @ArthurZucker Can we merge this PR?

Thanks for the ping! Merged 🤗