noamaanMulla-03/transformers-issues huggingface dataset:
[xpu] [OOB] `pip install -e ".[torch]"` wrongly installed the CUDA version dependencies
Author: Stonepia
Pull request: no
Labels: name: bug

### System Info

When trying to install transformers with the following commands:



It installed the CUDA version of pytorch and related dependencies:

```
Collecting triton==3.5.1 (from torch>=2.2->transformers[torch])

Using cached torch-2.9.1-cp312-cp312-manylinux_2_28_x86_64.whl (899.7 MB)
Using cached nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl (594.3 MB)
Using cached nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (10.2 MB)
Using cached nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (88.0 MB)
Using cached nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (954 kB)
Using cached nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl (706.8 MB)
```

As is shown above, the `torch` and `triton` package will override the local XPU packages. Thus causing errors when  installing transformers with this way. Using `python setup.py develop` won't have this issue anyway.

### Who can help?

@yao-matrix 


### Reproduction

```Bash
cd transformers
pip install -e ".[torch]"
```

### Expected behavior

Maybe we could have a `USE_XPU=1` env flag for control?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Gemma3 with flash-attention2 outputs warning
Author: tolgadur
Pull request: no
Labels: name: bug

### System Info

When training with the GRPO trainer with flash-attention2 or any other mechanism other than eager you get a warning that can be quite confusing. The warning comes from here: https://github.com/huggingface/transformers/blob/main/src/transformers/models/gemma3/modeling_gemma3.py#L657-L661

However, looking at the code and related issues https://github.com/huggingface/transformers/issues/31997#issuecomment-2840563627 https://huggingface.co/google/gemma-2-9b-it/discussions/9#66923856e338220687e4fe1e it looks like this issue doesn't really exist anymore fore Gemma3 with the latest flash attention package. Could we remove this warning?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run the grpo trl trainer with any gemma3 model.

### Expected behavior

not see a warning when using flash-attention2

Comments:
Hi, i'm also run into this issue,
I'm using transformers==4.53.2, is the implementation of Gemma3 is fine for flash-attention2 ?

> Hi, i'm also run into this issue,
> I'm using transformers==4.53.2, is the implementation of Gemma3 is fine for flash-attention2 ?

You have to configure the eod token in the tokenizer otherwise it works.
[`Tests`] Fix inputs placement
Author: vasqu
Pull request: yes

As per title, we init our model on CPU but the inputs are not guaranteed to be on CPU as well. Forcing CPU here

Related failure: https://github.com/huggingface/transformers/actions/runs/20375489555/job/58553450832

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42963). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix formatting of trackio model tag
Author: abidlabs
Pull request: yes

Quick followup fixing: https://github.com/huggingface/transformers/pull/42850/files#r2637239463

cc @SunMarc @qgallouedec @julien-c 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42973). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Hardcode the factor in caching allocator
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. It's a nit, but as quantizers got their behavior aligned for warmup, I feel like it's clearer like this

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42996). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix dtype mismatch in in modeling_llava_next
Author: Godkunn
Pull request: yes

# What does this PR do?

Fixes #42968

This PR fixes a `RuntimeError` when running `LlavaNextForConditionalGeneration` with mixed precision (e.g., BFloat16 vs Float16).

**The Fix:**
I added a cast `.to(self.lm_head.weight.dtype)` to `hidden_states` before passing it to `lm_head`. This ensures the input tensor matches the linear layer's weight type.

**Verification:**
Verified locally on T4 GPU using the reproduction script from the issue. The forward pass now completes without crashing.

## Who can review?
@zucchini-nlp
<img width="1919" height="839" alt="image" src="https://github.com/user-attachments/assets/97fb9339-b5b7-4e21-ab5c-58cd00538e96" />
<img width="1772" height="626" alt="image" src="https://github.com/user-attachments/assets/1c7dd455-3ecf-4bb5-a594-45d7e852fb36" />

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llava_next

I don't think this is the right fix for this issue. It could actually come from the loading logic:

```python
>>> from transformers import AutoModelForImageTextToText
>>> model = AutoModelForImageTextToText.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf", dtype="auto")
>>> model.language_model.dtype, model.vision_tower.dtype
(torch.bfloat16, torch.float16)
```

but we expect `(torch.float16, torch.float16)`

this is probably the issue https://huggingface.co/llava-hf/llava-v1.6-mistral-7b-hf/blob/main/config.json#L43


Yep, this works:

```python
import torch
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf", dtype="auto", revision="refs/pr/46")
input_ids = torch.randint(0, model.config.text_config.vocab_size, (2, 8))
model(input_ids=input_ids)
```

Thanks @zucchini-nlp and @qgallouedec for the deep dive!

That makes total sense regarding the dispatching logic in the base modeling class. Since you've identified the root cause and will be implementing a central fix (and deprecating the mixed backbone dtypes), I'll close this PR so it doesn't clutter the queue.

Glad I could help bring attention to the issue! Happy to help verify the fix once it's live if needed...
Fix issue loading ASR pipelines
Author: pcuenca
Pull request: yes

Introduced in #42806

Before this PR, the following failed:

```py
from transformers import pipeline
pipe = pipeline("automatic-speech-recognition")
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42970). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Closing in favor of #42920.
Switch to sdpa_kernel api with newer torch version
Author: mobicham
Pull request: no
Labels: name: Documentation

name: Good Second Issue

name: bug

### System Info

- `transformers` version: 4.45.2
- Platform: Linux-5.15.0-89-generic-x86_64-with-glibc2.35
- Python version: 3.10.14
- Huggingface_hub version: 0.26.1
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 4090


### Who can help?

I noticed that many files in transformers use the older sdp api `torch.backends.cuda.sdp_kernel`. We just discovered a bug in Pytorch 2.5.0 and the old sdp api that would make it run slower https://github.com/pytorch/pytorch/issues/138386

It would be a good idea to update to the new api (`from torch.nn.attention import sdpa_kernel, SDPBackend`) and set the appropriate compile flag to avoid losing as much as 20% of the performance !

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Gist here as a reference example: https://gist.github.com/mobicham/aa1e77689d9cf866cbea2cb75a53a9e4 
More details in the torch issue: https://github.com/pytorch/pytorch/issues/138386

### Expected behavior

Examples using sdp with torch 2.5.0 should run at least as fast as 2.4.1

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

cc @LysandreJik @ArthurZucker here I think, since it's a core framework-level decision that affects many model files

Makes sense! But I am only seeing `torch.backends.cuda.sdp_kernel` being used in tests / documentation. 
Let's open this to the community to use the decorator! (in models we just use `torch.nn.functional.scaled_dot_product_attention` without a context manager)

Hi! @ArthurZucker @LysandreJik is there still work to be done for this issue? Thanks!

Closing this because it was still open and attracting the swarm of code agents
Feature/add max eval steps
Author: priyankaz-ml
Pull request: yes
Labels: name: Code agent slop

Summary:
This PR introduces a new optional argument max_eval_steps to TrainingArguments and integrates it into the Trainer evaluation loop. When set, evaluation will stop after the specified number of batches, even if the dataset has more batches.

Changes:

Added max_eval_steps to TrainingArguments with documentation.

Updated Trainer.evaluation_loop() to break early when max_eval_steps is reached.

Updated evaluate() docstring to mention the new behavior.

Added inline comments for traceability (#31561).

Motivation:
Large evaluation datasets can be time-consuming. This feature allows users to limit evaluation runtime for quicker experiments and debugging.

Issue Reference: Fixes #31561

Comments:
Hi! This PR adds max_eval_steps to limit evaluation batches (fixes #31561). Let me know if anything needs adjusting — happy to revise!
Switch to sdpa_kernel API for PyTorch 2.3+
Author: lukepayyapilli
Pull request: yes
Labels: name: Code agent slop

## Summary

- Updates tests and documentation to use the newer `torch.nn.attention.sdpa_kernel` API instead of the deprecated `torch.backends.cuda.sdp_kernel` API
- The old API causes performance issues in PyTorch 2.5.0 as documented in pytorch/pytorch#138386
- Adds backward-compatible helper functions in quantization tests that use the appropriate API based on PyTorch version

## Changes

- `tests/quantization/spqr_integration/test_spqr.py`: Added `sdpa_kernel` helper function and updated usage
- `tests/quantization/aqlm_integration/test_aqlm.py`: Added `sdpa_kernel` helper function and updated usage  
- `docs/source/ko/llm_optims.md`: Updated Korean documentation to reference the new API

## Test plan

- [x] The helper functions maintain backward compatibility for PyTorch < 2.3
- [x] The new API is used correctly for PyTorch >= 2.3
- [x] Quantization tests (SPQR, AQLM) should pass with both old and new PyTorch versions

Fixes #34411

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aqlm_integration, spqr_integration
Fix evaluation loop behavior when max_eval_steps is set
Author: PRATHAMESHG1234
Pull request: yes
Labels: name: Code agent slop

This PR follows up on #41299 and fixes remaining issues around max_eval_steps.

- Adds max_eval_steps to TrainingArguments
- Limits evaluation by number of batches
- Fixes num_samples handling for iterable datasets
- Preserves callback event ordering

All trainer tests pass locally.


Comments:
View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42980&sha=ed59aa

The `tests_torch` failure is due to a known ResNet initialization bug (see #31841), not this PR.

**Failing test:** `tests/models/resnet/test_modeling_resnet.py::ResNetModelTest::test_can_load_ignoring_mismatched_shapes`
**Error:** `AssertionError: 0.11898990720510483 not less than or equal to 0.1` for `classifier.1.bias`

This is a pre-existing issue where ResNet's `ignore_mismatched_sizes=True` produces out-of-bound initialization values. My changes only modify evaluation loop behavior (`max_eval_steps`) and don't touch model initialization code.

All other tests passed. Requesting maintainer review.

@sgugger @ArthurZucker @muellerzr   Could you review this PR? The ResNet test failure appears unrelated to the changes.
Timm doesnt work well with AutoModelForImageClassification, num_labels doesn't work as expected.
Author: lucian-student
Pull request: no
Labels: name: bug

### System Info

'4.57.1'

```
RuntimeError: Error(s) in loading state_dict for Linear:
	size mismatch for bias: copying a param with shape torch.Size([1000]) from checkpoint, the shape in current model is torch.Size([6]).
```

### Who can help?

Practically anybody, probably an easy fix.

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
model = AutoModelForImageClassification.from_pretrained("timm/efficientvit_b3.r256_in1k",num_labels=6)
```

### Expected behavior

Whenever i pass num_labels i get this since effiicient_net was trained using 1000 labels. 

timm.create_model works fine, so I assume timm integration isn't refined. 

Expected behaviour that chechkpoint will be loaded using timm. Not other means.

Comments:
You probably want to set `ignore_mismatched_sizes=True`!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix incorrect library name in BitNet integration warning
Author: leaderofARS
Pull request: yes

# Fix Typos and Documentation Issues in BitNet Integration

## Summary
This PR addresses three minor issues in [src/transformers/integrations/bitnet.py](/transformers/src/transformers/integrations/bitnet.py:0:0-0:0) to improve code quality, documentation accuracy, and consistency.

## Issues Fixed
1. **Incorrect Warning Message**:
   The warning message logged when no linear modules are found during [replace_with_bitnet_linear](/transformers/src/transformers/integrations/bitnet.py:314:0-369:16) incorrectly referenced "eetq" instead of "bitnet" (likely due to a copy-paste error from the EETQ integration).

2. **Docstring Grammar**:
   Fixed a grammar error in the [unpack_weights](/transformers/src/transformers/integrations/bitnet.py:54:0-120:33) docstring where the word "we" was repeated ("we we").

3. **Docstring Parameter Mismatch**:
   The [activation_quant](/transformers/src/transformers/integrations/bitnet.py:166:4-188:43) method in [BitLinear](/transformers/src/transformers/integrations/bitnet.py:123:0-207:16) accepts an argument named `input`, but the docstring incorrectly listed it as `x`.

## Solution
- Updated the log message to accurately reflect the "bitnet" integration, avoiding confusion during debugging.
- Removed the duplicate "we" from the [unpack_weights](/transformers/src/transformers/integrations/bitnet.py:54:0-120:33) docstring for better readability.
- Corrected the parameter name in the [activation_quant](/transformers/src/transformers/integrations/bitnet.py:166:4-188:43) docstring to match the function signature (`x` → `input`).

## Files Changed
- `src/transformers/integrations/bitnet.py`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42966). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[loading][TP] Fix device placement at loading-time, and simplify sharding primitives
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title.
Device placement was not correct during loading. It would be loaded on cpu, and only moved thanks to the `param_value = DTensor.from_local(` call in `set_param_for_module` function that would implicitly set it to the correct device. However, this is less efficient than doing it during loading directly.

For llama3 70B and TP 8, it goes from ~9s to ~6.5s in loading time for the following code:

```python
from transformers import AutoModelForCausalLM
import torch
import time
import os

model_id = "meta-llama/Llama-3.3-70B-Instruct"

rank = int(os.environ["RANK"])
world_size = int(os.environ["WORLD_SIZE"])
device = torch.device(f"cuda:{rank}")
# Need to be initialized explicitly to use the `barrier` before loading
torch.distributed.init_process_group(backend="nccl", rank=rank, world_size=world_size, device_id=rank)

torch.cuda.synchronize(device)
torch.distributed.barrier()
t0 = time.time()
model = AutoModelForCausalLM.from_pretrained(model_id, tp_plan="auto")
torch.cuda.synchronize(device)
torch.distributed.barrier()
dt = time.time() - t0
if rank == 0:
    print(f"Took {dt:.2f} s")

torch.distributed.destroy_process_group()
```

So it's quite a big difference.

Also, there was a lot of redundancy and useless args in sharding primitive functions. This PR cleans it.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43003). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix deepspeed + quantization
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR continues to try fixing some tests on deepspeed and quantization side.

One big issue was that we were not moving the missing weights correctly, making the initialization unsuccessful. Another one was that the checkpoint where not loaded correctly due to the prefix not being fixed. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43006). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bnb
Do not return a tuple in mistral tokenizer Automapping
Author: molbap
Pull request: yes

# What does this PR do?

As per title. Currently seems to break `from_pretrained` IIUC

repro:

```python
from transformers import AutoTokenizer
model_id = "mistralai/Mistral-Small-3.2-24B-Instruct-2506"
tokenizer = AutoTokenizer.from_pretrained(model_id)

print(tokenizer("about mistral"))
```

breaks on `main`

cc @itazap @ArthurZucker 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42997). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

Added a basic test just to check that we indeed map to strings/valid class names!
[Whisper] Pipeline: handle long form generation
Author: eustlb
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/34210 https://github.com/huggingface/transformers/issues/31942 #36602

In the tokenizer decoding logic for the pipeline, timestamp offsetting when the call to Whisper's generate have seeking (meaning generating for a new segment).

# EDIT
⚠️ ❗
This also fixes another issue, indirectly spotted in #36612: when `condition_on_prev_tokens=True`, we need to use last generated tokens as `decoder_input_ids`. Nevertheless, this requires skipping one of the double ending tokens (cf #34537) to match OAI implementation, done via in place modification `tokens=tokens[:-1]`. But we actually need this token to be kept for further decoding (also cf #34537) !!  

# TODO
- [x] make sure the edge cases are correctly handled: what about `chunk_length_s=60` e.g. ? →  actually Whisper just should not be used with `chunk_length_s` set! **Added a warning**
- [x] add a test for above mentioned edge case →  done via complexifying `test_large_timestamp_generation` with `condition_on_prev_tokens=True`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35750). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I tried the code in this PR on a sample audio. The chunk timestamps go out of sync with the audio, and it gets worse the longer the input audio is.  

Hi @eustlb, could you please tell what is the status of this PR?

Hey @as-suvorov, it's waiting for a core maintainer's approval to merge. @ArthurZucker I've addressed your comment, ready to merge 🤗
cannot import name 'ALLOWED_LAYER_TYPES' from 'transformers.configuration_utils'
Author: AndreasKaratzas
Pull request: no
Labels: name: bug

### System Info

transformers-5.0.0.dev0
Platform: both NVIDIA (CUDA) and AMD (ROCm)
Python 3.12.12

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce the behavior:

1. `pip install --upgrade git+https://github.com/huggingface/transformers`
2. Create a script: 

```python
from vllm import LLM

lm = LLM(
    model="gpt2", 
    model_impl="transformers",
    load_format="dummy",
    gpu_memory_utilization=0.8,
)
```

And then run it. 

### Expected behavior

```
Traceback (most recent call last):
  File "/tmp/tr_test.py", line 1, in <module>
    from vllm import LLM
  File "/usr/local/lib/python3.12/dist-packages/vllm/__init__.py", line 74, in __getattr__
    module = import_module(module_name, __package__)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/llm.py", line 20, in <module>
    from vllm.config import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/config/__init__.py", line 18, in <module>
    from vllm.config.model import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/config/model.py", line 14, in <module>
    from transformers.configuration_utils import ALLOWED_LAYER_TYPES
ImportError: cannot import name 'ALLOWED_LAYER_TYPES' from 'transformers.configuration_utils' (/usr/local/lib/python3.12/dist-packages/transformers/configuration_utils.py). Did you mean: 'ALLOWED_MLP_LAYER_TYPES'?
```

Comments:
This is a vllm issue. I made a mistake posting this here. Closing