noamaanMulla-03/transformers-issues huggingface dataset:
[loading] Use fewer threads by default for much better performances
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title.
After a lot of experimentation and reading on the subject, having fewer threads is much more performant than 16 for our loading pipeline (I/O bound operations). This is because otherwise they compete too much with each others.

For example, on a 45B parameters models, the following snippet:

```python
from transformers import AutoModelForCausalLM
import torch
import time

model_id = "mistralai/Mixtral-8x7B-v0.1"
device = "auto"

torch.cuda.synchronize()
t0 = time.time()
model = AutoModelForCausalLM.from_pretrained(model_id, device_map=device, dtype=torch.float16)
torch.cuda.synchronize()
dt = time.time() - t0
print(f"Took {dt:.2f} s")
```

takes about `51.07 s` on `main` vs `20.98 s` on this branch, on our cluster (H100 gpus)

So having fewer threads actually goes more than 2x faster!! 
For completeness, if we try using `GLOBAL_WORKERS = 1` (i.e. no parallelism), then it takes about `30.02 s`. Using `8` is still too much and gives `27.58 s`.
So 4 seem to be the sweet spot.

NOTE FOR THE FUTURE:
The benchmark uses fp16, which is not the native dtype of the checkpoint (bf16), so an additional copy to cast is needed. When loading in bf16, using 4 or 16 threads leads to ~15s for both. So it seems that 4 is anyway the best default here, but the bottleneck is maybe not I/O bound -> depending on whether `tensor.to(dtype, device)` performs the dtype casting on cpu or on the device where it's sent (not sure of the operation order), it may be cpu-bound

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42324). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`CI`] Skip `EfficientLoFTR` test
Author: vasqu
Pull request: yes

As per title, added a fixme @yonigozlan 

This is hotfix (as it blocks CI) but tbh we should not inherit from other classes, i.e. SuperGlue. Not sure how many of these cases are still hidden.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: efficientloftr

We should probably look into something along #42247 but for any type of test

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42327). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Attn Masks`] Lift bidirectional mask restriction on eager
Author: vasqu
Pull request: yes

Given that the torch compiler fuses our eager attention to sdpa automatically under compile, we should take more advantage and allow no masks on bidirectional masks without padding == fa backend. This restriction is also unnecessary and I'm not sure why I had this one to begin with.

Fuse strategies can be seen here:
- https://github.com/pytorch/pytorch/blob/main/torch/_inductor/fx_passes/fuse_attention.py

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42325). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Good point, let me check if we properly skip the mask in a test here. 

run-slow: bart,bert,llama

This comment contains `run-slow`, running the specified jobs: 

models: ["models/bart", "models/bert", "models/llama"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19578011236)

✅ No failing test specific to this PR 🎉 !
Remove generic output_attentions warning
Author: Aravind-11
Pull request: yes

# What does this PR do?

Removes the overly broad generic warning for `output_attentions=True` that was added in #40597. This warning incorrectly flagged custom `AttentionInterface` backends that properly return attention weights.

Fixes #41924

## Changes

**Removed from `src/transformers/utils/generic.py` (lines ~1013-1025):**
- Generic warning that fired for ALL non-eager attention implementations

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you write any new necessary tests?


## Who can review?



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: owlvit
MistralCommonTokenizer does not match PreTrainedTokenizer
Author: Fhrozen
Pull request: no
Labels: name: bug

### System Info

on docker
os: ubuntu 24.04
transformers: 4.55.0.dev0
mistral_common: 1.8.3

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Command to lauch container:

```bash
docker run --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model mistralai/Voxtral-Mini-3B-2507
```


### Expected behavior

The output will finish in:

```bash
vllm-1  |   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/tokenizer_group.py", line 24, in __init__  
vllm-1  |     self.tokenizer = get_tokenizer(self.tokenizer_id, **tokenizer_config)
vllm-1  |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vllm-1  |   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/tokenizer.py", line 309, in get_tokenizer
vllm-1  |     tokenizer = get_cached_tokenizer(tokenizer)
vllm-1  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vllm-1  |   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/tokenizer.py", line 104, in get_cached_tokenizer
vllm-1  |     tokenizer_all_special_tokens = tokenizer.all_special_tokens
vllm-1  |                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vllm-1  | AttributeError: 'MistralCommonTokenizer' object has no attribute 'all_special_tokens'. Did you mean: '_all_special_ids'?
```

vLLM docker server uses the pretrained tokenizer format:
https://github.com/vllm-project/vllm/blob/49314869887e169be080201ab8bcda14e745c080/vllm/transformers_utils/tokenizer.py#L97-L101

Which must include: `all_special_ids`, `all_special_tokens`, `all_special_tokens_extended` default properties. However, MistralCommonTokenizer does not have implemented them. Is there a plan to standarize both tokenizers?


Comments:
Hey Can i Help You to solve this issue..

I also get tokenization error using mistral-common tokenizer

`transformers/tokenization_mistral_common.py", line 566, in _piece_to_id
    assert len(pieces) == 1, f"Expected to decode 1 token, got {len(pieces)}"
AssertionError: Expected to decode 1 token, got 3`

Any solution for this?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How to fine-tune SAM 3D models?
Author: bruno686
Pull request: no
Labels: name: New model

### Model description

The recently released SAM 3D work is truly remarkable. Do you plan to integrate it into Transformers and enable fine-tuning?
https://huggingface.co/facebook/sam-3d-objects

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Fix: handled index_error in set_zero3_state
Author: Aaraviitkgp
Pull request: yes

Error #41762

This PR fixes a potential `IndexError` and inconsistent initialization behavior that can occur during DeepSpeed initialization (`set_zero3_state`) and model weight setup (`initialize_weights`).

Added conditional statement to resolve issue.
@3outeille @ArthurZucker

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41769). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

before I review, can you take a look at the linting please ? It seems like a lot code changes are due to linting/reformating 

Is it possible to have as well a minimum working example for me to test please ? (with pinned version of deepspeed)

added test file and did formating through ruff  and black so there should be no issue.

@3outeille please review all the changes I have made.

@Aaraviitkgp Error https://github.com/huggingface/transformers/issues/41762 
Maybe this problem has been fixed
DataCollatorWithFlattening incompatible with sequence classification
Author: LPirch
Pull request: no
Labels: name: bug

### System Info

-  OS: Ubuntu 24.04
- python version 3.11.9
- transformers version 4.56.2


### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```python
from datasets import Dataset
from transformers import DataCollatorWithFlattening

ds = Dataset.from_list([
    {"input_ids": [0, 1, 2, 3], "labels": 1}, {"input_ids": [4, 5, 6], "labels": 0}
])

collator = DataCollatorWithFlattening()
batch = collator(ds)
print(batch)
```

### Expected behavior

## Observed Error

```
   2026 batch["input_ids"] += input_ids
   2027 if is_labels_provided:
-> 2028     batch["labels"] += [separator_id] + sample["labels"][1:]
   2029 else:
   2030     batch["labels"] += [separator_id] + input_ids[1:]

TypeError: 'int' object is not subscriptable
```

## Expected Behavior
Batch the labels in a `[batch_size,]` int64 tensor

## Additional Context

This [line](https://github.com/huggingface/transformers/blob/8cb5963cc22174954e7dca2c0a3320b7dc2f4edc/src/transformers/data/data_collator.py#L2146C17-L2146C73) in the `DataCollatorWithFlattening` class raises an exception if the "labels" column in a dataset is a literal instead of a sequence. This happens in sequence classification where there is a label for the whole sequence instead of individual tokens.

Comments:
I'm not sure `DataCollatorWithFlattening` was intended for this case! Maybe just use the normal collator without flattening?

A normal collator (i.e. adding padding) doesn't work with flash attention 2. This is what I've tried before but it leads to NaN gradients during training.

I think sequence classification is a common enough learning task to be covered by a data collator.

@LPirch i would like to contribute to this issue so if you dont mind could you assign it to me 

I can't assign people. But if it helps, I could also prepare a PR.

@LPirch I have created a pr #41687  fixing this issue  could you please review it 

Ah, I'm sorry! I didn't realize `DataCollatorWithFlattening` was intended as a FlashAttention workaround. I've checked the source now - I'll continue over to reviewing #41687.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Incorrect Access of Dataset Fields in check_tokenizers.py
Author: vignesh1507
Pull request: no

The `premise` field in the XNLI dataset is a string, not a dictionary**. Calling `.values()` on a string will raise an error.
Similarly, `hypothesis` is also a string (not a dictionary), so `"hypothesis"]["translation"` will fail.

Comments:
Fixed the issue #41631 

ذذذر

can i work on it...

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add support for passing in `BatchFeature` to `Wav2Vec2FeatureExtractor.__call__`
Author: ad12
Pull request: no
Labels: name: Feature request

### Feature request

Currently, `Wav2Vec2FeatureExtractor.__call__` takes in `raw_speech` as one of these types: `Union[np.ndarray, list[float], list[np.ndarray], list[list[float]]].` Include `BatchFeature` as a supported type. This new type should also be supported by all subclasses.




### Motivation

When using `Wav2Vec2FeatureExtractor`, it is possible that the data is already in a batched format (e.g. padded torch tensor). This may be because of a variety of reasons:
* tensor has already been moved to a specific device
* certain functional operations require the data being in the framework's primitive tensor format and is faster to run when batched
 
We should be able to construct a `BatchFeature` instance using given the metadata (i.e. `input_values` and `attention_mask`) and pass that as an input into `Wav2Vec2FeatureExtractor`. 

### Your contribution

I can help with discussion if this is something that is on the roadmap to implement

Comments:
@ad12 
I created a PR that contains this feature. You can review and give me your feedback

https://github.com/huggingface/transformers/pull/42343

Hey!

The `BatchFeature` is usually reserved to format the output from processors, not the input. It's not really clear in what situation you will have the same audio fed into the processor twice

I believe what the issue requests is an option to handle padded arrays as input, in which case @eustlb can tell more if we need it or not

Thanks for the feedback. You're right that this doesn't make much sense - I misunderstood the issue.

Looking at it again, I think the user wants to pass already-padded tensors (maybe already on GPU) without having them go through the padding step again. Our implementation just runs everything through the same pipeline, which defeats the purpose.

@ad12   could you clarify what behavior you're looking for here? Should the feature extractor skip padding when it detects that the input is already batched/padded?
Fix ChineseCLIPModel.get_text_features
Author: JiangJQ2000
Pull request: yes

# What does this PR do?

Currently, `ChineseCLIPModel.get_text_features` will throw the error
```TypeError: linear(): argument 'input' (position 1) must be Tensor, not NoneType.```
This happens because `ChineseCLIPTextModel` inside `ChineseCLIPModel` is instantiated with `add_pooling_layer=False`, while PR #40555 incorrectly treats its output as `BaseModelOutputWithPooling`.
Therefore, I reverted the changes related to this part.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: chinese_clip

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42351). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Gemma3 hybrid fix
Author: remi-or
Pull request: yes

In `gemma3` modeling tests, there are still mentions of the deprecated cache type `hybrid`
This PR removes those mentions and adjust the expectations when needed.

Comments:
run-slow: gemma3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/gemma3"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42287). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19508088646)

### Model CI Report

#### ❌ Failed tests

- [gemma3](https://github.com/huggingface/transformers/actions/runs/19508088646/job/55840459910):
    tests/models/gemma3/test_modeling_gemma3.py::Gemma3IntegrationTest::test_model_4b_batch



run-slow: gemma3

💔 This comment contains `run-slow`, but unknown error occurred and [the workflow run](https://github.com/huggingface/transformers/actions/runs/19509436703) aborted!

run-slow: gemma3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/gemma3"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19509988797)

✅ No failing test specific to this PR 🎉 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3

run-slow: gemma3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/gemma3"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19629325526)

✅ No failing test specific to this PR 🎉 !
fix(benchmarks): correct sdpa_backend inconsistency and attn_implementation for continuous batching
Author: engmohamedsalah
Pull request: yes

## Summary

This PR fixes two bugs in `BenchmarkConfig` reported in issue #42211:

1. **sdpa_backend inconsistency (line 105)**: The warning message states "sdpa_backend must be None" but the code was setting it to "math". Changed to None to match the warning message.

2. **Invalid attn_implementation (line 243)**: Changed from "paged|sdpa" to "sdpa". Using "paged|sdpa" directly bypassed the validation logic since it only checks for exactly "sdpa". The "paged|" prefix is automatically added by `init_continuous_batching()`.

## Technical Details

**Bug 1 - sdpa_backend inconsistency:**
- Location: `benchmark_v2/framework/benchmark_config.py:105`
- Issue: Warning says "setting it to None" but code sets to "math"
- Fix: Changed `self.sdpa_backend = "math"` to `self.sdpa_backend = None`
- Reasoning: Setting to None allows PyTorch to auto-select the appropriate SDPA backend for continuous batching scenarios with custom attention masks

**Bug 2 - attn_implementation bypass:**
- Location: `benchmark_v2/framework/benchmark_config.py:243`
- Issue: Using "paged|sdpa" bypasses validation at lines 91-105
- Fix: Changed from `attn_implementation="paged|sdpa"` to `attn_implementation="sdpa"`
- Reasoning: The "paged|" prefix is automatically added by `init_continuous_batching()` in `continuous_api.py` (lines 744-756), so configs should use plain implementation names

Both bugs were introduced in commit 069684ef87 (PR #41916).

Fixes #42211

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42339). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[kernels] Update npu kernel repository and layer name
Author: zheliuyu
Pull request: yes

# What does this PR do?

Still going strong and continuing to attempt. 😆

This is a proof-of-concept experiment for https://github.com/huggingface/transformers/issues/39105#issuecomment-3540644473

## Prepare the env
cann= 8.3.RC1
torch= 2.7.1
torch_npu= 2.7.1
device= Atlas 900 A2 * 8

```
pip install -e kernels

git clone https://github.com/zheliuyu/transformers-dev
pip install -e transformers-dev

pip install llamafactory
```

Using [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory), we fine-tuned Qwen3-8B.

```
llamafactory-cli train custom.yaml
```

## custom.yaml
```
### model
model_name_or_path: Qwen/Qwen3-8B
trust_remote_code: true

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 8
lora_target: all

### dataset
dataset: identity,alpaca_en_demo
template: llama3
cutoff_len: 2048
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16
dataloader_num_workers: 4

### output
output_dir: saves/Qwen/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
save_only_model: false
report_to: none  # choices: [none, wandb, tensorboard, swanlab, mlflow]

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
resume_from_checkpoint: null
```

## result

### No kernels

Set `use_kernels=False`

```
***** train metrics *****
"epoch": 3.0,
"total_flos": 2.8124,
"train_loss": 1.159449,
"train_runtime": 303.7698,
"train_samples_per_second": 10.7650,
"train_steps_per_second": 0.1780
```

### Use this pr

Set `use_kernels=True`

```
***** train metrics *****
"epoch": 3.0,
"total_flos": 2.8124,
"train_loss": 1.159411,
"train_runtime": 272.5237,
"train_samples_per_second": 11.9990,
"train_steps_per_second": 0.1980
```

(303.7698 - 272.5237) / 303.7698 ≈ 10.2%

The results show an approximate 10% speedup from w/o this pr.



Comments: