noamaanMulla-03/transformers-issues huggingface dataset:
Fix GPT-OSS TP IndexError and unwrapping DTensor
Author: akshan-main
Pull request: yes

# [GPT-OSS] Fix Tensor Parallelism IndexError and DTensor casting

## What does this PR do?

This PR fixes two specific issues preventing **GPT-OSS** models from training with Tensor Parallelism (TP) and FSDP, as reported in #41819.

**The changes are:**

1.  **Fix `IndexError` in TP Hooks:**
    * **Bug:** The `tensor_parallel` hooks in transformers expect the input tensor (hidden states) to be passed as the first positional argument (`args[0]`). The `GptOssDecoderLayer` was previously passing `hidden_states` as a keyword argument, causing the hook to fail with `IndexError: tuple index out of range`.
    * **Fix:** Updated `GptOssDecoderLayer.forward` to pass `hidden_states` as the first positional argument.

2.  **Fix DTensor Casting in Eager Attention:**
    * **Bug:** When TP is enabled, `module.sinks` is wrapped as a `DTensor`. The `eager_attention_forward` function attempts to `torch.cat` this with `attn_weights` (a local tensor), causing a crash.
    * **Fix:** Added a check to detect if `sinks` is a `DTensor` and unwrap it before concatenation.

**Status:**
I have applied these changes to `modular_gpt_oss.py` and ran `make fix-copies`

Fixes #41819


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://github.com/huggingface/transformers/issues/41819)? Please add a link
      to it if that's the case. (Linked above)
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@3outeille 
@ArthurZucker

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gpt_oss

Hey @3outeille @ArthurZucker  ! It'd be great if this pr can be reviewed and you'd want me to integrate anything else

sorry context switching so hard that I forgot to click on `submit the reviews` lol

> sorry context switching so hard that I forgot to click on `submit the reviews` lol

haha understandable! I will work on trying to fix this

Hi @akshan-main,  I am also seeing DTensor type for `self.weights` and `self.bias` in `GptOssTopKRouter` module. And at multiple other places


@akshan-main  @quic-akuruvil encountered the issue again and had to fix it: https://github.com/huggingface/transformers/pull/42906
Add support for GPT_OSS with tp_plan or enable native tensor parallelism
Author: quic-akuruvil
Pull request: no
Labels: name: New model

### Model description

 #[https://huggingface.co/docs/transformers/main/perf_infer_gpu_multi?tp_plan=auto+plan](url)

> https://github.com/huggingface/transformers/issues/41819

There are a list of supported models here, but GPT-OSS is not one of them. Please add support for GPT_OSS too to enable `tp_plan`. Please help me understand when model is prepared for TP in accelerate initiation, is there some native support needed in model for enabling TP.

I have tried this example TP script [https://github.com/huggingface/accelerate/blob/main/examples/torch_native_parallelism/nd_parallel.py](url) with pure TP, on GPT-OSS-20B model and getting same error as mentioned in this already open issue:
[]([https://github.com/huggingface/transformers/issues/41819](url).)

After handling `DTensor` sinks as mentioned as a fix in above issue, still I find many such `DTensors` at multiple other places which is causing below error, due to incompatibility between ` DTensor ` and `torch.Tensor`. 

`raise RuntimeError(
[rank0]: RuntimeError: aten.bmm.default: got mixed torch.Tensor and DTensor, need to convert all torch.Tensor to DTensor before calling distributed operators!`

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
see https://github.com/huggingface/transformers/pull/42906
[Quantization] CI green by end of year
Author: MekkCyber
Pull request: yes

# What does this PR do?

Fixing some tests related to `compute_model_sizes` in bnb, and other tests for `compressed-tensors`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42951). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bnb
Threaded loading puts tensors on wrong device when device="cuda"
Author: kmod
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.0.0rc1
- Platform: Linux-6.14.0-36-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 1.2.1
- Safetensors version: 0.7.0
- Accelerate version: 1.12.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1+cu128 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition

### Who can help?

When I upgraded to v5.0.0rc1 my single-node DDP (via accelerate) setup broke, where during model loading GPU0 will end up having most of the tensors loaded onto it causing an OOM. It looks like what's going on is that #41580 (@ArthurZucker ) made threaded loading the default, and the cuda default device is thread-local, and I had been using `device="cuda"` which loads onto the default device, so in threaded mode all the default devices reset to 0 and all the weights are loaded on gpu0 causing an oom. If I set `device=accelerator.device` or HF_DEACTIVATE_ASYNC_LOAD=1 then the error goes away. On v4.57.3 if I set HF_ENABLE_PARALLEL_LOADING=1 I get the same issue.

Currently (v4.57.3), setting device="cuda" works with accelerate because accelerate will set the default device during Accelerator construction. The [accelerate docs](https://huggingface.co/docs/accelerate/en/basic_tutorials/migration) recommends changing device="cuda" to device=accelerator.device but doesn't mention it as a requirement. I don't know if device_map="cuda' is now considered a user error when using multiple gpus, but if so it would be nice to see that documented or ideally caught. Perhaps the threaded loading code could assert that if the device_map is cuda that there isn't also a non-zero default device set, since in this case the tensor will end up in a different place depending on whether or not threaded loading is enabled.

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. Create a script with DP, ie dp_replicate_size>1
2. Load the model with device_map="cuda"
3. Launch the script with either torchrun or `accelerate launch`
4. See that GPU0 has much more memory used or that it OOM'd

### Expected behavior

- No OOM, like in v4.57.3

Comments:
This seems like an interaction between accelerate and threaded loading so cc @arthurzucker @sunmarc

Wow haha I did not even know `device_map="cuda"` would work that way before. We should prob just map automatically to `torch.default_device()` 

Thanks for the nice report ! Can you try the PR above ? 

That PR worked for me, thanks!
[kernels] Fix failling tests
Author: MekkCyber
Pull request: yes

# What does this PR do?

After moving the use of `get_kernel` from `kernels` to using a wrapper around it from `hub_kernels.py`, we need to adapt the tests to override `get_kernel` inside `integrations/hub_kernels.py`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42953). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix concat order
Author: eustlb
Pull request: yes

# What does this PR do?

Had one in two chances, picked the wrong order 😅
here's[ the original code for reference](https://github.com/facebookresearch/perception_models/blob/3e352cca660658d4b5c90f42a7808b11469e4c66/core/audio_visual_encoder/pe.py#L436-L447)


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: pe_audio_video

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42946). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix StaticCache crash when batch size changes between runs
Author: lukepayyapilli
Pull request: yes

## Summary
Fix `StaticCache` crash when batch size changes between inference runs, even when the new batch size is smaller than `max_batch_size`.

## Problem
When using `StaticCache` with different batch sizes across multiple inference runs (after calling `reset()`), it would crash with:
```
RuntimeError: index_copy_(): Source/destination tensor must have same slice shapes.
Destination slice shape: 4 20 64 at dimension 2 and source slice shape: 8 20 64 at dimension 0.
```

This is problematic for users who want to reuse a pre-allocated cache with CUDA graphs / `torch.compile`, but need to handle variable batch sizes.

## Solution
- Use tensor slicing instead of `index_copy_` to update only the relevant batch positions
- Return only the batch slice that was actually used

## Changes
- `StaticLayer.update()`: Use slicing to support variable batch sizes
- `StaticSlidingWindowLayer.update()`: Same fix for sliding window variant
- Added regression test `test_static_cache_variable_batch_size`

Fixes #42454

Comments:
Closing as duplicate of #42467. I didn't check for existing PRs before submitting - apologies for the noise\! The fix in #42467 is more comprehensive and handles edge cases with torch.compile and encoder-decoder models.
Fix QA max_answer_len validation error message
Author: antznette1
Pull request: yes

Summary
This PR fixes a small typo in QuestionAnsweringPipeline._sanitize_parameters() where the ValueError raised for invalid max_answer_len was missing a closing parenthesis in the "(got ...)" portion of the message.

Changes
- Fix malformed ValueError message for max_answer_len validation in:
  - src/transformers/pipelines/question_answering.py
- Add a lightweight regression test (no model downloads) that asserts the error message is well-formed:
  - tests/pipelines/test_pipelines_question_answering.py

Testing
- python -m pytest tests/pipelines/test_pipelines_question_answering.py -k sanitize_parameters_max_answer_len_error_message -q

Comments:
Rolled into #42948 per request.
Fix infinity in JSON serialized files
Author: LysandreJik
Pull request: yes

Currently, there are configuration classes like Mamba2 that serialize non-JSON floats like +/- Infinity or NaN.

This adds an encode/decode attribute at JSON load and save.

This prevents situations such as https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 where the config.json cannot be parsed by the Hub as it contains non-JSON values.

Comments:
View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42959&sha=177df0

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42959). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix tests trainer again
Author: SunMarc
Pull request: yes

# What does this PR do?

Fixes trainer tests. A managed to fix a couple of deepspeed tests due to weights that were not correctly initialized and more. 

Comments:
run-slow: deepspeed, fsdp, trainer

This comment contains `run-slow`, running the specified jobs: 

models: ["deepspeed", "fsdp", "trainer"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42933). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20312131789)

✅ No failing test specific to this PR 🎉 !


run-slow: deepspeed, fsdp, trainer


This comment contains `run-slow`, running the specified jobs: 

models: ["deepspeed", "fsdp", "trainer"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20377547759)

✅ No failing test specific to this PR 🎉 !
Add SDPA and FlashAttention-2 support to LayoutLMv3
Author: jackiehimel
Pull request: yes

# What does this PR do?

Adds SDPA and FlashAttention-2 support to LayoutLMv3 using the unified attention interface pattern, following the same architecture used in BERT and other recent model implementations.

Fixes #35467

## Implementation

- Refactored `LayoutLMv3SelfAttention` to use `ALL_ATTENTION_FUNCTIONS` interface instead of separate attention classes
- Created `layoutlmv3_eager_attention_forward` function that implements the CogView attention mechanism (alpha=32 scaling) with support for LayoutLMv3's relative position bias and spatial attention bias
- Added `_supports_sdpa = True` and `_supports_flash_attn = True` flags to `LayoutLMv3PreTrainedModel`
- Updated mask creation to use `create_bidirectional_mask` (replacing `get_extended_attention_mask`)
- Threaded `layer_idx` parameter through attention classes for consistency
- **Added automatic enforcement in `LayoutLMv3Config`** to set `attn_implementation="eager"` when relative or spatial attention biases are enabled (default behavior)

**Note**: SDPA and FlashAttention-2 are incompatible with LayoutLMv3's relative position bias and spatial attention bias. The config automatically enforces eager attention when these biases are enabled (the default). To use SDPA/FlashAttention-2, users must disable both biases (has_relative_attention_bias=False and has_spatial_attention_bias=False).

## Type of change

- [x] New feature (non-breaking change which adds functionality)

## How has this change been tested?

- CI tests passing, linting and formatting locally validated.
- Added test skips for SDPA/Flash comparison tests (LayoutLMv3 defaults to eager when biases are enabled) and overrode `test_batching_equivalence` to ensure eager attention is used.
- Implementation follows the unified attention pattern from BERT.

## Before submitting

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes?
- [ ] Did you write any new necessary tests?

## Who can review?

@vasqu - Thanks for the feedback on #41801! I've refactored this to use the unified attention interface pattern as you suggested. Would appreciate another look when you have time.

@ArthurZucker @CyrilVallez - attention implementation reviewers

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: layoutlmv3
[`Generation`] Fix default overwrite for non-`None` defaults
Author: vasqu
Pull request: yes

Followup to #42702 where defaults are now either `None` or a boolean value. 

The logic was dependent on the values being `None` which ignored overriding other values, e.g. `do_sample`. The fails due to this can be seen in a few integration tests, e.g. gpt2.

Comments:
run-slow: gpt2,whisper,qwen2_5_vl,glm4v,biogpt

This comment contains `run-slow`, running the specified jobs: 

models: ["models/biogpt", "models/glm4v", "models/gpt2", "models/qwen2_5_vl", "models/whisper"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42958). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: whisper,gemma3n,gpt2,moshi,bert

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20373740604)

⚠️ No test being reported (jobs are skipped or cancelled)!


💔 This comment contains `run-slow`, but unknown error occurred and [the workflow run](https://github.com/huggingface/transformers/actions/runs/20375269407) aborted!

run-slow: whisper,gemma3n,gpt2,moshi,bert

This comment contains `run-slow`, running the specified jobs: 

models: ["models/bert", "models/gemma3n", "models/gpt2", "models/moshi", "models/whisper"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20376273736)

✅ No failing test specific to this PR 🎉 !


View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42958&sha=202fd1
dia model generation precision error because of generation config change.
Author: jiqing-feng
Pull request: no
Labels: name: bug

### System Info

regression PR: #42702 

### Who can help?

@zucchini-nlp 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Applied from https://github.com/nari-labs/dia/blob/main/hf.py 

```python
from transformers import set_seed, AutoProcessor, DiaForConditionalGeneration

set_seed(42)

torch_device = "cuda"
model_checkpoint = "nari-labs/Dia-1.6B-0626"

text = [
    "[S1] Dia is an open weights text to dialogue model. [S2] You get full control over scripts and voices. [S1] Wow. Amazing. (laughs) [S2] Try it now on Git hub or Hugging Face."
]
processor = AutoProcessor.from_pretrained(model_checkpoint)
inputs = processor(text=text, padding=True, return_tensors="pt").to(torch_device)

model = DiaForConditionalGeneration.from_pretrained(model_checkpoint).to(torch_device)
outputs = model.generate(**inputs, max_new_tokens=3072, guidance_scale=3.0, temperature=1.8, top_p=0.90, top_k=45)

print(outputs)
outputs = processor.batch_decode(outputs)
processor.save_audio(outputs, "dia_example.mp3")
```

Output before the PR:
```
tensor([[[1026, 1026, 1026,  ..., 1026, 1026, 1026],
         [ 232, 1026, 1026,  ..., 1026, 1026, 1026],
         [ 628, 1026, 1026,  ..., 1026, 1026, 1026],
         ...,
         [1025, 1025, 1025,  ..., 1024,  659,  374],
         [1025, 1025, 1025,  ..., 1025, 1024,  610],
         [1025, 1025, 1025,  ..., 1025, 1025, 1024]]], device='cuda:0')
```
The audio output sounds correct.

Output after the PR:
```
tensor([[[1026, 1026, 1026,  ..., 1026, 1026, 1026],
         [ 568, 1026, 1026,  ..., 1026, 1026, 1026],
         [ 568, 1026, 1026,  ..., 1026, 1026, 1026],
         ...,
         [1025, 1025, 1025,  ..., 1024,   30,  199],
         [1025, 1025, 1025,  ..., 1025, 1024,  364],
         [1025, 1025, 1025,  ..., 1025, 1025, 1024]]], device='cuda:0')
```
The audio output is just noise.

### Expected behavior

Hi @zucchini-nlp . Would you please check it? Thanks!

Comments:
Is it fixed by https://github.com/huggingface/transformers/pull/42958, can you check @jiqing-feng ?

Yes, it has been fixed by the PR. Thanks!
Xiao mi_mo implementation
Author: Aaraviitkgp
Pull request: yes

Issue : #42954

I have implemented mimo model,
I have just changed rope_type as it was default which was causing error so now it is linear by default also changed and have made some changes in class MiMoV2FlashRotaryEmbedding, added files for mimo integration.

 @ArthurZucker 



Comments:
Add MiMo V2 Flash model integration
Author: Aaraviitkgp
Pull request: yes

Issue : https://github.com/huggingface/transformers/issues/42954

I have implemented mimo model,
I have just changed rope_type as it was default which was causing error so now it is linear by default also changed and have made some changes in class MiMoV2FlashRotaryEmbedding, added files for mimo integration.

@ArthurZucker

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42991&sha=acedb3
fix device dismatch issue for pe_audio_video model parallelism
Author: kaixuanliu
Pull request: yes


Comments:
I also made some adjustment in `prepare_config_and_inputs` func in test file. As current implementations will have the chance to set all 0 for one video input frames in `padding_mask_videos`, as a result the output will have `Nan` value for this input, which will cause failed output tensor comparison in [L3041](https://github.com/huggingface/transformers/blob/v4.57.3/tests/test_modeling_common.py#L3041). I tested on both A100 and Intel XPU, w/o this adjustment these test cases will get failed sometimes due to `Nan` output:
```
tests/models/pe_video/test_modeling_pe_video.py::PeVideoEncoderTest::test_disk_offload_bin
tests/models/pe_audio_video/test_modeling_pe_audio_video.py::PeAudioVideoEncoderTest::test_cpu_offload
tests/models/pe_video/test_modeling_pe_video.py::PeVideoEncoderTest::test_disk_offload_safetensors
``` 
after this PR, they can get passed all the time.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: pe_audio, pe_audio_video, pe_video, timm_wrapper

run-slow: pe_audio, pe_audio_video, pe_video, timm_wrapper

This comment contains `run-slow`, running the specified jobs: 

models: ["models/pe_audio", "models/pe_audio_video", "models/pe_video", "models/timm_wrapper"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42917). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20422991545)

✅ No failing test specific to this PR 🎉 !