noamaanMulla-03/transformers-issues huggingface dataset:
Fix inits in modernbert
Author: remi-or
Pull request: yes

# What does this PR do?
This PR fixes a few `modernbert` tests that are failing due to the new initialization scheme, only when `flash_attention` is available.

## Failure cause and fixes
The class  `ModernBertUnpaddedRotaryEmbedding` was not included in `init_weights`, which led to random initialization of `inv_freq`. It is now included and the initialization function used is the one of the parent class.

## Models affected
Modernbert only. 

## Fixed tests
- tests/models/modernbert/test_modeling_modernbert.py::ModernBertModelTest::test_cpu_offload
- tests/models/modernbert/test_modeling_modernbert.py::ModernBertModelTest::test_disk_offload_bin
- tests/models/modernbert/test_modeling_modernbert.py::ModernBertModelTest::test_disk_offload_safetensors
- tests/models/modernbert/test_modeling_modernbert.py::ModernBertModelTest::test_init_weights_can_init_buffers
- tests/models/modernbert/test_modeling_modernbert.py::ModernBertModelTest::test_model_parallelism
- tests/models/modernbert/test_modeling_modernbert.py::ModernBertModelTest::test_save_load



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43115). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: modernbert
[CI] Fixing some AMD failures
Author: Abdennacer-Badaoui
Pull request: yes

This PR fixes several failures on AMD for Qwen2, Qwen2.5-Omni, and Qwen2.5-VL.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42879). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: qwen2, qwen2_5_omni, qwen2_5_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen2", "models/qwen2_5_omni", "models/qwen2_5_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20301331240)

✅ No failing test specific to this PR 🎉 !


run-slow: qwen2, qwen2_5_omni, qwen2_5_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen2", "models/qwen2_5_omni", "models/qwen2_5_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20722104240)

✅ No failing test specific to this PR 🎉 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen2, qwen2_5_omni, qwen2_5_vl
Avoid inline .item() sync in decoder start token check
Author: pythongiant
Pull request: yes

# What does this PR do?

This PR refactors the decoder start token check in `_prepare_decoder_input_ids_for_generation` to avoid an inline
`.all().item()` call.

The boolean condition is now computed as a tensor on-device and converted to a Python boolean at a single, explicit synchronization point. This keeps the logic composable with other boolean checks and enables future batching of multiple conditions into a single GPU–CPU sync, in line with the motivation discussed in #43089.

There is no functional or behavioral change. The refactor is purely structural and preserves all existing semantics and special-case handling (e.g. Donut and Whisper models).


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

This change does not require new tests as it preserves existing behavior
and only restructures the boolean check to avoid an inline synchronization.


## Who can review?

This change affects generation utilities.

Tagging:
- @gante
- @zucchini-nlp


Comments:
[CB] Support the `num_return_sequences` argument
Author: remi-or
Pull request: yes

## Summary
This PR adds the options to fork requests during continuous batching, which duplicates the request and uses as much as possible the existing cache. This is then leveraged to make the `num_return_sequences` argument available in CB. 
This PR enables parallel decoding, which will be useful for RL workflows.

## Performance 
| Samples | Attention         | Add Prefix | Source  | Duration (s) | Generated tokens | Throughput (tok/s) |
|---------|-------------------|------------|---------|--------------|------------------|--------------------|
| 100     | flash_attention_2 | False      | With PR | 6.82         | 17480            | 2563.24            |
| 100     | flash_attention_2 | False      | On main | 6.91         | 17698            | 2562.81            |
| 100     | sdpa                    | False      | With PR | 21.77        | 17637            | 810.33             |
| 100     | sdpa                    | False      | On main | 21.81        | 17234            | 790.03             |
| 500     | flash_attention_3 | True       | With PR | 16.4         | 113054           | 6895.14            |
| 500     | flash_attention_3 | True       | On main | 16.73        | 112333           | 6715.63            |
* `--compile` is always on, thus the number of tokens varies between runs because the compiled kernels do

## Tests
Tests pass, including the one added to test the feature.

## Sanity check
Looks good

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42921). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix shapes in modular_gpt_oss.py
Author: tanmay2004
Pull request: yes

# What does this PR do?

This PR fixes a comment in `modular_gpt_oss.py` which has the incorrect shape written in the description for GptOssExperts.  I fix the annotated shape for routing_experts from `(batch_size * token_num, num_experts)` to `(batch_size * token_num, top_k)`. Looking at the git blame, I think the original implementation of GPT OSS had the original shape which was later modified but the comment was not edited. I also made a few other edits to the shape annotations for MoE in this file for consistency (assuming `num_tokens = batch_size * seq_len`). Please let me know if the changes make sense!

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gpt_oss

Done!

Hey, any update @ArthurZucker?
[`GPT OSS`] Fix false flag
Author: vasqu
Pull request: yes

#42736 removed flex attn support but it turns out that #41083 added support for it in flex attention. There were typos that indicated wrong support for attn flavors, sorry about that should've noticed it / checked it properly.

So gpt oss does not support sdpa but everything else (limited FA support to the specific kernel).

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gpt_oss

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43120). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Ernie 4.5 VL Moe`] Post merge adjustments
Author: vasqu
Pull request: yes

As per title, relevant PRs
- #42088
- #42697

Note that `grouped_mm` in moe leads to small fluctuations making the integration tests flaky

Comments:
cc @IlyasMoutawwakil re moe integration tests (fluctuations)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ernie4_5_vl_moe

run-slow: ernie4_5_vl_moe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43117). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This comment contains `run-slow`, running the specified jobs: 

models: ["models/ernie4_5_vl_moe"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20721504024)

✅ No failing test specific to this PR 🎉 !


Yea, lemme do a follow-up for the other moes - wasn't sure so waited for the nightly ci report
[Quantization] torchao serialization
Author: MekkCyber
Pull request: yes

# What does this PR do?

Fixes torchao serialization, some tests are still failling for Int8/Int4 quantization with: 

```
ValueError: Unsupported tensor type: <class 'torchao.dtypes.affine_quantized_tensor.AffineQuantizedTensor'>
``` 

which seems to be a problem on torchao's end



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: torchao_integration

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43127). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix test_all_tensors_are_parameter_or_buffer
Author: remi-or
Pull request: yes

# What does this PR do?

This PR fixes the test `test_all_tensors_are_parameter_or_buffer` across 5 models, effectively making the test pass for all models. 

## Failure cases and fixes

There are 2 reasons this test used to fail:
- the model has some triton code which crash when we try to do a forward pass on the CPU:
To address this, we add a keyword arg `non_meta_device` which is the device the model is transferred to. It defaults to `cpu`, so the default test behavior does not change. If set to `cuda`, the model is transferred to the GPU and the triton code can run.
- the model has inputs with nested tensors (list, dict, or objects with tensor attibutes):
 To address this, we add the `transfer_nested_tensors` flag, turned off by default so the default test behavior does not change. If the flag is passed, we go through inputs and try to transfer nested tensors whether they are part of a list, tuple, dict or plain attributes. 

## Models affected

No modeling code was changed, but the test was overloaded with kwargs in the following models:
- layoutlmv2 (the `images` input has a nested tensor attribute)
- modernbert (cannot run on cpu because of triton)
- pe_audio (cannot run on cpu because of triton)
- pe_video (cannot run on cpu because of triton)
- perceiver (the model has nested tensors in its inputs)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43111). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Nice, thanks for catching those! Instead of adding the keyword args, we can simply do `model.to(device=torch_device)` I believe, which is the default auto-detected device on which the tests are run. That would avoid overiding for the 5 models.
The logic of the test is not impacted by the device being cpu or cuda or whatever else, we only want to make sure it won't be meta!
Should for sure make the trick instead of `non_meta_device`. For `transfer_nested_tensors` I'm unsure, but I assume those models should prepare their nested inputs on the correct device (`torch_device`) by default as well, making it redundant as well?

Using `torch_device` is a much better solution tbh! Still leaving 3 overloads for the triton-related failures because they wont work on CPU, so they need the decorator `require_torch_gpu`

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: modernbert, pe_audio, pe_video

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43111&sha=41087a
🚨Fix MobileViT image processor default interpolation to BICUBIC
Author: lukepayyapilli
Pull request: yes

## Summary

Updates MobileViT image processor default interpolation from BILINEAR to BICUBIC to match:
- [Apple's original ml-cvnets implementation](https://github.com/apple/ml-cvnets)
- [timm's MobileViT config](https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/mobilevit.py) which uses `'interpolation': 'bicubic'`

## Changes

- `image_processing_mobilevit.py`: Updated default resample to BICUBIC
- `image_processing_mobilevit_fast.py`: Updated default resample to BICUBIC

Contributes to #28180

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mobilevit

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43024). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

No problem - thank you for the review + merge @yonigozlan!
Fix buffer offloading
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

This has been broken since https://github.com/huggingface/transformers/pull/42941, but was only surfaced after https://github.com/huggingface/transformers/pull/42697 as that PR changed the testers of MoE models with buffers so that they would always have at least 1 moe layer (which should have always been the case, as otherwise we test only half the model arch...)

Comments:
View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43131&sha=9763a2

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43131). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Moe`] Post interface fixes
Author: vasqu
Pull request: yes

As per title, a few tests fail with `grouped_mm` implementation. Way less integration tests fail than I initially expected. 
Related failures: https://huggingface.co/datasets/hf-internal-testing/transformers_daily_ci/raw/3fed21c3fdb63a1b19d6030d493139a947135fd3/2026-01-06/ci_results_run_models_gpu/new_failures_with_bad_commit_grouped_by_authors.json (note that there are still some offloading issues but they are unrelated to moe but a buffer issue)

Kind of a general follow up to #43117 and #42697

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ernie4_5_moe, lfm2_moe

run-slow: ernie4_5_moe,lfm2_moe,deepseek_v3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/deepseek_v3", "models/ernie4_5_moe", "models/lfm2_moe"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43129). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20750848067)

✅ No failing test specific to this PR 🎉 !
[`Ernie 4.5 VL Moe`] Fix non contiguous params
Author: vasqu
Pull request: yes

As per title, this is caused by the `transpose` conversion op which can cause non-contiguous layouts (kind of expected tbh). Simply forcing contiguous solves this.

Repro:
```python
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "baidu/ERNIE-4.5-VL-28B-A3B-PT",
    device_map="auto",
    dtype="auto",
    revision="refs/pr/10",
)

assert all(param.is_contiguous() for param in list(model.parameters())), "Some parameters are not contiguous"
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43134). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: timm_backbone
Do not use global variable, and improve context manager coverage
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42957). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42957&sha=c02cbe

Hi @pranjaldoshi96, I noticed you keep randomly approving pull requests without any reason. We do not tolerate this kind of behavior simply to try to maximize your github activity or something, as it can give false signal to members (PRs appear as approved - even though your approval is meaningless). This is my first and only warning, if I see you doing it again I will be forced to ban you from the repo

@Cyrilvallez I'm wondering if it would make sense to re-import `no_init_weights` into `modeling_utils.py` for backwards compatibility (with potential deprecation).

Humm, we could import it explicitly in `modeling_utils` so that the object is still reachable as before - would that work for you @BenjaminBossan? But be careful about it, we want to move away from it, and the only reason it still exists is due to deepspeed when model instantiation does not happen on meta device yet (but hopefully in the future)

> would that work for you

Actually, I don't directly need it, I just saw that the latest gptqmodel version fails because of this (they have a fix ready) and was concerned that there might be other code out there that depends on this function. If it is intended to be removed completely, I'd say it would be nice to have a deprecation warning and to add the re-import.

I did not add any deprecation cycle, as this function is originally coming from accelerate (and was only copy/pasted in Transformers directly for convenience, and recently), so I assumed anyone that wanted to use it/import it would do it directly from accelerate
Empty tensor in torch model trace for concat operation
Author: apbose
Pull request: no
Labels: name: bug

### System Info

transformer - 4.57.1 
GPT2 model trace leads to torch.cat getting empty tensor

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Repro code:

```
import torch
from transformers import AutoTokenizer, GPT2LMHeadModel
from torch._functorch.aot_autograd import aot_export_joint_simple
from typing import Sequence, Any

tokenizer = AutoTokenizer.from_pretrained("gpt2")
prompt = "GPT2 is a model developed by."
input_id= tokenizer(prompt, return_tensors="pt").input_ids
model = GPT2LMHeadModel.from_pretrained("gpt2").eval().to("cuda")

def custom_backend(gm: torch.fx.GraphModule, sample_inputs: Sequence[Any], **kwargs: Any):
    torch_inputs = [input for input in sample_inputs if isinstance(input, torch.Tensor)]
    def print_cat_inputs(*args, **kwargs):
        print("\n[cat] Intercepted torch.cat call")
        print("args:", args)
        print("kwargs:", kwargs)

        if len(args) > 0 and isinstance(args[0], (list, tuple)):
            cat_inputs = args[0]
            for idx, t in enumerate(cat_inputs):
                if isinstance(t, torch.Tensor):
                    print(f"  - Input {idx}: shape={t.shape}, device={t.device}, dtype={t.dtype}")
                else:
                    print(f"  - Input {idx}: non-tensor input: {t}")
        else:
            print("[cat] Could not detect list of tensors")

        return original_cat(*args, **kwargs)

    original_cat = torch.cat
    torch.cat = print_cat_inputs
    gm = aot_export_joint_simple(
        gm,
        torch_inputs,
        trace_joint=False,
        )
    return gm

cur_input = input_id.to("cuda")
model.forward = torch.compile(model.forward, backend = custom_backend)

```


### Expected behavior

The prints above show that the concat operation receives empty tensors. This comes in` transformers 4.57.1 `but does not in `transformers 4.52.4`

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi any updates on this?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.