noamaanMulla-03/transformers-issues huggingface dataset:
Fix non-deterministic output in MoE experts with batched_mm/grouped_mm
Author: raimbekovm
Pull request: yes

# What does this PR do?

Fixes non-deterministic output when using `do_sample=False` with MoE models and `experts_implementation='batched_mm'` or `'grouped_mm'`.

The issue is `index_add_` with duplicate indices on CUDA. When `top_k > 1`, `token_idx` contains duplicates (`[0,0,1,1,2,2,...]`) and CUDA's `atomicAdd` doesn't guarantee operation order. Since FP addition isn't associative, this causes different rounding errors on each run.

The fix replaces `index_add_` with `view().sum()` - this works because each token appears exactly `top_k` times in sequence, so we can reshape and sum deterministically.

Fixes #43360

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #43360
- [ ] Did you make sure to update the documentation with your changes?
- [ ] Did you write any new necessary tests?

## Who can review?

@IlyasMoutawwakil

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm_image

thanks for the quick review!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43362). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43362&sha=fc0823
[v5] do_sample false not working ('batched_mm'). transformers 5.0 main branch
Author: Jintao-Huang
Pull request: no
Labels: name: bug

### System Info


<img width="951" height="450" alt="Image" src="https://github.com/user-attachments/assets/88db516b-ebb6-4435-ac65-483bb2275494" />


main branch transformers 5.0.0.dev


```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "zai-org/GLM-4.7-Flash"
messages = [{"role": "user", "content": "hello"}]
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
model = AutoModelForCausalLM.from_pretrained(
    pretrained_model_name_or_path=MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    experts_implementation='batched_mm',
)

inputs = inputs.to(model.device)
generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
output_text = tokenizer.decode(generated_ids[0][inputs.input_ids.shape[1]:])
print(output_text)
```












### Who can help?

-

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

.

Comments:
The issue is index_add_ with duplicate indices on CUDA. When top_k > 1, token_idx has duplicates ([0,0,1,1,2,2,...]) and atomicAdd doesn't guarantee order of operations. FP addition isn't associative so you get different rounding errors each run.

Thanks @Jintao-Huang for finding this ! and @raimbekovm for the fix !
Add support for SNAC model type
Author: rtATvw
Pull request: no
Labels: name: Feature request

### Feature request

ORTModel.from_pretrained("onnx-community/snac_24khz-ONNX", subfolder="onnx", file_name="encoder_model_quantized.onnx", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])

ValueError: The checkpoint you are trying to load has model type `snac` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`

### Motivation

Allows rendering of Orpheus inferences within transformers

### Your contribution

Not likely

Comments:
I’m interested in taking this on and would like to give it a try. I will update the issue after checking it on my end. 

sorry for the confusion, I will pass this and let the experts handle the integration. 
thanks @Rocketknight1 

FWIW The attached program generates the first word of the prompt when curl -d '{"prompt":"Man, the way social media has, um, completely changed"}' http://127.0.0.1:8081

[twisted_orph.py](https://github.com/user-attachments/files/24771802/twisted_orph.py)
Add support for Microsoft VibeVoice models
Author: AliBagheriNejad
Pull request: no
Labels: name: New model

name: Good Second Issue

### Model description

## Model Details + Issue Description 

VibeVoice-1.5B and VibeVoice-Realtime-0.5B, Microsoft TTS models. It seems that these models are not yet available in Transformers. When trying to load them with the following code:

```python
from transformers import pipeline
pipe = pipeline("text-to-speech", model="microsoft/VibeVoice-1.5B")
```
I get the error below:

```python
ValueError: The checkpoint you are trying to load has model type `vibevoice` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+[https://github.com/huggingface/transformers.git`](https://github.com/huggingface/transformers.git%60)
```

Am I doing something wrong or checkpoint for these models is not going to be available in Transformers.

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

https://huggingface.co/microsoft/VibeVoice-1.5B
https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B

Comments:
These models aren't yet supported in Transformers, and didn't supply remote modeling code. Their structure is a little unusual, but if someone wants to attempt a PR to add them we'd definitely be interested.

@Rocketknight1  I’m relatively new to adding new model architectures to Transformers. would you be able to suggest one or two existing TTS models in the codebase that would be good references for implementing `VibeVoice`, in terms of structure or pipeline integration?

That would help me follow established patterns and keep the PR aligned with Transformers conventions. Thanks!



The VibeVoice TTS models are being added with these PRs:
- 1.5B and 7B: https://github.com/huggingface/transformers/pull/40546
- 0.5B realtime (depends on above): https://github.com/huggingface/transformers/pull/42781
Change Torchvision functional alias to TVF
Author: yonigozlan
Pull request: yes

Change Torchvision functional alias to TVF in order to avoid conflict (in modular files mostly) where we need both `torch.nn.functional` , and `torchvision.transforms.v2` which are both usually aliased as `F`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43263). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

vF sounds good to me as well! or tvF, which makes it clear that this is an alias for torchvision

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: beit, bridgetower, chameleon, cohere2_vision, conditional_detr, convnext, deepseek_vl, deepseek_vl_hybrid, deformable_detr, depth_pro, detr, dinov3_vit, donut, dpt, efficientloftr, efficientnet
Support any capturing groups in WeightTransform reverse mapping
Author: yonigozlan
Pull request: yes

Fix reverse mapping for capturing groups in WeightTransform

Previously assumed capturing groups were always `(.+)` when restoring `\1` in reverse mapping, breaking patterns like `(\d+)` used by DETR in [this PR](https://github.com/huggingface/transformers/pull/41549/changes#diff-211df432018ea754cd45bffaa082f70f9263875332b9bdaa0ac70eb3a2caf7acR169-R170). 

Now stores actual capturing group patterns and correctly restore them during reverse transforms.

Also adds a fix to `test_reverse_loading_mapping` in `test_modeling_common.py`, to reverse target pattern before using it in a regex search, also needed for https://github.com/huggingface/transformers/pull/41549

Cc @Cyrilvallez 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43205). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43205&sha=43b550
Divergence in UMT T5 Encoder numerical values from v4.57.3 to v5.0.0.rc*
Author: SolitaryThinker
Pull request: no
Labels: name: bug

### System Info

Is this expected and intended? I am from FastVideo (https://github.com/hao-ai-lab/FastVideo) repo and we compare against transformer's `UMT5EncoderModel` in our CI. However upgrading from 4.57.3 -> v5.0.0 or later causes our CI to fail with a big numerical difference. 




### Who can help?

@ArthurZucker @Cyrilvallez 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Here's our CI test https://github.com/hao-ai-lab/FastVideo/blob/4112507e99b06ffca502d34fe2da1120371c2bd5/fastvideo/tests/encoders/test_t5_encoder.py#L48

Sorry I do not have bandwidth to give a smaller repro code

### Expected behavior

After upgrading to 5.0.0rc* our CI fails with follow numerical difference

<img width="1540" height="228" alt="Image" src="https://github.com/user-attachments/assets/e4465a2f-df88-42ea-9ad2-a6d1dc234816" />

For context this passes on transformer v4.57.3


Here's our CI test https://github.com/hao-ai-lab/FastVideo/blob/4112507e99b06ffca502d34fe2da1120371c2bd5/fastvideo/tests/encoders/test_t5_encoder.py#L48

Comments:
Hey! can you try with `rc1`? 🤗 otherwise will investigate!

it also occurs on rc1. I noticed that I get the following token embed missing warning on 5.0.0+ that does not appear on 4.57.3

<img width="2336" height="1518" alt="Image" src="https://github.com/user-attachments/assets/a61c5d1a-d167-4cf4-9408-e805e96829f3" />

Should be fixed in https://github.com/huggingface/transformers/pull/42815

TL;DR: we need to force weight tying for a few more models than currently

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
`torch._grouped_mm` support for `GptOssExperts`
Author: mattteochen
Pull request: no

I've seen that https://github.com/huggingface/transformers/pull/42697 introduced the possibility of using grouped matrix multiplication for MoE layers. Any reason why `gpt-oss` was left out (I know that its mlp layer is decorated with `MegaBlocksMoeMLP`)?

Comments:
I think it's due to the implementation differing between training/inference which made this not super obv but IMO we should still be able to add the decorator cc @IlyasMoutawwakil

IIRC`gpt-oss` needs a different activation and bias handling. Gate/up/down weights format (interleaved vs chunked) and shape might be different too wrt to other models.

i did manage to make it work with gpt oss (cc @ArthurZucker) however it does require some additional work, the cleanest way is to add support for transposing the expert matrix in the weight loader, the other way is to just support transposed expert matrices in the implementations themselves (which complicates their code a bit)

Transpose is already supported https://github.com/huggingface/transformers/blob/63b49261294c373933fad9d079c4fb2154b9dc0f/src/transformers/core_model_loading.py#L405

(needed this for #39585)

@vasqu awesome ! i will attempt using it ! although the problem I encountered with transposition was very subtle, it was in the fact that the original and resulting matrices had the same name (source and target) which didn't work during the reverse operation (saving of the weights) because of some changes we make to these two.

cc @Cyrilvallez for the weight loading, imo it would be nice if we could support this type of mapping regardless
[llama4] update rope with new parameter
Author: kashif
Pull request: yes

# What does this PR do?

use `config.rope_parameters["rope_theta"]`

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43311). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llama4

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43311&sha=d23c19

closed as its duplcate of #43402
Fix failing `Llama4IntegrationTests`
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?

Fixes these failing [Llama4IntegrationTests](https://github.com/huggingface/transformers/actions/runs/21199210283/job/60982517174#step:14:328)


<img width="2332" height="244" alt="image" src="https://github.com/user-attachments/assets/62d6bdb4-004a-4c54-9c12-40d5ede1a0ec" />



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@Rocketknight1 @vasqu

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llama4

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43402&sha=1befab

kindly close as its a duplicate of #43281 

> kindly close as its a duplicate of #43281

Yup. Didn't saw that one.

me neither haha
`Llama4VisionRotaryEmbedding` access deprecated config attribute `rope_theta`
Author: Tcc0403
Pull request: no
Labels: name: bug

### System Info

transformers==v5.0.0rc2

### Who can help?

@zucchini-nlp 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`Llama4VisionRotaryEmbedding` expects `Llama4VisionConfig` has `rope_theta` attribute
https://github.com/huggingface/transformers/blob/57278c904c5158999d31a0db8bfcd63360c37b48/src/transformers/models/llama4/modeling_llama4.py#L1019


### Expected behavior

`config.rope_parameters.rope_theta` instead

Comments:
Indeed, might have slipped off when I was updating RoPE modules. Would you like to submit a PR?

TBH some of the vision rope layers like in qwen-vl were not updated at the time, because it requires to completely change the RoPE module. I've been planning to work on it next

Sure, I'll submit a PR. Do I need to cover all or just llama4? I haven't gone through other modeling files.

Only llama4, the others will be a bigger refactor and I'd prefer someone from maintainers to work on it
Add @torch.no_grad() to cache layer update methods
Author: yashwantbezawada
Pull request: yes

Fixes #43010

### What does this PR do?

Adds `@torch.no_grad()` decorator to cache layer `update()` methods that use in-place operations, preventing `torch.func.grad` from failing with "in-place operation would mutate a captured Tensor" errors.

### Why only StaticLayer and StaticSlidingWindowLayer?

After investigation, I found that only these two classes need the decorator:

| Cache Layer | Operation | In-Place? | Needs `@torch.no_grad()`? |
|-------------|-----------|-----------|---------------------------|
| `DynamicLayer` | `torch.cat()` | No | No - breaks gradient flow |
| `DynamicSlidingWindowLayer` | `torch.cat()` | No | No - breaks gradient flow |
| `StaticLayer` | `index_copy_()` | **Yes** | **Yes** |
| `StaticSlidingWindowLayer` | `copy_()`, `index_copy_()` | **Yes** | **Yes** |
| `QuantizedLayer` | `torch.cat()` | No | No - breaks gradient flow |

Adding the decorator to `DynamicLayer` (and subclasses) would break gradient flow because `torch.cat()` creates new tensors that participate in the computation graph. Models like T5 use `DynamicCache` and need gradients to flow through cached key/values.

### Changes

Added `@torch.no_grad()` decorator to:
- `StaticLayer.update()` 
- `StaticSlidingWindowLayer.update()`

### Testing

- All 9 cache unit tests pass
- Verified `torch.func.grad` works with StaticCache
- Verified gradient flow preserved for DynamicCache

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43041). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Closing - this is probably okay, but it's incomplete and obviously written by a code agent, which makes it very hard to discuss and iterate on. I'd prefer to talk a bit with the original issue author and make a more principled PR that ensures consistent behaviour between the base cache classes and the custom model caches!
Fix always issuing generation deprecation warning
Author: githubnemo
Pull request: yes

# What does this PR do?

**Note:** already discussed with @zucchini-nlp internally

Commit 38e5987 from PR #43194 introduced a minor bug that causes the issuing of a deprecation warning regardless of the user using the deprecated interface.

For example:
```
model.generate(**input_ids, do_sample=False)
```
would issue a deprecation warning not to pass `GenerationConfig` as well as generation kwargs even though we're not passing a `GenerationConfig`.

Along the fix this also introduces a unit test to check both the intended and the bug behavior.




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43389). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43389&sha=e7cd07