noamaanMulla-03/transformers-issues huggingface dataset:
SeamlessM4TProcessorTest::test_save_load_pretrained_addition
Author: ydshieh
Pull request: yes

# What does this PR do?

https://app.circleci.com/pipelines/github/huggingface/transformers/155601/workflows/cfa672e3-a164-4bd3-b434-23015ab25d2d/jobs/2045707

> tests/models/seamless_m4t/test_processing_seamless_m4t.py::SeamlessM4TProcessorTest::test_save_load_pretrained_additional_features 
Too long with no output (exceeded 10m0s): context deadline exceeded

It hangs at the line

>  self.assertEqual(processor.tokenizer.get_vocab(), tokenizer_add_kwargs.get_vocab())


This happens after the PR 

rm slow tokenizers (#40936)

After internal discussion, we decided to delete this test for this model

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: seamless_m4t

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42562). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix eetq quanto quant methods
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR fixes EETQ and quanto integration ! We switched to kernels for eetq and we simplified a bit quanto. All tests are passing. 

Comments:
@bot /style

Style bot fixed some files and pushed the changes.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42557). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: eetq_integration, quanto_integration
Add backward compatibility for methods which have been moved to `RotaryEmbeddingConfigMixin`
Author: hmellor
Pull request: yes

This function was moved in https://github.com/huggingface/transformers/pull/42255. Many custom code models import this function, which is arguably good practise as it shows they are leveraging our first party utilities.

This PR reinstates the function by calling the methods which now live in `RotaryEmbeddingConfigMixin` so that these custom code models will continue to work with Transformers v5.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42517). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix parse_response after tokenizer refactor
Author: Rocketknight1
Pull request: yes

Since we're likely to be updating the schemas used by `parse_response`, this PR removes the documentation references to it for now. We're keeping it as a softlaunched prototype feature, and will re-add documentation and do an official launch once it's stabilized!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42300). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Also re-adding `Tokenizer.parse_response()` which got accidentally nuked in #40936

Also guarding the checks for `tokenizer.response_schema`, since that attribute was also nuked in https://github.com/huggingface/transformers/pull/40936
[Quantization] regression fix in FP8
Author: MekkCyber
Pull request: yes

# What does this PR do?

fixes https://github.com/huggingface/transformers/issues/42565

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42568). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix regression
Author: SunMarc
Pull request: yes

# What does this PR do?
This PR fixes https://github.com/huggingface/transformers/issues/42565. There was a minor issue in the FP8 logic. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42569). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Kernel mapping error resolve
Author: Aaraviitkgp
Pull request: yes

issue: #42451

This PR fixes a bug in KernelConfig.create_compatible_mapping where defining a kernel mapping for multiple devices (e.g., both cuda and rocm) caused the last defined device to overwrite the active device configuration.

@Rocketknight1 @MekkCyber 

Comments:
@Rocketknight1  I noticed that the run test issues that is occuring  ColQwen2ForRetrieval: Tensor vlm.lm_head.weight seems to have nothing to do with the changes made in pr.

cc @mekkcyber for kernels I think! also @Aaraviitkgp that `ColQwen2` issue should be fixed now if you rebase, sorry about that!

@Rocketknight1 the test_processor and test_tokenization have similar issue they fail but no changes in pr related to them can you look into that aswell 😁.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42466). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Transformers serve fix
Author: LysandreJik
Pull request: yes

Fix a few issues with `transformers serve`:
- Some models don't have EOS tokens; on these, we cannot check if that token is generated and purely rely on the length for the `length` cutoff.
- Some models (like Gemma3) load an image processor with `AutoProcessor` even if the image processor cannot be loaded. For these, we try/except and load with a tokenizer if we fail.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42570). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[SAM3] Compute masks once instead of per-layer, fix fa2 crash
Author: yonigozlan
Pull request: yes

Some small improvements for SAM3. There will be more to come as we discussed @molbap ;), including using modular.

This PR adds the following improvements:
- Compute attention masks once instead of per layer in the DETR encoder and decoder.
- Fall-back to sdpa when using FA2 in DETR decoder layer attention, as we pass a relative position bias matrix as an attention mask but fa2 doesn't support float attention masks. Cc @vasqu Is there any better ways to handle this?
- Remove unnecessary logic for presence token, pad rpb matric/attention mask directly.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42543). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam3
Allow fallback to loading from Auto"SubProcessor".from_pretrained when model_type can't be inferred from config
Author: yonigozlan
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/41846

Also remove some dead code in `processing_auto.py`

Cc @tomaarsen ;) 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42402). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks, Yoni. Yeah, let's try to get this and my PR merged for v5.

Thanks for the reviews! @zucchini-nlp could you approve pls ;) 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto
[`CI`] Fix copies
Author: vasqu
Pull request: yes

As per title

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: fast_vlm

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42571). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
docs: add doctest for SqueezeBERT
Author: arrdel
Pull request: yes

Adds documentation test for SqueezeBertForMaskedLM as part of the Doc Tests Sprint (#16292).

- Added example to forward() method showing masked language modeling
- Used squeezebert-uncased checkpoint for testing
- Added file to slow_documentation_tests.txt

# What does this PR do?

Fixes #16292 (partial - adds SqueezeBERT)


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: squeezebert
[Quantization] per tensor quantization kernel
Author: MekkCyber
Pull request: yes

# What does this PR do?

Adds a simple kernel for per tensor quantization, where the matmul is done per blocks of 128x128, and the weights scales, and activation scales are expected to be scalars

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42560). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Bug] Ministral 3 model loading fails with TypeError after PR #42455
Author: Aratako
Pull request: no
Labels: name: bug

### System Info

transformers: 5.0.0.dev0 @5efd0d4aa5ba4455f229b10667718cc623ac56af
python: 3.12
torch: 2.8.0+cu129 and 2.9.1+cu128
platform: Ubuntu (WSL) and Windows

### Who can help?

@SunMarc @MekkCyber 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

After merging PR \#42455, loading `mistralai/Ministral-3-14B-Instruct-2512` fails with a `TypeError: 'NoneType' object is not subscriptable`.

It seems that `block_size` is becoming `None` inside `finegrained_fp8.py`, causing the crash when trying to access `block_size[0]`.

**To Reproduce**

1.  Install `transformers` from main:

    ```bash
    pip install git+https://github.com/huggingface/transformers.git@main
    ```

2.  Run the following python script:

    ```python
    from transformers import Mistral3ForConditionalGeneration

    model = Mistral3ForConditionalGeneration.from_pretrained(
        "mistralai/Ministral-3-14B-Instruct-2512"
    )
    ```


### Expected behavior

The model should load successfully without errors.

**Traceback**

```text
Traceback (most recent call last):
  File "test_ministral3.py", line 3, in <module>
    model = Mistral3ForConditionalGeneration.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/transformers/modeling_utils.py", line 250, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/transformers/modeling_utils.py", line 3980, in from_pretrained
    hf_quantizer.preprocess_model(
  File ".../site-packages/transformers/quantizers/base.py", line 243, in preprocess_model
    self._process_model_before_weight_loading(model, **kwargs)
  File ".../site-packages/transformers/quantizers/quantizer_finegrained_fp8.py", line 172, in _process_model_before_weight_loading
    model = replace_with_fp8_linear(
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/transformers/integrations/finegrained_fp8.py", line 514, in replace_with_fp8_linear
    new_module = FP8Linear(
                 ^^^^^^^^^^
  File ".../site-packages/transformers/integrations/finegrained_fp8.py", line 328, in __init__
    scale_out_features = (out_features + block_size[0] - 1) // block_size[0]
                                         ~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
```

As a workaround, installing the commit before the merge fixes the issue:
`pip install git+https://github.com/huggingface/transformers.git@29e8522b85d9bf4287f63611aaa1a7db1fdc6489`

Comments:
Thanks for the report @Aratako, this should be fixed in the PR above !
Training hangs at the first gradient syncing of an MoE model while using deepspeed 
Author: negar-foroutan
Pull request: no
Labels: name: DeepSpeed

### System Info

- `transformers` version: 4.39.3
- Platform: Linux-5.15.0-94-generic-x86_64-with-glibc2.35
- Python version: 3.12.2
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.2
- Accelerate version: 0.28.0
- PyTorch version (GPU?): 2.2.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@pacman100

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I'm training a customized MoE language model using 8 GPUs on one node, and it works fine using Accelerate (without DeepSpeed). However, when I enable DeepSpeed (both ZerO 1 and 2), the training hangs at the first gradient syncing and will end up with an NCCL timeout after a few minutes. 
The MoE is built based on the XGLM architecture and the task as language modeling.
DeepSpeed works fine when I train a dense model (not MoE).  
I also set `deepspeed_moe_layer_cls_names` to my MoE block, but it doesn't seem to work. 
I use the `accelerate launch` command to run my experiments.
I guess the problem with MoE is that not all GPUs use the same parameters (experts) in one forward pass, and that's why GPUs are waiting for each other to receive all gradients.

### Expected behavior

The gradient syncing and training continue properly, and we are able to train an MoE (sparse network) using DeepSpeed.

Comments:
cc @SunMarc @muellerzr 

Hi @negar-foroutan, Can you give us the code to reproduce the results, it would be very helpful

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@negar-foroutan hi, how do you solve this problem?

@negar-foroutan We're you ever able to solve this problem?
Add option to only install AutoTokenizer for production environment
Author: michaelnny
Pull request: no
Labels: name: Feature request

### Feature request

Add option to only install AutoTokenizer for production environment, so we can minimize the dependencies and footprint in a production system.

### Motivation

When we deploy LLM models to a Triton inference server, we often convert the model to a optimized engine for a specific backend (ONNX, TensorRT-LLM etc.), these optimized engine does not require the transformers library anymore. 

However, we are still required to install the full transformers library, as we need the AutoTokenizer to handle pre/post processing steps to handle tokenization.

### Your contribution

Not sure

Comments:
transformers.Adafactor is almost 2x slower on Windows than Linux - even WSL is slow what can be reason?
Author: FurkanGozukara
Pull request: no
Labels: name: bug

I am training Qwen Image model with Kohya Musubi tuner : https://github.com/kohya-ss/musubi-tuner
 
Exactly same setup and same machine on Linux is almost 2x faster

9.5 second / it vs 5.8 second / it

On Windows it can't utilize GPU power it utilizes like 250 watt out of 575 watt

What can be culprit?

transformers==4.54.1
torch 2.8
CUDA 12.9

tested on RTX 5090

this is what codex tells but i don't know if it is true doesnt make sense to me

<img width="1637" height="736" alt="Image" src="https://github.com/user-attachments/assets/81b687c7-801e-4265-a2fd-6d1eae065637" />

### Who can help?

trainer: @SunMarc 
kernels: @MekkCyber @drbh 




Comments:
Hey @FurkanGozukara, I don't think this is a `transformers` issue, it might be related to your windows system using the gpu driver for display as well as compute (wddm mode) but in linux it's only used for compute (tcc mode). It might also be related to how the OS handle kernel launches, or allocates pageable / non pageable memory, or IO overhead ...

> Hey [@FurkanGozukara](https://github.com/FurkanGozukara), I don't think this is a `transformers` issue, it might be related to your windows system using the gpu driver for display as well as compute (wddm mode) but in linux it's only used for compute (tcc mode). It might also be related to how the OS handle kernel launches, or allocates pageable / non pageable memory, or IO overhead ...

Hey thanks a lot for answer

After doing much more debugging we found this

Do you have any idea how we can improve it?

 https://github.com/kohya-ss/musubi-tuner/issues/685#issuecomment-3446989522

it happens due to RAM - GPU transfer synching

It is way way faster on Linux 



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[Quantization] fix fbgemm
Author: MekkCyber
Pull request: yes

# What does this PR do?

Fixes fbgemm

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42561). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: fbgemm_fp8
Add NVIDIA Cosmos
Author: zucchini-nlp
Pull request: yes
Labels: name: New model

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/35565

Conversion and generation with video only model is tested on 4B checkpoint. Next steps:
- [x] adapt conversion for 11B and 12B models if needed
- [x] verify that video-text generation works as well
- [x] match logits
- [x] add tests


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36476). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ready for review. Note that most parts of the model are similar to existing Emu3 but we cannot copy with modular. Modular can't handle cases with `if/else` or for loops. Thus it was rewritten, making the modular file huge 🥲 

Is this PR still going on? I am willing to help, if updates on this are still needed. @ArthurZucker @zucchini-nlp 

@1himan the model got a bit low in priority and there has been almost no community request to add it. So I put it aside. Do you want to take over the PR? 

> @1himan the model got a bit low in priority and there has been almost no community request to add it. So I put it aside. Do you want to take over the PR?

If there were no community requests, then, no :)
Fix kernel mapping bug when multiple devices exist (Issue #42451)
Author: ada-ggf25
Pull request: yes

# Fix kernel mapping bug when multiple devices exist (Issue #42451)

Fixes #42451

## Summary

Fixes issue #42451 where `add_to_mapping()` would overwrite existing device entries when `kernel_mapping` contained multipl
same layer (e.g., both "cuda" and "rocm").

## Problem

When `kernel_mapping` contains multiple devices for the same layer, calling `add_to_mapping()` repeatedly would overwrite t
entry instead of adding to it. This occurred because the function was completely replacing `compatible_mapping[layer_name]`

**Example of the bug:**
```python
# Before fix: second call overwrites the first
add_to_mapping("RMSNorm", "cuda", "repo:layer", Mode.INFERENCE, compatible_mapping)
add_to_mapping("RMSNorm", "rocm", "repo:layer", Mode.INFERENCE, compatible_mapping)
# Result: only "rocm" exists, "cuda" was lost
```

## Solution

Updated `add_to_mapping()` to check if `layer_name` and `device` entries already exist before overwriting. The function now
1. Initialises the `layer_name` entry if it doesn't exist
2. Initialises the `device` entry if it doesn't exist  
3. Adds the mode entry (which can overwrite if mode already exists, which is fine)

This ensures all devices are preserved in the `compatible_mapping` structure.

## Changes

- **Modified**: `src/transformers/utils/kernel_config.py`
  - Updated `add_to_mapping()` function to preserve existing device entries
  
- **Added**: `tests/kernels/test_kernels.py`
  - `test_add_to_mapping_multiple_devices`: Verifies multiple devices are preserved
  - `test_add_to_mapping_single_device`: Ensures backward compatibility
  - `test_add_to_mapping_multiple_modes`: Verifies multiple modes work correctly

## Testing

All tests pass:
- `test_add_to_mapping_multiple_devices` - PASSED
- `test_add_to_mapping_multiple_modes` - PASSED
- `test_add_to_mapping_single_device` - PASSED
- All existing tests in `TestKernelUtilities` - PASSED



Comments:
cc @mekkcyber

> Thank you @ada-ggf25 ! The issue is being handled here #42466. Sorry about that 🙏

No worries. Happy to help!