noamaanMulla-03/transformers-issues huggingface dataset:
Utility script for generating CI reports locally
Author: ahadnagy
Pull request: yes

# What does this PR do?

This PR adds a small util that lets us perform CI runs locally with report generation as it happens during the prod run. This is used for debugging purposes and for debugging new hardware.
Tied to @ivarflakstad's PR: https://github.com/huggingface/transformers/pull/37976 .


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Is this still relevant @ahadnagy ?

There's a newer version that got merges as part of: https://github.com/huggingface/transformers/pull/40978:
I'll close this PR.
[`Kernels Attention`] Change fallback logic to error out on explicit kernels request and include FA3
Author: vasqu
Pull request: yes

As per title. Atm, we just fallback to sdpa/eager even when a kernel was explicitly requested and it caused some hidden states when I ran tests for the bert refactor (not good!). 

Imo, just as any other attention type, we should error out when the explicit request cannot be met. Fallbacks are just more confusing in that case.

Additional change:
- Flash attention 3 was originally not considered in this fallback option. Let's do this here as well (?)

Test via `tests/utils/test_modeling_utils.py -k TestAttentionImplementation` (requires different deps, e.g. kernels + no flash, no kernels + no flash, ...)

cc @ArthurZucker @Cyrilvallez 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41010). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Added some tests and updated some older tests which would be incompatible @ArthurZucker
enable flex attention ut cases on XPU
Author: yao-matrix
Pull request: yes

enable flex_attention test cases on XPU, it's supported by XPU torch now.

@ydshieh , pls help review, thx very much.

Comments:
@ydshieh , could you pls help review? Thx very much.

@ydshieh , could you pls help take a review? Thx very much

the ci error is not brought by this PR


@SunMarc @ydshieh , pls help review, thx very much

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40989). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deepseek_v3, t5gemma, zamba2

@SunMarc , pls feel free to let me know anything need i do before merging, thx very much.
Add EdgeTAM
Author: yonigozlan
Pull request: yes
Labels: name: run-slow

# What does this PR do?

Add [EdgeTAM](https://github.com/facebookresearch/EdgeTAM)

Largely based on Sam2 with modular 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39800). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: auto, edgetam, edgetam_video

Hey @ArthurZucker! Thanks for the reviews, I addressed your comments, it should be ready for another round ;) 

Hey @ArthurZucker, I made most of the fixes you suggested, especially to maximize readability and standardization. It should be ready to merge!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, edgetam, edgetam_video, gemma3n, sam2, sam2_video, timm_wrapper
Fix 8bit bnb loading
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR fixes some code related to 8-bit models that was changed during a recent refactor https://github.com/huggingface/transformers/pull/41138. The issue was that `SCB` value was actually modified when moving the param from cpu to cuda. 

Comments:
run-slow: bnb

This comment contains run-slow, running the specified jobs: 

models: []
quantizations: ['quantization/bnb'] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41200). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: bnb

This comment contains run-slow, running the specified jobs: 

models: []
quantizations: ['quantization/bnb'] ...

Tests are passing !
[WIP] standardize audio kwargs
Author: eustlb
Pull request: yes

# What does this PR do?

This PR standardizes audio-related kwargs naming across the library to ensure future integrations follow consistent best practices.  
A dedicated decorator is introduced to maintain backward compatibility, with an INFO-level log message for visibility.  

> [!NOTE]  
> While renaming `input_features` to `audio_spectrogram` may appear unnecessarily breaking, it is worth noting that `input_features` is **always** used for audio spectrograms throughout the library.  
> Our naming conventions should be explicit and aligned with related libraries (e.g., torchaudio, torchcodec, vLLM, OpenAI API). Let's improve! and ensure we deprecate in a user-friendly manner. 🤗

## Breakdown
### Modeling kwargs
- `input_values` → `audio`
- `input_features` → `audio_spectrogram` (supports classical, mel, log-mel, MFCC)

### Feature extractor kwargs
- `sampling_rate` → `sample_rate`




Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: whisper

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41202). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Embed interactive timeline in docs
Author: yonigozlan
Pull request: yes

# What does this PR do?
As the title says

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41015). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hey @stevhliu and @LysandreJik ! As discussed, here's an attempt to embed the model timeline in the doc (in index.md). Happy to make any modifications needed! I'm using scale(0.8) in the iframe as otherwise the timeline is too zoomed in and difficult to read, and so this needs to be compensated with width/max-width to 125%. Not sure if this is the best way to do this though.
Also added a night theme to the space that (should) follow the theme set in the docs/huggingface hub. 

Really nice! 👏 

I wonder if it may make more sense to give it its own dedicated doc in maybe the Resources section with a brief paragraph about its intended purpose (ie, how do you use it? what do you use it for?, etc.) ? I think adding this extra context would be super helpful. We can link to it from the `index.md` as well to ensure it continues to receive good visibility :)

Some other notes:

- I also find it a bit difficult to read the text when you click on a model due to the smaller font size. Would it be possible to increase that a bit?
- For models at the end of the timeline, when you click on the model card some of the text is truncated. Would it be possible to make the model card scrollable or maybe reposition it?

<img width="900" height="426" alt="Screenshot 2025-09-22 at 10 07 07 AM" src="https://github.com/user-attachments/assets/b3df1921-e7b6-4e65-9588-a19e8db1111b" />


Hey @stevhliu, agreed with all your advice and remarks! I created a separate page for the timeline, and fixed the issues you mentioned in the timeline itself :). Happy to get another review when you have some time!
Ensure testing MobileNetV2 image processor for both slow and fast
Author: namgyu-youn
Pull request: yes

### What does this PR do?

- Remove test skipping for importing `is_torch_available & is_vision_available`, because test result without it doesn't make sense.
- Incline (run directly) unittest for `MobileNetV2ImageProcessor` and `MobileNetV2ImageProcessorFast` 
- Related Issue/PR: https://github.com/huggingface/transformers/pull/40086


### Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


### Who can review?

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface, @SunMarc and @qgallouedec
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->
cc @yonigozlan 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mobilenet_v2
docs: update speech recognition examples
Author: tayo4christ
Pull request: yes

This PR modernizes the ASR examples and improves cross-platform onboarding.

## What’s changed
- **Pin CTC example commands** to Hub datasets (e.g., `mozilla-foundation/common_voice_17_0`) instead of `common_voice` local scripts.
- **Add a Whisper section tip** clarifying that `--dataset_name` should point to a Hub dataset to avoid “dataset scripts are no longer supported” errors.
- **Add a short Windows helper note** (PowerShell venv activation; formatter/linter commands without `make`).
- Minor copyedits for clarity.

## Why
Recent `datasets` versions deprecated local dataset scripts. Using Hub IDs is the supported path and:
- Prevents setup failures across environments (especially Windows),
- Reduces support overhead,
- Keeps docs future-proof and consistent.

## Files touched
- `examples/pytorch/speech-recognition/README.md`
  - CTC commands now use `mozilla-foundation/common_voice_17_0` (retained existing `--dataset_config_name` values).
  - Whisper section includes a one-liner tip about Hub datasets.
  - Added a short Windows (PowerShell) note near the existing OMP note.

## How I tested
- Verified the updated commands on Windows (PowerShell, Python 3.12) in a clean venv.
- Ran Black/Ruff on the changed paths.

## Related
Fixes #40977

## Checklist
- [x] Docs/examples updated
- [x] Format/lint pass on changed files (Black/Ruff)

## Reviewers
@stevhliu (docs) @eustlb (speech)

Comments:
Opened PR #41027 to pin CTC examples to Hub datasets and add a Whisper tip + Windows notes.

@stevhliu @eustlb ready for review — pins CTC commands to Hub datasets, adds a Whisper tip, and Windows notes. Tested on Windows (PowerShell, Python 3.12). Fixes #40977.

@stevhliu @eustlb Changes pushed:
• Removed the “Loading Common Voice from the Hub (recommended)” subsection
• Removed the Whisper tip block
• Aligned dataset_name/dataset_config_name help with Datasets docs
• Kept CTC commands pinned to Hub datasets
Black/Ruff clean. Ready for another look 

@bot /style

Style bot fixed some files and pushed the changes.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41027). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove unnecessary Optional typing
Author: cyyever
Pull request: yes

# What does this PR do?

As the title says

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam2

@ydshieh Sometime there is special handling of `is None` for some Optional types, but I can't find it for the changes.

Actually, `Optional` can be used even without a default value. As official [documentation](https://docs.python.org/3/library/typing.html#typing.Optional) says, `Optional[X]` is equivalent to `Union[X, None]`
They also point out that `def foo(x: int = 0)` does not require `Optional`

@Tialo That is correct. I have checked that the annotated variables in this change don't really have `None` semantics. 

Thanks a lot for the information !

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41198). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Qwen3-Omni audio_token_id serialization issue
Author: goodhee
Pull request: yes

# What does this PR do?

This PR fixes a critical bug in Qwen3-Omni model configuration where `audio_token_id` was not properly preserved during `save_pretrained`/`from_pretrained` operations, causing incorrect inference results in training scenarios.

## Problem

When using `save_pretrained()` followed by `from_pretrained()` on Qwen3-Omni models, the `audio_token_id` value was being reset to the default value (151646) instead of preserving the original value (e.g., 151675), leading to abnormal audio training/inference results.

## Root Cause

The issue was caused by a mismatch between the `attribute_map` in `Qwen3OmniMoeThinkerConfig` and the actual implementation:

- `attribute_map` was mapping `audio_token_id` → `audio_token_index`
- But the modular implementation deletes `audio_token_index` and uses `audio_token_id` directly
- This caused serialization/deserialization to fail silently

## Solution

Remove the conflicting `attribute_map` from `Qwen3OmniMoeThinkerConfig` to allow `audio_token_id` to be serialized directly, matching the actual implementation pattern used in the modular version.

## Verification

- ✅ Tested with real Qwen3-Omni model: `audio_token_id` now preserves correctly (151675 → 151675)
- ✅ Basic configuration tests pass
- ✅ No breaking changes to existing functionality

Fixes #41191

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue? Yes: #41191
- [x] Did you make sure existing tests pass?
- [x] This is a bug fix that doesn't require documentation updates

## Who can review?

@ArthurZucker @SunMarc (text models and configuration)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41192). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[BUG] Qwen3-Omni: save_pretrained causes abnormal audio_token_id, leading to incorrect inference (in training scenario)
Author: Jintao-Huang
Pull request: no



```
MODEL_PATH = "Qwen/Qwen3-Omni-30B-A3B-Instruct"
a = AutoConfig.from_pretrained(MODEL_PATH)
a.save_pretrained('abcd')
b = AutoConfig.from_pretrained('abcd')
print(f'a.thinker_config.audio_token_id: {a.thinker_config.audio_token_id}')
print(f'b.thinker_config.audio_token_id: {b.thinker_config.audio_token_id}')
assert a.thinker_config.audio_token_id == b.thinker_config.audio_token_id
```

<img width="590" height="134" alt="Image" src="https://github.com/user-attachments/assets/dcf4b5d1-9f75-480a-9221-dd380e096d2f" />


https://github.com/huggingface/transformers/blob/071eb5334f5a9ac2c7a13515219be8a272388ec6/src/transformers/models/qwen3_omni_moe/modular_qwen3_omni_moe.py#L272

Comments:
cc @zucchini-nlp, see also the fix at #41192
Deprecate `half_precision_backend`
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR deprecates `half_precision_backend` arg in Trainer. We don't really need cpu and apex backend.
For apex, they even removed the amp folder from their project since they've upstreamed everything to torch a long time ago.
Also the installation process for apex seems way too complicated, so i'm pretty sure there is no real usage. 
For cpu, we were using `torch.autocast` but we don't really need that as this is supported in accelerate. By the way, the fp16 case with cpu was already handled by accelerate.
In the end, this shouldn't impact users at all. I left a deprecation message but I don't think this is really needed. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41134). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

apex can be dropped IMO..

> We can remove the if not isinstance(autocast_ctx, contextlib.nullcontext) branch in compute_loss_context_manager

It's more for backwards compatibility in case someone overwrote `autocast_smart_context_manager` in their codebase

cc @winglian for visibility
Use device agnostic torch-amp
Author: cyyever
Pull request: yes

# What does this PR do?

Before this PR, `torch.autocast` is used only for CPU. It is possible to  extend it to other accelerator types.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: metaclip_2

trainer changes so cc @sunmarc

@SunMarc I have thoughts to unify `cpu_amp`. However, `cpu_amp` is exported as formal API (no deprecation message about it..), so I don't know whether I can remove it in this PR. 

> @SunMarc I have thoughts to unify cpu_amp. However, cpu_amp is exported as formal API (no deprecation message about it..), so I don't know whether I can remove it in this PR.

We can add a deprecation warning and remove it in the next version ! But it would be nice to remove the logic related to cpu_amp as this is already handled by accelerate

@SunMarc Another consideration is that the user may not (want to) install `accelerator`. Do you suggest completely removing the `torch.amp` code and delegate the function to `accelerator`?  I don't think it is a good idea.

actually, trainer have a hard dependency on accelerate, so it should be fine ! 

We have the following ! Feel free to propose a PR 
```
@requires(
    backends=(
        "torch",
        "accelerate",
    )
)
class Trainer:
```

@SunMarc Another question is that `transformers` assumes `accelerate>=0.26.0` but clearly we need a more recent version? Any suggestions to lift the minimum version?

> Another question is that transformers assumes accelerate>=0.26.0 but clearly we need a more recent version? Any suggestions to lift the minimum version?

Why do you need a more recent version ? We can force the user to install a more recent version but we usually do that when there is compatibility issue or we want to simplify a bit the current code. 

I think you saw but I started just deprecating half_precision_backend here https://github.com/huggingface/transformers/pull/41134