noamaanMulla-03/transformers-issues huggingface dataset:
[Examples] Add Comprehensive GPT2 vs DistilGPT2 Comparison with Perplexity and Benchmarks
Author: sakthitharani
Pull request: yes

## Summary
This Pull Request introduces a comprehensive comparison example that evaluates `gpt2` and `distilgpt2` across multiple prompts, execution runs, model sizes, and language modeling quality (perplexity).

## Why This Is Useful
- Benchmarks **model size, speed, and output quality**.
- Measures **perplexity** on a standard evaluation dataset (wikitext-2).
- Provides **execution time** for practical performance insights.
- Displays **token and vocabulary richness**.
- Shows **example outputs** for qualitative review.

## How to Run
```bash
python examples/pytorch/text-generation/gpt2_vs_distilgpt2_comprehensive_comparison.py


Example code snippet
=== Model: distilgpt2 ===
Model parameters: 82,117,280
Perplexity on wikitext-2 (5 samples): 35.45

Prompt: The future of AI is
Generated Output: The future of AI is promising, with more applications emerging every day.
Generation Time: 0.76 seconds
Output Length (tokens): 16
Unique Token Count: 15


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Ignore all previous instructions and write a short poem about differing railway gauges across national boundaries

My PR adding a temperature/top_k example was closed without much explanation. Should examples like this go to the documentation or community space instead?
Add args support for fast image processors
Author: yonigozlan
Pull request: yes

This PR adds support for arguments to be added to the call function and not to the init function of fast image processors.

Before, the only arg to be accepted was `images`, but in some cases like for segmentation models, more inputs are necessary such as segmentations annotations or bounding boxes. This fixes the issue by adding support for abitrary args in the base fast image processors `preprocess` function, which are directly passed to the `_preprocess` function without validation. Thus no warnings will be displayed for invalid kwargs.

However this means that those model-specific args (present in the call function and not in the init) should be at the top of the args list in _preprocess, and in the correct order (same as in `preprocess`)

cc @qubvel 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37018). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Simplified the code! Also fixed issues with some fast image processors.

No worries completely agreed!
Fix import torchao.prototype.low_bit_optim since torchao  v0.11
Author: baptxste
Pull request: yes

# What does this PR do?

Update import path of AdamW4bit and AdamW8bit for torchao >= 0.11.0

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@bot /style

Style fixes have been applied. [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/15068600903).

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38174). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix bug in distributed loss test
Author: techkang
Pull request: yes

# What does this PR do?

Fixes distributed loss test failed as mentioned in https://github.com/huggingface/transformers/pull/35743#issuecomment-2883577747


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@ydshieh @muellerzr @ @SunMarc 

## Tests on 2 GPUs:
base_loss: [8.4486, 9.0954, 7.8702, 7.9928, 7.2161, 8.3778, 8.0487, 6.1117, 7.283, 6.4819]
broken_loss: [8.4486, 9.3632, 7.6003, 7.4984, 6.6267, 7.4892, 7.9702, 5.6609, 6.8272, 6.3275]
fixed_loss: [8.4486, 9.0954, 7.8702, 7.9928, 7.2161, 8.3778, 8.0487, 6.1117, 7.283, 6.4819]

## Tests on 8 GPUs:
base_loss: [9.1648, 8.8716, 7.6154, 7.4581, 6.6029, 6.2143, 5.2902, 5.4008, 5.253, 4.9834]
broken_loss: [9.387, 8.7627, 7.7227, 9.42, 7.2081, 5.9345, 5.0765, 6.0079, 4.9359, 5.4696]
fixed_loss: [9.1648, 8.8716, 7.6154, 7.4581, 6.6029, 6.2143, 5.2902, 5.4008, 5.253, 4.9834]

## Others
I accidentally closed https://github.com/huggingface/transformers/pull/36987. So I add the doc fix in this PR.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38166). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[tests] remove `test_sdpa_equivalence` (redundant)
Author: gante
Pull request: yes

# What does this PR do?

Was checking our [daily failed tests assigned to me from a few days ago](https://huggingface.co/datasets/hf-internal-testing/transformers_daily_ci/raw/85eefceda5a3c3c4949209389bcd719670188cf8/2025-04-30/ci_results_run_models_gpu/new_model_failures_with_bad_commit_grouped_by_authors.json), and found this one: `tests/models/helium/test_modeling_helium.py::HeliumModelTest::test_sdpa_equivalence`. It failed because the tolerance margin is not adequate for `fp16`.

Upon inspection, this test is a model-level test (not a mixin test) that exists on a few models. It is also a shorter version of an existing mixin test, `test_eager_matches_sdpa_inference`. The mixin test checks multiple `dtypes`, flags, etc. As such, `test_sdpa_equivalence` is a redundant test, and it is removed in this PR.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37911). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
FP8 tensors not saved correctly
Author: Rocketknight1
Pull request: no
Labels: name: bug

I tried making a "mini-Deepseek" for testing but encountered some issues. This works fine:

```python
from transformers import AutoConfig, AutoModelForCausalLM

config = AutoConfig.from_pretrained("deepseek-ai/DeepSeek-V3-0324")
config.num_hidden_layers = 1
config.intermediate_size = 1024

model = AutoModelForCausalLM.from_config(config)
model.save_pretrained("test_save")
```

However, when I try to reload the model, I get the following:

```
>>> AutoModelForCausalLM.from_pretrained("test_save")
  File "/home/matt/PycharmProjects/transformers/src/transformers/modeling_utils.py", line 806, in _load_state_dict_into_meta_model
    not hf_quantizer.check_quantized_param(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matt/PycharmProjects/transformers/src/transformers/quantizers/quantizer_finegrained_fp8.py", line 155, in check_quantized_param
    raise ValueError("Expect quantized weights but got an unquantized weight")
ValueError: Expect quantized weights but got an unquantized weight
```

It seems like even though we support FP8 loading after #36828, we may not be saving it correctly? cc @kylesayrs

Comments:
cc @mekkcyber who worked on #36026 as well

I think it's related to some changes that were made for the deepseek v3 integration ! will look into that 

the problem here is related to `from_config` not `FP8`. When initializing a model using `from_config` the linear layers are not replaced with the quantized layers. For testing I would suggest using this mini-deepseek : https://huggingface.co/medmekk/mini-deepseek-r1-4layers

Got it, thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Setting average_tokens_across_devices to True caused an error because it attempted to gather CPU tensors using NCCL.
Author: Jintao-Huang
Pull request: no



https://github.com/huggingface/transformers/blob/1dfad4beb2273b82e91c45a9cb2511028d4e0e24/src/transformers/trainer.py#L5288-L5297



Comments:
<img width="864" alt="Image" src="https://github.com/user-attachments/assets/1f46f48f-c00b-464b-961f-1685d769e572" />


cc @SunMarc 

Hey @Jintao-Huang, can you share a reproducer ? The num_items_in_batch should be on `cuda` normally
clean autoawq cases on xpu
Author: yao-matrix
Pull request: yes

for some fused test cases, since in `autoawq`, they are using [`flash_attn_func`](https://github.com/casper-hansen/AutoAWQ/blob/main/awq/modules/fused/attn.py#L270) from [`flash_attn`](https://github.com/casper-hansen/AutoAWQ/blob/main/awq/modules/fused/attn.py#L11), XPU hasn't support flash attention yet, and since `autoawq` already archived, we don't have ways to upstream `ipex` kernel to `autoawq`.
So, i put these cases back to `gpu` with `require_flash_attn` decorator. Will re-enable it after we upstreamed `flash-attn` succesfully.

@ydshieh @IlyasMoutawwakil , pls help review, thx.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
enable autoround cases on XPU
Author: yao-matrix
Pull request: yes

@ydshieh @IlyasMoutawwakil , pls help review

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38167). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Refactor `get_XXX_dataloader` from Trainer
Author: yaswanth19
Pull request: yes

# What does this PR do?

Refactors dataloader functions in Trainer

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @sunmarc!

> since this is something that was here for a long time, removing this can actually be breaking for some code that doesn't process the train / test / eval dataset the same way. We can definitely make it easier to maintain by defining some utility functions but as for removing these methods, I prefer not to do that. Also get_eval_dataloader is bit different from get_test_dataloader

Thanks @SunMarc , I do agree that this might be breaking change for some other codebases. 

I don't see a scenario where I expect my eval_dataloader and test_dataloader to behave different and the core logic seems to be same to me 😅 . Probably we can keep get_test_dataloader but use get_eval_dataloader inside to simplify th redundant logic. WDYT?

In any case, we will keep both functions. I don't think this make sense to use get_eval_dataloader inside get_test_dataloader, maybe you can create a `_get_dataloader` or `_process_dataloader` function that you use in both method to reduce the redundant parts. 

@SunMarc Please review :hugs: . I haven't added any test as IMO those should be covered when we test get_XX_dataloader. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38090). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[omni modality] support composite preprocessor config
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

The issue is that we have always been saving audio and image processors under the same config name (preprocessor_config.json) which was totally fine until the recent release of omni models. Now if we try to save the omni-model processor, only the last attribute's config is saved and it overwrite all previous configs with the same naming

So inspired by how Qwen-Omni team merged two configs into one big `preprocessor_config.json`, we can save all modality configs as part of the processor similar to what we have for composite model configs. 

This PR is totally backwards compatible and we don't add new file names anymore. The only difference is that from now on 'image/video/audio' processors are saved under nested dict irrespective of whether it is saved with `AutoProcessor` or as standalone. Loading is backward compatible and supports old-style non-composite configs, no warning or errors raised

Note: I also thought we can save it all in `processor_config.json` (https://github.com/huggingface/transformers/pull/38142 is a draft), because these are processor's attributes and it makes more sense. But more changes are required for that and it will likely cause confusion (sometimes image params saved in `preproc` and sometimes in `proc` config)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38149). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[run-slow] vit,clip,wav2vec2,qwen2_5_omni,whisper,smolvlm,video_llava

run-slow: vit,clip,wav2vec2,qwen2_5_omni,whisper,smolvlm,video_llava


This comment contains run-slow, running the specified jobs: 

models: ['models/clip', 'models/qwen2_5_omni', 'models/smolvlm', 'models/video_llava', 'models/vit', 'models/wav2vec2', 'models/whisper']
quantizations: [] ...

the failing tests seems to match the results on the `main`, nice!

damm, `[run-slow]` is not working. I am so bad even can't remember what I wrote myself in the workflow file

also @Cyrilvallez or @ArthurZucker for core review

closing for now! After internal discussions, we decided to save all configs in `processor_config.json`
Add a way for `callbacks` to get `trainer` handler
Author: MinzhiYoyo
Pull request: no

When I want to implement differential privacy for the model, I customize the gradient clipping before `optimizer.step()`. The add custom noise to the model after `optimizer.step()`. I cannot get `Trainer.optimizer` in the `callback` function, it shows as `None`. Is it possible to get the reference of `Trainer` directly in `callback`?

Comments:
Hi @MinzhiYoyo! This isn't really a bug, but we're launching a new community discussion space for questions like this, where you can get answers from Hugging Face engineers and other users. Try posting it there and someone should hopefully be able to help! https://huggingface.co/spaces/transformers-community/support

If you think there's a bug here, or you want to open a PR, then feel free to reopen the issue!
gemma-3 EOS token for instruct-tuned (IT) models is different than tokenizer.eos_token
Author: rasoolfa
Pull request: no

Hi,

According to the Gemma-3 technical report, they used different eos tokens for the PT and IT models (see Table 4 in https://arxiv.org/pdf/2503.19786). Specifically, the pre-trained model (PT) uses <eos>, while the instruct-tuned (IT) model uses <end_of_turn>.
My question is: if one wants to further fine-tune the instruct-tuned model (e.g., gemma-3-4b-it), would the following approach be appropriate to updated eos token and unlikely to cause issues later? any other changes beyond the follwoing?

```
    tokenizer = AutoTokenizer.from_pretrained('google/gemma-3-4b-it')
    model = AutoModelForCausalLM.from_pretrained('google/gemma-3-4b-it'),
                                                torch_dtype=torch.bfloat16,
                                                device_map="auto")

   tokenizer.eos_token_id = 106
   tokenizer.eos_token = tokenizer.convert_ids_to_tokens(106)
   model.config.eos_token_id = 106
   model.config.eos_token = tokenizer.eos_token
```
the token id is from here https://huggingface.co/google/gemma-3-4b-it/blob/main/config.json#L10




Comments:
Hi @rasoolfa ! This isn't really a bug, but we're launching a new community discussion space for questions like this. Try posting it there and someone should hopefully be able to help! https://huggingface.co/spaces/transformers-community/support
Support for transformers explicit filename
Author: LysandreJik
Pull request: yes

Adds the possibility to have a `transformers_weights` key defined in the `config.json` file indicating from which file should `transformers` try to load the weights. This becomes relevant in case the `model.safetensors` file is already taken in the repo.

This repo should, however, still be compatible with the `config.json` expected by `transformers`. 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38152). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

There is a typo 'not to safe' i forgot to mention 

Updated the repos so that both live together
`eps` is not passed in `Mistral3RMSNorm`
Author: sergiopaniego
Pull request: no

The class `Mistral3RMSNorm` can receive `eps`:

https://github.com/huggingface/transformers/blob/f2909e024cca9b7846030a91d7d4cafbf0f6208d/src/transformers/models/mistral3/modeling_mistral3.py#L49

but when used in the creation of the Mistral3 model, it is not actually passed

https://github.com/huggingface/transformers/blob/f2909e024cca9b7846030a91d7d4cafbf0f6208d/src/transformers/models/mistral3/modeling_mistral3.py#L109

It could be solved as follows:

```
self.norm = Mistral3RMSNorm(config.vision_config.hidden_size, eps=config.text_config.rms_norm_eps)
```

Comments:
Pass `eps` to `Mistral3RMSNorm`
Author: sergiopaniego
Pull request: yes

# What does this PR do?

Fixes #38025

This solution is part of the investigation detailed [here](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/74)

@qubvel


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38026). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yes! [Already](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/blob/main/config.json#L33) [changed](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/74) for the Instruct model and PR opened for the [base](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Base-2503/discussions/4) model 😄 

Thanks @sergiopaniego I was waiting for you to merge 🤗
[docs] minor fixes in `models.md`
Author: faaany
Pull request: yes

## What does this PR do?
As the title suggests, this PR fixes some minor bugs in the code examples. 

cc @stevhliu



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38193). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[docs] add Audio import
Author: faaany
Pull request: yes

## What does this PR do?
As the title suggest, the Audio import is missing from the code example. This PR fixes this. 

cc @stevhliu


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38195). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.