noamaanMulla-03/transformers-issues huggingface dataset:
fix link in kv_cache.md
Author: manueldeprada
Pull request: yes

Fix link reference in cache docs.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37652). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update longformer.md
Author: JihadHammoud02
Pull request: yes

Refactored Longformer docs
Added examples for pipeline, Automodel and cli
Added quantization
Did not add a Attention visualizer, from what I researched it doesn't support it, if it is not the case I am happy to add it !
Added a note concerning versions < 4.37.0.dev

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37622). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Refactor phi doc
Author: JihadHammoud02
Pull request: yes

Refactored the phi.md model card to follow the Hugging Face documentation conventions for model docs.

Reorganized the structure for consistency with other model docs (title, model overview, usage, and notes).

Included usage examples for:

  pipeline (text generation)
  
  AutoModel (text generation with AutoModelForCausal)
  
  transformers-cli (text classification, to diversify usage)

Added general notes about the architecture and tokenizer.

What was left out:
  
  The original documentation included:
  
  Usage instructions for Phi-2
  
  Flash Attention 2 integration tips
  
  A performance graph

I didn’t include those in this version because there wasn’t a clear conventional place for them in model docs. I focused instead on making the file compliant with existing documentation structure.

If you'd like me to include the extra content from the original docs (e.g., FlashAttention setup, performance benchmarks, or Phi-2 usage), I’d be happy to revisit and integrate it in a way that fits well with the current doc style.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Thanks for your return, I will work on it.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37583). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Gemma3ForCausalLM base_model_prefix
Author: calpt
Pull request: yes

# What does this PR do?

The base model attribute for Gemma3ForCausalLM is "model", this updates the base_model_prefix accordingly so `model.base_model` correctly returns the Gemma3TextModel instance.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Ah yes, my bad, didn't think of the consequences for model loading of VL models, thanks for clarifying!
Would be nice to be able to access via `model.base_model` eventually, lmk if I can help anything on refactoring.
Error when tokenizer is set to string: `AttributeError: 'str' object has no attribute 'pad_token_id'`
Author: TrevinAvery
Pull request: no

I'm using the [SageMaker HuggingFace inference toolkit](https://github.com/aws/sagemaker-huggingface-inference-toolkit/blob/main/src/sagemaker_huggingface_inference_toolkit), with the `image-text-to-text` task. I am getting the following error from [this line](https://github.com/huggingface/transformers/blob/2c2495cc7b0e3e2942a9310f61548f40a2bc8425/src/transformers/pipelines/base.py#L1011) in `transformers/pipelines/base.py`:

```AttributeError: 'str' object has no attribute 'pad_token_id'```

This toolkit passes a string for the `tokenizer` parameter to the `pipeline` function. The passed string matches `model_dir` (as seen [here](https://github.com/aws/sagemaker-huggingface-inference-toolkit/blob/5a7519da0ba37895d9e07712124e820c62ec4e56/src/sagemaker_huggingface_inference_toolkit/transformers_utils.py#L272)).

I expect the string to either 1) be used to load a valid tokenizer object, 2) be dropped because it is unused, or 3) throw an error because it is invalid. However, instead, it just passes the string forward to `pipeline_class` (in this case `ImageTextToText`), which does not accept a string for the `tokenizer` parameter.

Suggested fix:

When [evaluating `load_tokenizer`](
https://github.com/huggingface/transformers/blob/2c2495cc7b0e3e2942a9310f61548f40a2bc8425/src/transformers/pipelines/__init__.py#L1023-L1051), if it is false, it should either set `tokenizer` to `None` or raise an exception.

Comments:
Is this happening with Gemma3?

It should have been fixed by recent patch on the release branch, we had a bug where the tokenizer wasn't added to model-mapping

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
`torch.compile` custom backend called by AotAutograd triggers recompiles when used with `CompileConfig`
Author: shaurya0
Pull request: no
Labels: name: bug

### System Info

transformers==4.49.0

### Who can help?

@gante @zucchini-nlp 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

If we use a custom backend with `torch.compile` called by `AotAutograd` as described in: https://pytorch.org/docs/stable/torch.compiler_custom_backends.html#custom-backends-after-aotautograd and use it with the [CompileConfig](https://github.com/huggingface/transformers/blob/9215cc62d4366072aacafa4e44028c1ca187167b/src/transformers/generation/configuration_utils.py#L1584) each call to `generate` will trigger a recompile.
Minimal reproducer:

```
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, CompileConfig
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"
from functorch.compile import make_boxed_func
from torch._dynamo.backends.common import aot_autograd

dtype = torch.float32
model_path = "hf-internal-testing/tiny-random-LlamaForCausalLM"
tokenizer = AutoTokenizer.from_pretrained(model_path, torch_dtype=dtype)
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=dtype)
inputs = tokenizer(["The quick brown "], return_tensors="pt", padding=True)

kwargs = dict(inputs)
kwargs.update(
    {
        "do_sample": False,
        "max_new_tokens": 10,
    }
)
model.generation_config.cache_implementation = "static"


def my_compiler(gm, example_inputs):
    return make_boxed_func(gm.forward)

my_backend = aot_autograd(fw_compiler=my_compiler)

model.generation_config.compile_config = CompileConfig(
    backend=my_backend,
    mode=None
)
model.generation_config.compile_config._compile_all_devices = True


with torch.no_grad():
    for i in range(torch._dynamo.config.cache_size_limit + 1):
        output = model.generate(**kwargs)
    print(output)
    
```

This can be run with `TORCH_LOGS="recompiles"` for recompile logs and will result in an error due to the cache size limit being exceeded. 
If we wrap the call to `aot_autograd` as:
`my_backend = torch._dynamo.disable(aot_autograd(fw_compiler=my_compiler))`

then it runs without recompiles.

### Expected behavior

Recompiles should not be happening and it shouldn't be necessary to disable dynamo with `aot_autograd`.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[fix gemma] Set default value for output_attentions parameter in Gemma2 and Gemma…
Author: chenin-wang
Pull request: yes


# What does this PR do?

Set default value for output_attentions parameter in Gemma2 and Gemma3


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


@ArthurZucker  @zucchini-nlp

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

https://github.com/huggingface/transformers/pull/37609

Why does it work locally without errors, but always fails in CircleCI? @zucchini-nlp 

![image](https://github.com/user-attachments/assets/841a3a77-106e-4993-9151-26a2f72c899c)


@chenin-wang hm, there are changes in unrelated files which is causing red CI because we then need to update all files from where it was copied. Usually it is fixed by `make fix-copies`

I am not sure why it is being modified, prob `ruff` version, can you re-install transformers from `main` ?

@zucchini-nlp Hi, I've found the problem. Can you help me review it?
Restructure torchao quantization examples
Author: jerryzh168
Pull request: yes

Summary:
Mainly structured the examples by hardwares and then listed the recommended quantization methods for each hardware H100 GPU, A100 GPU and CPU

Also added example for push_to_hub

Test Plan:
not required

Reviewers:

Subscribers:

Tasks:

Tags:



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @MekkCyber @SunMarc please take a look

one thing is I'm not sure if people use float8 on CPU, haven't heard of this before actually

@MekkCyber I dropped float8 cpu example since that's not commonly used and doesn't yield speedup according to benchmarks done by @jainapurva

Thanks for the review @MekkCyber and @SunMarc, I addressed the comments and also merged dynamic and weight only quant examples to make it simper, now people mainly see:
```
hardware:
1 example for 8 bit (int8/float8)
1 example for 4 bit (int4)
```
and there will be more details inline, either dynamic/weight only or batch size 1/N
Paligemma: fix generation with Gemma2
Author: zucchini-nlp
Pull request: yes
Labels: name: for patch

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/36029 and adds tests for the model, imo we need tests with different LM backbone because Gemma-2 is special

This is a quick fix but I think we should make this kind of fix on LM work out-of-the-box, by adding it as `kwargs` for example. Most LMs accept `loss_kwargs` thus we can make all multimodal models also accept kwargs that are simply passed further to the LM. WDYT?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36044). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I think making it explicit that kwargs will be used by an only an LM was better

For transparency, this commit needs to be modified for the patch, only applying changes for PaliGemnma2

Hi @zucchini-nlp @ArthurZucker , I find this PR can lead to an abnormal loss value if gradient accumulation is enabled. Initially reported in https://github.com/hiyouga/LLaMA-Factory/issues/7443 , the trainer assumes these model accept loss kwargs because of the existence of `lm_kwargs` (1), but actually they do not (2), resulting in an unexpected loss value. This is also related to the gradient accumulation fix PR #34511 , cc @muellerzr 

1. https://github.com/huggingface/transformers/blob/v4.50.0/src/transformers/trainer.py#L633-L640
2. https://github.com/huggingface/transformers/blob/v4.50.0/src/transformers/models/llava/modeling_llava.py#L460-L464 

> This is a quick fix but I think we should make this kind of fix on LM work out-of-the-box, by adding it as kwargs for example. Most LMs accept loss_kwargs thus we can make all multimodal models also accept kwargs that are simply passed further to the LM. WDYT?

Yeah, it is true that most LMs accept such kwargs, but multimodal models compute the loss themselves, unless we reuse the loss computed by the language model part.

A similar bug report regarding the gemma3 model: https://github.com/hiyouga/LLaMA-Factory/issues/7416

> Yeah, it is true that most LMs accept such kwargs, but multimodal models compute the loss themselves, unless we reuse the loss computed by the language model part.

Yep, I will look into that and find a better solution 

@hiyouga sorry, just got to look in detail at the issue. Indeed this causes problems when accumulating grads. I think the best solution would be to make all VLMs (touched by this PR first and then others not touched also) compute loss with `self.loss_function`

Would you like to submit a PR for that?

@zucchini-nlp Hi, I'm currently deeply occupied with writing my thesis and really don't have much spare time at the moment. Unfortunately, I won't be able to submit the PR. Thanks for understanding!

@hiyouga yeah, no problem :)
Add test to ensure unknown exceptions reraising in utils/hub.py::cached_files()
Author: manueldeprada
Pull request: yes

After fixing #37477 with #37525, a proper test was pending, so we ensure unknown Exceptions are properly re-raised.

Thanks again Cyril for the helpful review and guidance in the previous PR! Let me know if the test is adequate :)

## Who can review?

@Cyrilvallez

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37651). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[test] update `test_past_key_values_format`
Author: gante
Pull request: yes

# What does this PR do?

`tests/models/speecht5/test_modeling_speecht5.py::SpeechT5ForSpeechToTextTest::test_past_key_values_format` was flaky, so a revisit was in order.

This PR rewrites `test_past_key_values_format` so that:
1. It works with `Cache` objects
2. We can pass custom expected cache shapes for advanced models like DeepSeek V3
3. The checks performed are exactly the same as before

In the process, many model-level skips were removed 🤗 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37614). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix ValueError when eval_do_concat_batches=False with examples
Author: jeffhataws
Pull request: yes

https://github.com/huggingface/transformers/issues/37593

# What does this PR do?

Fixes # [37593](https://github.com/huggingface/transformers/issues/37593)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@SunMarc 

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37621). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Qwen2_5Omni training forward issue
Author: Kuangdd01
Pull request: no

I encountered this error when post-training Qwen2_5Omni, I guess it is just a typo.

https://github.com/huggingface/transformers/blob/4e63a1747ce6a4b5f75e8d2318857c2b76c3ba23/src/transformers/models/qwen2_5_omni/modeling_qwen2_5_omni.py#L2505
```
AttributeError: 'Qwen2_5OmniThinkerConfig' object has no attribute 'vocab_size'
```
==>
```python
loss = self.loss_function(logits=logits, labels=labels, vocab_size=self.vocab_size)
```

Another question is whether we can add `self.post_init()` in the initialization of `Qwen2_5OmniForConditionalGeneration`, or I met another error of `model._tp_plan is None` when I tried to train this model with multiple GPUs.
``` python
def __init__(self, config):
    super().__init__(config)

    self.thinker = Qwen2_5OmniThinkerForConditionalGeneration(config.thinker_config)

    self.has_talker = config.enable_audio_output
    self.speaker_map = {}
    if config.enable_audio_output:
        self.enable_talker()
    self.post_init() # Will this line lead to unexpected behaviors?
```

@zucchini-nlp 

## env
```
- `transformers` version: 4.52.0.dev0
- Platform: Linux-5.15.0-134-generic-x86_64-with-glibc2.39
- Python version: 3.10.0
- Huggingface_hub version: 0.30.0
- Safetensors version: 0.5.3
- Accelerate version: 1.4.0
- DeepSpeed version: 0.16.5
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- GPU type: NVIDIA A100-SXM4-40GB
```

Comments:
Oh right, has to be text_config's vocab size. And the `post_init` is also needed, indeed it is missing. Will fix it today
Correct warm-up with fp8
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

This PR starts the work of having fine-grained allocation when using quantizers. For now, it reflects the actual library behavior, while simply adjusting fp8 quantization (for Deepseek), but it should become more general in the future (i.e. probably the default should be 2 if most quantizers correctly pre-process? Did not check all for now), and be overriden only for a few.

cc @SunMarc @MekkCyber 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37670). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[docs] fix bug in quantization docs
Author: gante
Pull request: yes

# What does this PR do?

docbuilder is crashing because we're closing more sections than we open

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

fixed in #37517 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37669). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.