noamaanMulla-03/transformers-issues huggingface dataset:
_supports_static_cache disappear
Author: jiqing-feng
Pull request: no
Labels: name: bug

### System Info

transformers main branch

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I see the attr `_supports_static_cache` disappeared in the model. I used to check if `model._supports_static_cache` before setting `cache_implementation=True`. For now, can I assume all models support static cache?

### Expected behavior

All models support static cache as `_supports_static_cache` is deprecated. Or do we have other method to check if the model support static cache?

Comments:
cc @SunMarc 

Should I use `_can_compile_fullgraph` instead of `_supports_static_cache`?

Yeah, if you want to check if model is compilable the attribute is `_can_compile_fullgraph`. We deleted `_supports_static_cache` because all models from now on support new cache format and thus static cache as well. Now we just need to check if model arch allows fullgraph compilation, because using static cache when generating enables "auto-torch-compile"

Thanks!
[refactor] set attention implementation
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, refactors attention implementation setting and makes it a public API. We should encourage users to `model.set_attn_implementation()` whenever they want to change it after loading the model, instead of setting config's private attr `model.config._attn_implementation="sdpa"`

After the clean-up, we will be calling attention implementation setter only once per pretrained model class, when init the module. Since `from_pretrained/from_config` at the end call `init`, we don't need to keep it as a `classmethod`. Also setting attention after init allows us to know which backbones support attn or do not, and might be useful of we want to early raise errors in the future versions.

Also, removed redundant flags for FA2/FA3. Realized that we can use one flag for both versions :)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38974). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Failing tests are unrelated, ready for review

run-slow: bark, blip_2, instructblipvideo, modernbert, qwen2_5_vl, qwen2_vl, zamba

This comment contains run-slow, running the specified jobs: 

models: ['models/bark', 'models/blip_2', 'models/instructblipvideo', 'models/modernbert', 'models/qwen2_5_vl', 'models/qwen2_vl', 'models/zamba']
quantizations: [] ...

+ removed redundant flags for FA2/FA3. Realized that we can use one flag for both versions :)

We also need to make proper documentation about attentions, I am stuck at finding the right place. Existing docs are a bit dispersed and don't explain much how the API works internally :(

Let's leave it for next PR, might need to change doc structure slightly

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: aimv2, arcee, aria, audio_spectrogram_transformer, aya_vision, bamba, bark, bart, biogpt, bitnet, blenderbot, blenderbot_small, blip_2, chameleon, clip, cohere

(But indeed nice documentation about this seem super important for users, let's keep it in mind somewhere!)

Yep, doc coming in the next PR

This PR does not seem to work as expected. It breaks the default attention mechanism of ModernBERT (it should be fa2 when the user does not specify it, but now it is sdpa). Moreover, this function is not actually called when the model is constructed through class initialization.

@zucchini-nlp @ArthurZucker
BLIPs clean-up 
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Continuation of https://github.com/huggingface/transformers/pull/34502, which I separated out because BLIP wasn't ready to be merged in v4.48 release. The main reason is `Blip2Retrieval` model whose hub configs weren't updated 

Main changes:
- Clean up the `if/else` checks and leave only the new logic with expanded inputs
- Change the `Blip2Retrieval` model class to accept expanded inputs (open to feedback). Currently it expects non-expanded inputs to do ITM so we just crop the inputs removing extra image palceholders
- Opened PRs on the hub for the two checkpoints for Blip2Retrieval, will merge right after this PR
- The tokenizer's "max_length" now includes the image tokens which is breaking but is in line with other VLMs. I would like to keep the change even if breaking to be consistent, or should we wait for v5.0?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35560). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

+1, should have this so we can remove deprecated code. I will resolve conflicts and see if anything else needs to be done

run-slow: blip_2, instructblip, instructblipvideo

This comment contains run-slow, running the specified jobs: 

models: ['models/blip_2', 'models/instructblip', 'models/instructblipvideo']
quantizations: [] ...

run-slow: blip_2, instructblip, instructblipvideo

This comment contains run-slow, running the specified jobs: 

models: ['models/blip_2', 'models/instructblip', 'models/instructblipvideo']
quantizations: [] ...

run-slow: blip_2, instructblip, instructblipvideo

This comment contains run-slow, running the specified jobs: 

models: ['models/blip_2', 'models/instructblip', 'models/instructblipvideo']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: blip_2, instructblip, instructblipvideo

Let's merge it then, I will make a PR to isolate `get_image_mask` in a tiny separate fn
fix cache inheritance
Author: ArthurZucker
Pull request: yes
Labels: name: for patch

# What does this PR do?
Paged attention was broken because it is not layered yet! Needs to be in the patch!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39748). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Fix] import two missing typos in `models/__init__.py` for typo checking
Author: hebangwen
Pull request: yes

# What does this PR do?

I find Pylance in VSCode cannot import the `Gemma3NForConditionalGeneration` and `Qwen2_5OmniModel` automatically. The reason is that these symbols are not exported for type checking. When running in lazy import mode, this symbols will not be exported and thus no type hint.

<img width="741" height="85" alt="image" src="https://github.com/user-attachments/assets/e3901a31-d101-40a1-bc81-24cea04b0e49" />


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker



Comments:
update `GemmaIntegrationTest::test_model_2b_bf16_dola` again
Author: ydshieh
Pull request: yes

# What does this PR do?

Fixes the BC (#39636) invalids a previous PR

Update `GemmaIntegrationTest::test_model_2b_bf16_dola` (#39362)

and the original expect value works again.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39731). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🌐 [i18n-KO] Translated `text-to-speech.md` to Korean
Author: taemincode
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `<your_file>.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 KREW 팀원들에게 리뷰를 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
<!-- @jungnerd, @luckyvickyricky, @chelsseeey, @skwh54, @amo33, @maximizemaxwell, @D15M4S -->
<!-- @harheem, @nsbg, @Youngdong2, @xhaktm00, @ssunbear, @ChoHyoungSeo, @judy-choi -->
<!-- @4N3MONE, @Kim-Ju-won, @ahnjj, @FacerAin, @ssum21, @TaskerJang, @HyunZ118 -->
<!-- @yijun-lee, @songi104, @chhaewxn, @AhnJoonSung, @jihyun-0611, @seopp, @pyapyapya -->
@yijun-lee, @harheem, @4N3MONE, @jungnerd

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. KREW 팀원들의 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
Update IMPORTANT_MODELS list
Author: ivarflakstad
Pull request: yes

Updated according to internal discussion ☺️ 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39734). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Could we move to 

> utils/important_models.txt

and reuse that file in test_fetcher.py ?


> Could we move to 
> 
> 
> 
> > utils/important_models.txt
> 
> 
> 
> and reuse that file in test_fetcher.py ?
> 
> 

Might do that as a follow up PR 👍
Fix mamba regression
Author: manueldeprada
Pull request: yes
Labels: name: for patch

This fixes the sneaky regression introduced in #38086 causing loading errors for falcon_mamba:
```
 "line": "tests/models/falcon_mamba/test_modeling_falcon_mamba.py::FalconMambaModelTest::test_model_from_pretrained",
                "trace": "(line 2593)  RuntimeError: Error(s) in loading state_dict for FalconMambaMixer:"
            },
            {
                "line": "tests/models/falcon_mamba/test_modeling_falcon_mamba.py::FalconMambaIntegrationTests::test_batched_generation",
                "trace": "(line 2593)  RuntimeError: Error(s) in loading state_dict for FalconMambaMixer:"
            },
            {
                "line": "tests/models/falcon_mamba/test_modeling_falcon_mamba.py::FalconMambaIntegrationTests::test_generation_4bit",
                "trace": "(line 2593)  RuntimeError: Error(s) in loading state_dict for FalconMambaMixer:"
            },
            {
                "line": "tests/models/falcon_mamba/test_modeling_falcon_mamba.py::FalconMambaIntegrationTests::test_generation_fp16",
                "trace": "(line 2593)  RuntimeError: Error(s) in loading state_dict for FalconMambaMixer:"
            },
            {
                "line": "tests/models/falcon_mamba/test_modeling_falcon_mamba.py::FalconMambaIntegrationTests::test_generation_torch_compile",
                "trace": "(line 2593)  RuntimeError: Error(s) in loading state_dict for FalconMambaMixer:"
```

The gist of the problem is: modular forces the `super().__init__` call to be on top of FalconMambaConfig. However, before modular rewrite, it was at the bottom, which was critical for the `intermediate_size` property from the config file to take effect.

Second bug fixed: `tests/models/mamba/test_modeling_mamba.py::MambaIntegrationTests::test_compile_mamba_cache ` was failing due to a missplaced import.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39728). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: falcon_mamba

This comment contains run-slow, running the specified jobs: 

models: ['models/falcon_mamba']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: falcon_mamba, mamba

run-slow: falcon_mamba, mamba


This comment contains run-slow, running the specified jobs: 

models: ['models/falcon_mamba', 'models/mamba']
quantizations: [] ...
Use `--gpus all` in workflow files
Author: ydshieh
Pull request: yes

# What does this PR do?

As discussed with infra team

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39752). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
AMD disable torchcodec
Author: ivarflakstad
Pull request: yes

Temporarily disable `torchcodec` in AMD docker image which was introduced in #39669 because of strange segfault error.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39757). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Avoid OOM when other tests are failing
Author: ydshieh
Pull request: yes

# What does this PR do?

Avoid OOM when other tests are failing

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: musicgen_melody

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39758). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix GPT2 with cross attention
Author: zucchini-nlp
Pull request: yes
Labels: name: for patch

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/39746

Comments:
run-slow: gpt2, vision_encoder_decoder

This comment contains run-slow, running the specified jobs: 

models: ['models/gpt2', 'models/vision_encoder_decoder']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39754). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: decision_transformer, gpt2, vision_encoder_decoder

This comment contains run-slow, running the specified jobs: 

models: ['models/decision_transformer', 'models/gpt2', 'models/vision_encoder_decoder']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: decision_transformer, gpt2, vision_encoder_decoder

run-slow: decision_transformer, gpt2, vision_encoder_decoder

This comment contains run-slow, running the specified jobs: 

models: ['models/decision_transformer', 'models/gpt2', 'models/vision_encoder_decoder']
quantizations: [] ...
[XPU] Model get OOM when loading models
Author: Stonepia
Pull request: no

When `caching_allocator_warmup()` trying to allocate memory, XPU lacks of a logic of query the existing memory, so that it gets OOM on:

https://github.com/huggingface/transformers/blob/947a37e8f5bc50bc0e9a77c0d16b038adcb056d0/src/transformers/modeling_utils.py#L6136-L6152

We need to add similar check for XPU as well.

### Example Error Msg
For example, when running 

```
allocate byte_count:  29540067328  factor:  4  byte_count//factor:  7385016832

Traceback (most recent call last):
  File "D:\mint\modelzoo\models_v2\pytorch\llama\inference\cpu\inductor\run_generation.py", line 149, in <module>
    model = model_class[0].from_pretrained(
  File "C:\Users\ai01\miniforge3\envs\ao\lib\site-packages\transformers\models\auto\auto_factory.py", line 571, in from_pretrained
    return model_class.from_pretrained(
  File "C:\Users\ai01\miniforge3\envs\ao\lib\site-packages\transformers\modeling_utils.py", line 279, in _wrapper
    return func(*args, **kwargs)
  File "C:\Users\ai01\miniforge3\envs\ao\lib\site-packages\transformers\modeling_utils.py", line 4399, in from_pretrained
    ) = cls._load_pretrained_model(
  File "C:\Users\ai01\miniforge3\envs\ao\lib\site-packages\transformers\modeling_utils.py", line 4793, in _load_pretrained_model
    caching_allocator_warmup(model_to_load, expanded_device_map, factor=2 if hf_quantizer is None else 4)
  File "C:\Users\ai01\miniforge3\envs\ao\lib\site-packages\transformers\modeling_utils.py", line 5805, in caching_allocator_warmup
    _ = torch.empty(byte_count // factor, dtype=torch.float16, device=device, requires_grad=False)
torch.OutOfMemoryError: XPU out of memory. Tried to allocate 13.76 GiB. GPU 0 has a total capacity of 11.60 GiB. Of the allocated memory 0 bytes is allocated by PyTorch, and 0 bytes is reserved by PyTorch but unallocated. Please use `empty_cache` to release all unoccupied cached memory.
```

### Additional Info:

1. Currently, for BMG, there is a known bug for torch.xpu.mem_get_info() https://github.com/pytorch/pytorch/issues/157989
2. There is a ongoing PR to unify the query API to `torch.accelerator.mem_get_info()` https://github.com/pytorch/pytorch/pull/156812

Comments:
cc @sywangyi 

cc @IlyasMoutawwakil 

cc @yao-matrix 

Hi @Stonepia . Do you mind sharing your case so I can verify if #39654  can fix the issue? Thanks!

Emmm, sry I don't have a small reproducer, but you could try with the `deepseek-ai/DeepSeek-R1-Distill-Qwen-14B` with config on input tokens 1024, output tokens 1024, and try to load bf16 on BMG 12G.
Fix version issue in modeling_utils.py
Author: Cyrilvallez
Pull request: yes
Labels: name: for patch

# What does this PR do?

As per the title. `nn.RMSNorm` was added in torch 2.4, which created friction when calling the function. This fixes it by simply removing it from the `isinstance` check, as the name test is enough anyway and already covers it

Comments:
I guess a comment like:

`# TODO add nn.RMSNorm to isinstance check once support for torch 2.4 is dropped`

could be added, in case we don't want to rely solely on the name, but it's no big deal.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39759). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

module name is actually better as it's otherwise redundant! Because all other modules use a custom RMSNorm. So no need for a comment!
As the failing tests are unrelated, I'm merging this one!
Adding custom 3d mask into ModernBert
Author: bvantuan
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Feature requested by #38040

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@ArthurZucker
<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
**Update**:
The custom 3D mask is supported only for eager and SDPA attention modes.
A test, `test_model_3d_mask_shapes`, has been added to verify the correctness of this feature.

Cc @ArthurZucker ! Could you please review when convenient?
Support loading Qwen3 MoE GGUF
Author: ctcanbol
Pull request: no
Labels: name: Feature request

### Feature request

Currently, GGUF versions of Qwen3 MoE models raises "GGUF model with architecture qwen3moe is not supported yet" error. 

### Motivation

Qwen3 GGUF models with MoE will successfully run.

### Your contribution

This PR resolves this issue: #39638 

Comments:
Fixed with https://github.com/huggingface/transformers/pull/39638
add `libcst` to `extras["testing"]` in `setup.py`
Author: ydshieh
Pull request: yes

# What does this PR do?

So tests like `tests/commands/test_serving.py` and `tests/utils/test_add_new_model_like.py`   in the job `tests_non_model` could run.

This job uses `"huggingface/transformers-torch-light` which has `testing` in the docker file.

(see https://app.circleci.com/pipelines/github/huggingface/transformers/139901/workflows/3cbe7abf-c504-458c-85b8-d7de51ded579/jobs/1853203/tests)



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39761). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.