noamaanMulla-03/transformers-issues huggingface dataset:
🌐 [i18n-KO] Translated `bamba.md` to Korean
Author: taemincode
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `bamba.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 KREW 팀원들에게 리뷰를 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
<!-- @jungnerd, @luckyvickyricky, @chelsseeey, @skwh54, @amo33, @maximizemaxwell, @D15M4S -->
<!-- @harheem, @nsbg, @Youngdong2, @xhaktm00, @ssunbear, @ChoHyoungSeo, @judy-choi -->
<!-- @4N3MONE, @Kim-Ju-won, @ahnjj, @FacerAin, @ssum21, @TaskerJang, @HyunZ118 -->
<!-- @yijun-lee, @songi104, @chhaewxn, @AhnJoonSung, @jihyun-0611, @seopp, @pyapyapya -->
@yijun-lee @harheem @4N3MONE @jungnerd

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. KREW 팀원들의 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
[qwen-vl] fix beam search with videos
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, the tests covered only image input so we never noticed the bug. It has been there since model release

Fixes https://github.com/huggingface/transformers/issues/39723


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39726). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4v, qwen2_5_vl, qwen2_vl
[gemma3] update conversion key mapping
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, fixes https://github.com/huggingface/transformers/issues/39763 and ensures Sequence Classification model works with official checkpoints

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39778). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Instantiating `google/gemma-3-4b-pt` with AutoModelForSequenceClassification Reports Unitialized Model
Author: pks
Pull request: no
Labels: name: bug

### System Info

I'm trying to use a Gemma3 model (non-instruction tuned) for a classification task. I was glad that I saw that the model seems to be supported in the current code for this task: https://github.com/huggingface/transformers/pull/39465

When trying
```
model = transformers.AutoModelForSequenceClassification.from_pretrained("google/gemma-3-4b-pt")
```
it essentially reports the model as being uninitialized (it lists both the `vision` and `language_model` weights), which is unexpected:
```
[...], 'model.vision_tower.vision_model.encoder.layers.9.self_attn.out_proj.weight', 'model.vision_tower.vision_model.encoder.layers.9.self_attn.q_proj.bias', 'model.vision_
tower.vision_model.encoder.layers.9.self_attn.q_proj.weight', 'model.vision_tower.vision_model.encoder.layers.9.self_attn.v_proj.bias', 'model.vision_tower.vision_model.enc
oder.layers.9.self_attn.v_proj.weight', 'model.vision_tower.vision_model.post_layernorm.bias', 'model.vision_tower.vision_model.post_layernorm.weight', 'score.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

My `transformers env`:

- `transformers` version: 4.55.0.dev0
- Platform: Linux-6.8.0-60-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 0.34.3
- Safetensors version: 0.5.3
- Accelerate version: 1.9.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.7.1+cu126 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: Yes
- GPU type: NVIDIA RTX A6000

I installed the current `HEAD` (abf101af) of the transformer repo via `uv`.

### Who can help?

@zucchini-nlp
@ArthurZucker

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
model = transformers.AutoModelForSequenceClassification.from_pretrained("google/gemma-3-4b-pt")
```

### Expected behavior

Loaded model with initialized weights.

Comments:
Prob the `conversion_mapping` is not correct or missing. I can take a look a bit later

```
model = transformers.AutoModelForSequenceClassification.from_pretrained("google/gemma-3-4b-pt")
```
Cannot load model either
[Mistral3] attn_implementation not applied to vision_tower.config in Mistral3Config due to init order
Author: starcatmeow
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.55.0
- Platform: Linux-6.8.0-60-generic-x86_64-with-glibc2.35
- Python version: 3.13.6
- Huggingface_hub version: 0.34.3
- Safetensors version: 0.6.1
- Accelerate version: 1.9.0
- Accelerate config:    not found
- DeepSpeed version: 0.17.4
- PyTorch version (accelerator?): 2.7.1+cu126 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: Yes
- GPU type: NVIDIA H100 80GB HBM3

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "mistralai/Mistral-Small-3.1-24B-Instruct-2503",
    torch_dtype="bfloat16",
    attn_implementation="flash_attention_2",
)

print(model.config._attn_implementation)               # 'flash_attention_2'
print(model.vision_tower.config._attn_implementation)  # 'sdpa'
```

### Expected behavior

Both `model.config._attn_implementation` and `model.vision_tower.config._attn_implementation` should match the passed `attn_implementation` argument.

### Cause

In Mistral3Config, `super().__init__` is called before self.vision_config is initialized.

https://github.com/huggingface/transformers/blob/f4d57f2f0cdff0f63ee74a1f16f442dfaf525231/src/transformers/models/mistral3/configuration_mistral3.py#L88

The `super().__init__` call triggers the `_attn_implementation` setter, which attempts to update the vision config — but at this point vision_config does not exist yet, so the update is skipped.

https://github.com/huggingface/transformers/blob/f4d57f2f0cdff0f63ee74a1f16f442dfaf525231/src/transformers/configuration_utils.py#L419

Comments:
Remove deprecated cache-related objects
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title! Both the CacheConfigs and the KeyValuesWrapper were scheduled for deprecation next release

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40035). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Move old generation modes to the Hub 🧹🧹🧹🧽🧽 
Author: manueldeprada
Pull request: yes

This PR cleans a lot old generation strategies in `GenerationMixin` class to prepare for moving several generation modes to a `custom_generate` repository on the Hub: DoLa decoding, contrastive search, group beam search, and constrained beam search.

EDIT: deleting PR to do one at a time

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39887). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Evolla and xLSTM tests
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39769). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: evolla, xlstm

@kpoeppel could you please have a look at the IntegrationTest, and fix the model please? 🤗 See [here](https://github.com/huggingface/transformers/pull/39769/files#diff-3ca56f8c5e2f7022c5f7cac127a2ebd5e92c9829e507fa6e85039bb0b87dd21bR267) in this PR where I had to skip them all for our nightly CI. Also, note that you previously added the wrong layer names in the Layer class, which were not matching your checkpoint [here](https://huggingface.co/NX-AI/xLSTM-7b) - I fixed it in this PR but let's try to be careful! 🤗 This basically rendered the model completely unusable because it would not be loaded correctly!

@kpoeppel Here is the job run page

https://github.com/huggingface/transformers/actions/runs/16535321606/job/46768762210

I see the failure is about

> FAILED tests/models/xlstm/test_modeling_xlstm.py::xLSTMIntegrationTest::test_batched_equivalence_with_cache - ValueError: Cannot instantiate this tokenizer from a slow version. If it's based on sentencepiece, make sure you have sentencepiece installed.

we do have `sentencepiece`  on CI runner, so it looks somehow strange.

But if I don't specify from_slow it works to load the tokenizer.

>>> model_id = "NX-AI/xLSTM-7b"
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained(model_id)



No no I removed the `from_slow`, failure is not due to it but because generation does not match expectation (and is very very weird/bad) so the model needs fixing

cc @kpoeppel 

Thanks @Cyrilvallez I will look into it the next days.

Hi @Cyrilvallez ,
I could pin down a first error, module renaming didn't cause the difference it seems. We had inserted [BOS] tokens manually as xLSTM-7b showed to need those for good evaluation results (they were always present in training). When restricting the inputs to ones with [BOS], all looks good for single sequence inputs. However, for multi-sequence batches, there is still some disparity to disentangle.
🌐 [i18n-KO] Translated `videomae.md` to Korean
Author: jihyun-0611
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `videomae.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `videomae.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 KREW 팀원들에게 리뷰를 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
<!-- @jungnerd, @luckyvickyricky, @chelsseeey, @skwh54, @amo33, @maximizemaxwell, @D15M4S -->
<!-- @harheem, @nsbg, @Youngdong2, @xhaktm00, @ssunbear, @ChoHyoungSeo, @judy-choi -->
<!-- @4N3MONE, @Kim-Ju-won, @ahnjj, @FacerAin, @ssum21, @TaskerJang, @HyunZ118 -->
@yijun-lee, @songi104, @chhaewxn, @AhnJoonSung, @jihyun-0611, @seopp, @pyapyapya

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. KREW 팀원들의 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
Update wavlm.md to match new model card template
Author: reedrya
Pull request: yes

# What does this PR do?
This PR updates the WavLM model card to comply with the format introduced in #36979.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@stevhliu 

## Notes
- I did not include the AttentionMaskVisualizer section since I'm unfamiliar. Please advise if that should be added.


Comments:
Fix `time_spent ` in `notification_service.py`.
Author: ydshieh
Pull request: yes

# What does this PR do?

Related to #40037 which has some bug. The duration could appear as

It could be 

`'71.60s', '(0:01:11)', '====\n'`

or

`'in', '35.01s', '================\n'`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40081). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix: Add missing 'architectures' field to YosoConfig
Author: SankethHonavar
Pull request: yes

This PR adds the missing `architectures` field to the `YosoConfig` class, as part of the issue [#39208](https://github.com/huggingface/transformers/issues/39208).

**Added:**
```python
architectures = ["YosoForMaskedLM"]
```

This helps ensure proper model auto-mapping and improves consistency across configuration classes.

Fixes #39424

Tagging @Narsil as the maintainer of this part of the repo. Please let me know if any changes are needed!


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: yoso

Hi @Narsil, this PR adds the missing `architectures` field to `YosoConfig` (issue #39208). The automated checks passed, and the bot suggests running `run-slow: yoso`.

Let me know if any changes are needed. Thank you!

Hi @SankethHonavar, I can't see any connection between this PR and issue #39208. Did you mean to link a different issue?

> Hi @SankethHonavar, I can't see any connection between this PR and issue #39208. Did you mean to link a different issue?

Thanks for the review, @Rocketknight1

I referred to issue #39208 because it outlines missing `architectures` fields in various model configs. Although `YosoConfig` wasn't explicitly listed, I ran the `add_missing_architectures.py` utility, and it flagged Yoso as missing the field, so I patched it.

Let me know if you'd prefer I open a separate issue for Yoso and link to that instead. Happy to update!


Hi @Rocketknight1, thanks again for the clarification!

I've updated the PR to correctly link to issue #39424 (instead of #39208).  
Let me know if anything else is needed.

Hi @SankethHonavar, I'm still extremely confused. Transformers does not have an `add_missing_architectures.py` utility like you described in your issue. In fact, when I google that filename, the only result is your issue!

Also, no other configuration classes in `transformers` have an `architectures` attribute set at the class level like this, so I'm not sure why we want this or what it fixes!

<img width="697" height="295" alt="image" src="https://github.com/user-attachments/assets/58f02466-c059-49c5-be21-915867e89e8f" />



Thanks @Rocketknight1 for the clarification!

Closing this PR as I now understand that:
- The `add_missing_architectures.py` utility is not part of the Transformers repo
- The `architectures` field should not be added at the class level

Appreciate the feedback --will be more careful next time to align with the repo’s conventions before opening PRs
[`GPT Big Code`] Fix attention scaling
Author: vasqu
Pull request: yes
Labels: name: for patch

Reported internally cc @hmellor @zucchini-nlp 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40041). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: gpt_bigcode

This comment contains run-slow, running the specified jobs: 

models: ['models/gpt_bigcode']
quantizations: [] ...

run-slow: gpt_bigcode

This comment contains run-slow, running the specified jobs: 

models: ['models/gpt_bigcode']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gpt_bigcode

Only some parallel tests fail (as in main). Fixed the old regression tests and added a new one that would only produce newlines before this fix - hopefully, this is a "tougher" test so small breaks are easier to detect before.

Will merge after CI is green
feat: extract rev in attn_implementation kernels via @
Author: drbh
Pull request: yes

This PR adds the ability to specify kernel revisions via the `@` symbol in  in the `attn_implementation` in `AutoModelForCausalLM.from_pretrained`


### Example usage

```bash
uv run repro.py
```

```python
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "accelerate",
#     "torch==2.7.0",
#     "transformers",
#     "kernels>=0.9.0",
# ]
#
# [tool.uv.sources]
# transformers = { git = "https://github.com/drbh/transformers.git", branch = "allow-kernel-rev" }
# ///
import time
import torch

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig


torch.set_float32_matmul_precision("high")

model_id = "meta-llama/Llama-3.2-3b-Instruct"
model = (
    AutoModelForCausalLM.from_pretrained(
        model_id,
        # attn_implementation="kernels-community/flash-attn@main",
        # attn_implementation="kernels-community/flash-attn@56449c1aa267bd0f48a191f0e6979dedf9f2ec32", # (mains sha)
        attn_implementation="kernels-community/flash-attn@09eec95", # main-1 commit sha
        torch_dtype=torch.bfloat16,
    )
    .eval()
    .cuda()
)
tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side="left")

print("+ Model loaded successfully.")

prompt = "What is the capital of France?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
generation_config = GenerationConfig(
    temperature=0.1,
    top_p=0.95,
    top_k=50,
    num_beams=1,
    max_new_tokens=50,
    do_sample=True,
    seed=42,
)
start_time = time.time()
with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        generation_config=generation_config,
    )
end_time = time.time()
print(f"+ Generation time: {end_time - start_time:.2f} seconds")
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
print("+ Inference completed successfully.")
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40009). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix repo consistency
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, I merged a PR without rebasing on `main`

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4v_moe

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40077). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

It looks like there’s a possible typo in the class name [Glm4vMoeisionMlp](https://github.com/huggingface/transformers/blob/1cea763ba422b83778a8db0374ea90f43b09992b/src/transformers/models/glm4v_moe/modeling_glm4v_moe.py#L459).

The `Moeision` part seems unintended — it might have been meant to be `Vision` for consistency and clarity.

Suggested rename:

 ```python
class Glm4vMoeVisionMlp(nn.Module):
```

This would make the naming consistent with related components and reduce confusion for future maintainers.



The PR didn't change class names, and it has a typo from release. Feel free to open a PR and tag me there. We didn't yet have a minor release, so changing the name will not be breaking :)
Inefficient default GELU implementation in GPT2
Author: null-pointer-access
Pull request: no

While profiling the HuggingFace GPT2 model, I found that the default GELU backend used is NewGELUActivation, which is inefficient in most cases. Instead of using a fused CUDA kernel, NewGELUActivation executes multiple separate PyTorch-level operators, leading to unnecessary kernel launches and memory overhead.

```python
# activations.py:L46
class NewGELUActivation(nn.Module):
    def forward(self, input: Tensor) -> Tensor:
        return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0))))
```

Is there a reason why NewGELUActivation is still used as the default for GPT2, rather than switching to nn.functional.gelu or another fused alternative?

I’d be happy to share profiler traces or help test a patch if helpful.

Comments:
No real reasons AFAIK, do you want to open a PR to fix? Having test to make sure results are the same would also be welcome! 🤗 

GPT2 is one of the oldest models!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> No real reasons AFAIK, do you want to open a PR to fix? Having test to make sure results are the same would also be welcome! 🤗

Hi @ArthurZucker ! This is fixed in #40059 . Would you like to have a look at this?