noamaanMulla-03/transformers-issues huggingface dataset:
[quantization] Skip Fp8 tests when hardware capability < 8.9
Author: MekkCyber
Pull request: yes

# What does this PR do?

Skips FP8 tests when hardware capability is < 8.9 (nvidia 4090)

It's erroring in the CI with gpus 8.6 : https://github.com/huggingface/transformers/actions/runs/18703774699/job/53337972742

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: finegrained_fp8

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41785). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Swap columns and rows of the grid layout in LFM2-VL
Author: ankke
Pull request: yes

Fixing swapped rows and columns that were resulting in incorrect positional special tokens. Fixed integration tests on the modeling file.

@zucchini-nlp

Comments:
run-slow: lfm2_vl

This comment contains run-slow, running the specified jobs: 

models: ['models/lfm2_vl']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41755). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Interesting, in the runners the batched test gives better generation such as `"In this image, we see a cat and a dog lying on a pink blanket. They are both sleeping peacefully. They are"` . Could then prob be smth with the versions of installed packaged or hardware 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: lfm2_vl
Add safety checking infrastructure for text generation
Author: jameslovespancakes
Pull request: yes
Labels: name: Code agent slop

# What does this PR do?

This PR adds a comprehensive safety checking infrastructure for text generation, enabling built-in content moderation similar to production LLMs (ChatGPT, Gemini, Claude).

## Issue

Open-source text generation models lack built-in safety moderation infrastructure. While production LLMs have safety systems, implementing effective moderation during inference is non-trivial for Transformers users. There's currently no native infrastructure to support this.

Issue: #41740


## Solution

This PR implements:
- **Abstract SafetyChecker base class** for pluggable safety implementations
- **KeywordSafetyChecker** reference implementation with keyword/pattern blocking
- **SafetyStoppingCriteria** to halt generation on safety violations
- **SafetyLogitsProcessor** to filter unsafe token continuations
- **GenerationConfig integration** with automatic processor/criteria instantiation
- **Comprehensive safety parameters** (check frequency, penalty values, violation handling)

## Key Features

- ✅ Automatic integration - just configure GenerationConfig, no manual setup
- ✅ String-based or instance-based safety checker specification
- ✅ Configurable modes (stop on violation, filter violations, or both)
- ✅ Performance tuning via check frequency
- ✅ Extensible architecture for custom safety checkers
- ✅ Batch processing support

## Example Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")

# Automatic safety integration
config = GenerationConfig(
    safety_checker="keyword",
    safety_checker_kwargs={"blocked_keywords": ["unsafe", "explicit"]},
    safety_stop_on_violation=True,
    safety_filter_violations=True,
)

outputs = model.generate(**inputs, generation_config=config, tokenizer=tokenizer)

Comments:
fix type annotation typo in docstring
Author: johntheprime
Pull request: yes

# What does this PR do?

This PR fix type annotation typo in docstring.

Fixes # (issue)


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41788). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix chat schema tests
Author: Rocketknight1
Pull request: yes

My bad - the [chat schema PR](https://github.com/huggingface/transformers/pull/40894) added some tests that were failing after my final commits, and I didn't realize because they weren't running in the CI for the PR.

The main problem was that I dropped support for `"type": "any"`, which is no longer part of the JSON schema standard, but left it in some tests, which caused errors. This PR changes the code to read a missing `type` annotation as implicitly allowing any type, which is compatible with the standard. It also removes a test for Processor schema save/loading, which was dropped from the PR (but will be added soon!)

Comments:
I did!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41793). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix attention mask in mamba layers
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title. It was reported by LFM-VL team that the batched generation is outputting garbage with one of the checkpoints. I found that the masking is not being applied at all for mamba layers

Firstly, mamba layer do not have a 4D attention weight and thus need a normal attention in 2D. Also we do not need to check if the attention has a certain shape, instead we only make sure it is applied in prefill stage

This fixes LFM-VL but Ig all mamba models are affected. I'm still surprised we didn't get issues before and that bigger checkpoints of LFM-VL generated normal text even without proper masking. I'm going to fix other mamba models and tag for review when ready

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41790). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: lfm2, lfm2_moe, lfm2_vl

This comment contains run-slow, running the specified jobs: 

models: ['models/lfm2', 'models/lfm2_moe', 'models/lfm2_vl']
quantizations: [] ...


run-slow: lfm2, lfm2_moe, lfm2_vl



run-slow: lfm2, lfm2_moe, lfm2_vl

This comment contains run-slow, running the specified jobs: 

models: ['models/lfm2', 'models/lfm2_moe', 'models/lfm2_vl']
quantizations: [] ...

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bamba, falcon_h1, granitemoehybrid, lfm2, lfm2_moe, lfm2_vl, mamba2, qwen3_next
upgrade xpu docker file to torch 2.8
Author: yao-matrix
Pull request: yes

@ydshieh , pls help review, thx very much.

pass rate is 92.4% (25446 pass over 27546 total cases), we are working on the issues found during this release cycle and PRs are merged or under development. Thx for great support, XPU pass rate on transformers is improving step by step.


Comments:
@ydshieh , it's OK now, thx very much.

> Thx for great support, XPU pass rate on transformers is improving step by step.

Thank you too, it's most you make it strong 💪 ❤️
Support encoder text classification for sequence to sequence models like BART and T5
Author: cbhyphen
Pull request: no
Labels: name: Feature request

### Feature request

Add support for encoder-only text classification for models like BART and T5.

### Motivation

The current sequence classifiers for BART and T5 include the full encoder-decoder.  While the decoder is used to produce the sentence representation for the classification head, the following paper shows that mean pooling of encoder final hidden states may be a better option (at least for T5):  https://arxiv.org/abs/2108.08877.  In addition to this, not using the decoder would reduce the size of the model and speedup fine-tuning as well as inference.

### Your contribution

PR Submission

Comments:
Hi @cbhyphen,

I’m interested in working on this feature request to add encoder-only text classification support for BART and T5. Could you please assign this issue to me? I’d love to contribute and submit a PR for Hacktoberfest.

Thank you!

Would love to see this!

I'd also find this useful 👍 

hey @Sheshank-singh thanks for your interest!  I have a [PR out here for T5](https://github.com/huggingface/transformers/pull/40898) but feel free to work on BART!

Thank you for letting me know, @cbhyphen ! 
Sure, I’ll start working on the BART encoder-only text classification implementation.

This is interesting. Have some ideas we might discuss. 

I would like to see this

This would be useful to me, but I'd love to see more encoders for sequence classification.
Increasing clarity
Author: FrogWarlord
Pull request: yes

added some missing things to increase clarity


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
🌐 [i18n-KO] Translated selecting.md to Korean
Author: maximizemaxwell
Pull request: yes

# What does this PR do?

Translated the `selecting.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [X] Check for missing / redundant translations (번역 누락/중복 검사)
- [X] Grammar Check (맞춤법 검사)
- [X] Review or Add new terms to glossary (용어 확인 및 추가)
- [X] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [X] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)
<!-- 1. 위 체크가 모두 완료된 뒤에만 KREW 팀원들에게 리뷰를 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
@4N3MONE, @yijun-lee, @jungnerd , @harheem 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
<!-- 2. KREW 팀원들의 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
LGTM! 가능하다면 테이블 안의 텍스트에서는 .(온점)을 빼는게 나을 것 같기는 합니다..!
[quantization] fix torchao tests after 0.14.0 release
Author: MekkCyber
Pull request: yes

# What does this PR do?

Fixes the deprecation of `int4_weight_only` deprecation in torchao>=0.14.0 https://github.com/pytorch/ao/pull/2994

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41777). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

failing tests are due to torchao: 
ERROR tests/quantization/torchao_integration/test_torchao.py - NameError: name 'Int4WeightOnlyConfig' is not defined

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: torchao_integration
flash attn pytest marker
Author: ydshieh
Pull request: yes

# What does this PR do?

A first step toward making flash attn test again 🔥 

@vasqu Do we really need to have both

    @pytest.mark.flash_attn_test

    @pytest.mark.flash_attn_3_test


or we can have only `@pytest.mark.flash_attn_test` which cover both?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41781). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: exaone4, idefics2, ministral, mistral, qwen2, qwen2_5_omni, qwen2_5_vl, qwen2_vl, qwen3, qwen3_omni_moe, qwen3_vl_moe, video_llama_3, autoawq
Bump AMD docker
Author: remi-or
Pull request: yes

This PR bumps the base version of the AMD docker to ROCm 7.0, which means it is compatible with MI355 which was not the case before. 
To avoid issues with detectron, the install is split from the rest of the pip install, and to support MI355 for flash attn, the gfx950 architecture was added to FA compilation.
I tested the docker for Mi325 and Mi355 on important models and there was no major new failures.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41792). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
`use_default_system_prompt` broken
Author: AjayP13
Pull request: no

### System Info

- Python 3.11.4
- 4.34.0

### Who can help?

@younesbelkada @ArthurZucker

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```bash
Python 3.11.4 (main, Jun 20 2023, 17:23:00) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import transformers
>>> transformers.__version__
'4.34.0'
>>> tokenizer = transformers.AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
>>> tokenizer.use_default_system_prompt = True
>>> tokenizer.apply_chat_template([{"role": "user", "content": "Hello"}], tokenize=False)
'

 Hello 
'
```

### Expected behavior

It should return the default LLaMa-2 system prompt (this was just working yesterday, something recently broke):
```bash
'
 
\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n
\n\nHello 
'
```

Comments:
@Rocketknight1 - This seems to stem from a commit pushed to `meta-llama/Llama-2-7b-chat-hf` on HuggingFace Hub: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/commit/af6df14e494ef16d69ec55e9a016e900a2dde1c8

Was this intentional?

@ajayp13 Yes, this was intentional! After testing between us and Meta, people felt that the default system prompt caused LLaMA-2 to refuse too many requests, even totally valid ones, which made it much less useful than it could be.

You can try running it without the default prompt, or with a shorter system prompt of your own - either of those should work!

I see, thanks! Closing.

If anyone else sees this issue and wants to keep using the original prompt, here is the original system message. You can add this as the first message in your conversation with role `system` in order to keep using it:

```
"""You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your \
answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure\
 that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not \
correct. If you don't know the answer to a question, please don't share false information."""
```

Can we put our custom system prompt into chat template whose format is jinja?
Add a safeguard around a flaky test in gemma2
Author: remi-or
Pull request: yes

The test `tests/models/gemma2/test_modeling_gemma2.py::Gemma2IntegrationTest::test_model_2b_pipeline_bf16_flex_attention` is flaky because of the `_compile` flag at https://github.com/huggingface/transformers/blob/fe11cbb808b4301399240e40c5cb6cca9bb00d4d/src/transformers/masking_utils.py#L675
The issue is solved when upgrading torch to nightly 2.10, so it makes little sense to change the code.
But when the test causes an error, it causes all subsequent tests to fail. So this PR adds the `@run_test_using_subprocess` decorator around the test so this does not happen anymore. And a TODO to remove this once the fix is in the stable version of torch.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma2

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41811). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Trainer] remove env vars 
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR removes env var as much as possible from trainer.
Similar to https://github.com/huggingface/transformers/pull/39483/files but with less changes. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41697). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run_slow: trainer

This comment contains run-slow, running the specified jobs: 

models: ['trainer']
quantizations: [] ...

run_slow: fsdp,deepspeed

This comment contains run-slow, running the specified jobs: 

models: ['deepspeed', 'fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...

run_slow: fsdp

This comment contains run-slow, running the specified jobs: 

models: ['fsdp']
quantizations: [] ...