noamaanMulla-03/transformers-issues huggingface dataset:
feat: Add Hugging Face Transformers Skill (closes #42971)
Author: cmakinet
Pull request: yes
Labels: name: Code agent slop

# What does this PR do?

Adds a comprehensive Hugging Face Transformers Skill following the Agent Skills Open Standard as requested in issue #42971.


Fixes # (issue)
1. Created a complete Hugging Face Transformers Skill package in `skills/huggingface-transformers/`
2. Includes 6 knowledge files covering:
   - Introduction to Transformers
   - Pipelines API usage
   - Models and tokenizers
   - Training and fine-tuning
   - Real-world examples
3. Follows Agent Skills specification (https://agentskills.io/)
4. Each file is under 500 lines with examples for every concept
## Files Added:
- `skills/huggingface-transformers/SKILL.md`
- `skills/huggingface-transformers/01-introduction.md`
- `skills/huggingface-transformers/02-pipelines.md`
- `skills/huggingface-transformers/03-models-tokenizers.md`
- `skills/huggingface-transformers/04-training.md`
- `skills/huggingface-transformers/05-examples.md`

## Testing:
- [x] Files validated to be under 500 lines each
- [x] Follows Agent Skills Open Standard format
- [x] Examples included with every concept
- [x] Self-contained (no external references)

## Related Issue:
Closes #42971

## Assignment Context:
This PR is part of my open-source contribution assignment (IT8303). I'm a student contributor learning Git/GitHub workflows.

## Checklist:
- [x] I have read the CONTRIBUTING guidelines
- [x] Code follows project conventions
- [x] Documentation is comprehensive
- [x] Files are properly structured


Student: Cherehani Makigo  
GitHub: cmakinet  
OS: Windows 10/11

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.



Comments:
ModernBERTConfig `norm_eps` type hint is incorrect
Author: fschlatt
Pull request: no

The type hint for `norm_eps` is int, but should be float.

https://github.com/huggingface/transformers/blob/78bb85146c59258a0710c8d08311d98d52303c38/src/transformers/models/modernbert/configuration_modernbert.py#L158

Comments:
[`Attn`] Fixup interface usage after refactor
Author: vasqu
Pull request: yes

As per title, the new way to call the attention interface has slipped through a refactor because it's too new and not too well known atm cc @yonigozlan 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43706). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: conditional_detr, d_fine, deformable_detr, detr, pp_doclayout_v3, rt_detr, rt_detr_v2

Yep missed this refactor, thanks for fixing!
remove ipex and ccl for xpu and cpu
Author: yao-matrix
Pull request: yes

remove ipex and ccl for xpu and cpu, since all ipex optimizations are upstreamed to PyTorch or kernels-community; ccl for xpu is upstreamed to PyTorch built-in xccl backend; ccl for cpu will use gloo

Comments:
@jiqing-feng , pls help review, thx.

LGTM.

pending on auto-round issue https://github.com/intel/auto-round/issues/1154 to make the auto-round case pass.

@SunMarc , pls help review, this is to fully remove ccl-backend and ipex from `transformers`, as planed.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42852&sha=3eac2a

the CI failure is not brought by this PR

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42852). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: autoround, gptq
Fix SwitchTransformers creating sparse layer when num_sparse_*_layers=0
Author: jonathan-fulton
Pull request: yes
Labels: name: Code agent slop

## What does this PR do?

Fixes #43335

### Problem

When configuring a SwitchTransformers model with `num_sparse_encoder_layers=0` (intending to have zero sparse layers), a sparse layer is still incorrectly created when `num_layers=1`:

```python
config = SwitchTransformersConfig(
    num_layers=1,
    num_sparse_encoder_layers=0,  # Want 0 sparse layers
    ...
)
model = SwitchTransformersModel(config)
encoder_sparse_count = sum(1 for block in model.encoder.block if block.is_sparse)
print(f"Encoder sparse layers: {encoder_sparse_count}")  # Prints 1, expected 0
```

### Root Cause

In `configuration_switch_transformers.py`, when `num_sparse_encoder_layers=0`:
```python
self.encoder_sparse_step = self.num_layers  # HACK: this will create 0 sparse layers
```

This works for most cases, but fails when `num_layers=1` because:
- `encoder_sparse_step = 1`
- In the modeling code: `is_sparse = (i % sparse_step == 1 or sparse_step == 1) if sparse_step > 0 else False`
- When `sparse_step == 1`: `is_sparse = True` (because of the `sparse_step == 1` condition)

### Solution

Set `*_sparse_step = 0` instead of `num_layers` when no sparse layers are desired. The modeling code already handles `sparse_step=0` correctly:
```python
is_sparse = ... if sparse_step > 0 else False  # sparse_step=0 → always False
```

This is a minimal change that correctly handles all edge cases.

## Who can review?

@Rocketknight1 @sayakpaul

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: switch_transformers

Already explained in the issue that we don't need PRs for this! Hugging Face is an AI company who are perfectly capable of running code agents to solve issues when we need to. Running your code agent on loads of issues you haven't read just spams us and wastes maintainer time!

You're absolutely right, and I sincerely apologize for the noise. 

I'm experimenting with how effective AI agents can be at contributing to open source, and clearly I need to be much more careful about:
1. Actually reading issues thoroughly before submitting PRs
2. Not spamming maintainers with low-quality contributions
3. Respecting that HuggingFace has their own capable engineering team

The good news is this experiment has taught me a lot about what NOT to do. I'll be much more selective going forward - focusing only on clear, reproducible bugs where I can add genuine value.

Again, sorry for wasting your time. Closing this PR.
Docs: fix typo in weight converter guide
Author: KOKOSde
Pull request: yes

## Summary
Fix a spelling typo in the dynamic weight loading / weight converter documentation.

**Change**: `formated` → `formatted` in `docs/source/en/weightconverter.md`

## Test plan
- N/A (documentation-only change)

Made with [Cursor](https://cursor.com)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43610). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add Promptable Visual Segmentation pipeline
Author: yonigozlan
Pull request: yes

Add pipeline for sam/sam2/edgetam/sam3_tracker task

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43613). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, edgetam, sam, sam2, sam3_tracker
Composite model inherit automatically all important properties from their children
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. All those important static properties (`_keep_in_fp32_modules`, `_no_split_modules`, etc) need to be determined automatically in the case of composite models.
This follows the same approach I previously added for parallelism plans et `tied_weights_keys`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43691). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: deformable_detr, grounding_dino, instructblip, instructblipvideo
Update doc preprocessing regex to prevent ReDoS
Author: WilliamRoyNelson
Pull request: yes

# Update doc preprocessing regex to prevent ReDoS

The regular expression for capturing docstrings is vulnerable to a [ReDoS attack](https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS)

The previous change dramatically improved performance, but is still vulnerable to the pattern:
```python
data= "```py" + ("\n" * i)
```

As i grows, the processing time increases in greater than linear time:
```
Execution time: 0.156596 s , iteration (i=10000)
Execution time: 0.615783 s , iteration (i=20000)
Execution time: 1.404632 s , iteration (i=30000)
Execution time: 2.502577 s , iteration (i=40000)
Execution time: 4.122107 s , iteration (i=50000)
Execution time: 6.003960 s , iteration (i=60000)
Execution time: 7.687220 s , iteration (i=70000)
Execution time: 10.054976 s , iteration (i=80000)
Execution time: 12.749450 s , iteration (i=90000)
Execution time: 17.755118 s , iteration (i=100000)
Execution time: 20.584845 s , iteration (i=110000)
Execution time: 24.046818 s , iteration (i=120000)
Execution time: 28.025028 s , iteration (i=130000)
Execution time: 31.564563 s , iteration (i=140000)
Execution time: 36.151067 s , iteration (i=150000)
Execution time: 40.400584 s , iteration (i=160000)
Execution time: 50.993116 s , iteration (i=170000)
Execution time: 66.691314 s , iteration (i=180000)
Execution time: 73.058775 s , iteration (i=190000)
Execution time: 81.073598 s , iteration (i=200000)
Execution time: 89.698897 s , iteration (i=210000)
Execution time: 98.433263 s , iteration (i=220000)
```

The replacement regex, `(```(?:python|py)\s{0,100}\n\s{0,100}>>> )(.*?```)` allows a **generous** 100 whitespace characters after the word python/py, and up to 100 whitespace characters before the `>>>`

This supplements https://github.com/huggingface/transformers/pull/36648

Fixes CVE-2025-2099
 https://nvd.nist.gov/vuln/detail/CVE-2025-2099



I found this tool useful for generating evil strings: https://devina.io/redos-checker
It couldn't seem to find a pattern that can break the new regular expression.


cc @ArthurZucker : Original Author (I think) https://github.com/huggingface/transformers/commit/2fdf0ef3ef3fa57145e8c98828d105143177d42d

@Rocketknight1 @ydshieh Wrote/Reviewed https://github.com/huggingface/transformers/pull/36648 and are hopefully familiar with the issue.


Comments:
Or maybe
`(```(?:python|py)[^\S\n]*\n\s*>>> )(.*?```)` instead?
I don't know the best way to do it.

That vulnerability is silly because this regex is run only on docstrings in our own codebase, lol.

Some people might say "Matt, didn't you approve a PR to fix that before?" To them I say "Yes. I too used to be silly."

For a minute, I was disappointed that this was closed, but I think the explanation "this regex is run only on docstrings in our own codebase, lol." will be useful in ameliorating security concerns if they come up. Thank you!
Account for custom trainers when trying to estimate the number of FLOPS
Author: qgallouedec
Pull request: no

The Trainer estimates the number of FLOPs using the number of elements in the input tensor associated with the key `"input_ids"`. However, in most custom trainers, the sampled data does not include the `"input_ids"` key. For example, for GRPO, the available key is `"prompt"`. As a result, the trainer issues the warning:

```
Could not estimate the number of tokens of the input, floating-point operations will not be computed.
```

It adds noise for a expected behaviour, and the only way we found to suppress this warning is to set the `"estimate_tokens"` key in the model's `"warnings_issued"` dictionary to True. Which is a bit hacky and fragile

https://github.com/huggingface/trl/blob/7a530ba6d2ebf4101aaeb8002cd51c9bde4fb721/trl/trainer/grpo_trainer.py#L528-L534


Comments:
Not needed anymore, see https://github.com/huggingface/trl/pull/4960
Added S110 - try-except-pass rule
Author: tarekziade
Pull request: yes

# What does this PR do?

- adds Bandit's S110 that makes sure we don't have a dry `Except`
- fixes all occurrences
- mark a couple of spots where we could tighten the `Exception` catch all

I focused on making changes under `src/transformers` keep the same behavior

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43687). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: auto, dab_detr, rt_detr

This comment contains `run-slow`, running the specified jobs: 

models: ["models/auto", "models/dab_detr", "models/rt_detr"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21637478192)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [5e6e55a9](https://github.com/huggingface/transformers/commit/5e6e55a9be0326f969a866fd94601bdab7d2583e) | merge commit |
| PR | [754f60e6](https://github.com/huggingface/transformers/commit/754f60e609fdc3f88c12f844d5eecc4773222880) | branch commit |
| main | [01e860eb](https://github.com/huggingface/transformers/commit/01e860ebc6b827c88e2d75e70864d1b618364653) | base commit |

✅ No failing test specific to this PR 🎉 👏 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, dab_detr, rt_detr
Update configuration_qwen3.py
Author: francesco-bertolotti
Pull request: yes

wrong `rms_norm_type`

# What does this PR do?

Small type error in the configuration of qwen3. `rms_norm_eps` should be a float and not an int.

## Before submitting
- [ X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @Cyrilvallez @stevhliu

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3

Thank you for your work on this great repo!
[Kernels] kernel migration updates for activation kernels
Author: ariG23498
Pull request: yes

CC: @MekkCyber 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43518). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

>  does this work with previous kernels versions too or only the latest ?

I am using the latest one.

Do you think we should add a test for previous kernel version?

Does this pr fail with previous kernels versions ?

> Does this pr fail with previous kernels versions ?

I could not give it a try, do you want me to try and revert back?

Yes I think it would be cool to test with other versions before merging this 🤗 

@MekkCyber running with `v0.12.1` and `v0.11.7` on the following
```
from transformers import AutoModelForCausalLM

model_id = "openai/gpt-oss-20b"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype="auto",
    device_map="auto",
    use_kernels=True,
)
```

resulted in proper downloading of kernels. (I had also removed the hf cache before running them sequentially)

@MekkCyber I think the failing CI is due to other issues (unrelated to the PR)

Let me know if there is something else I need to work on! 🤗 

cc @ArthurZucker ! if you can merge please 🤗 thanks !
Improve typing/explanations for general model properties
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. See https://github.com/huggingface/transformers/issues/42832#issuecomment-3840648709 where it was pointed out

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43712). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
🚨 Generation cache preparation
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

I also want to see if linear cache thing can be squeezed in this PR. If it requires big diffs, I'll split into two

Fixes https://github.com/huggingface/transformers/issues/43673

Sidenote: kinda breaking but in a good way. Prev models init their cache in `model.forward` without passing `config`. That means sliding windows weren't always respected (e.g. Afmoe or remote models). From now on we always respect sliding window if it is in config

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43679). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Remark: kinda breaking but in a good way. Prev models init their cache in model.forward without passing config. That means sliding windows weren't always respected (e.g. Afmoe). From now on we always respect sliding window if it is in config

Added a 🚨 


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: afmoe, blt, dia, janus, kyutai_speech_to_text

@vasqu can you force merge, CI is super flaky still and I already retriggered 5-6 times 😢
GenerationMixin cache missing in v5.0.0 during chunked_prefill
Author: SmerkyG
Pull request: no
Labels: name: bug

### System Info

The removal of v4.57.6's default insertion of DynamicCache https://github.com/huggingface/transformers/blob/v4.57.6/src/transformers/generation/utils.py#L2034
vs
https://github.com/huggingface/transformers/blob/v5.0.0/src/transformers/generation/utils.py#L2042
has made it so that in v5.0.0 chunked_prefill no longer works, complaining of 
`Cannot use prefill chunking without a cache`
due to line
https://github.com/huggingface/transformers/blob/v5.0.0/src/transformers/generation/utils.py#L3862
which checks that a cache exists. But per the old comment:
```
        # TODO (joao): remove this `else` when we remove the last traces of the legacy cache format (v4.58.0, search
        # for `instance(past_key_values, Cache)` as well). In general, if `cache_implementation` is unset, cache
        # initialization should happen inside the model at prefill time.
```
the cache no longer should exist until the model is first called.


I'm not sure what the most proper fix is, but I would suggest maybe testing for `if "past_key_values" not in model_kwargs and not model_kwargs.get('use_cache'):` instead?

### Who can help?

 @gante 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

use any model with chunked_prefill but do not pass an initial cache object, just use_cache=True

### Expected behavior

should work, but gives the error listed

Comments:
Indeed! I am not sure what was the plan for cache though I don't agree that initializing default cache should happen inside model's `forward` call. I think the best solution is to bring it back, lemme see
Fix dtype in image-text-to-text pipe
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/43716, as per title

We didn't see errors even if we test `bf16` because most backbone models (e.g. SigLIP) cast pixels to target dtype inside the model code. I fixed it in pipe and also in pixtral model following siglip pattern

Comments:
run-slow: pixtral, mistral3

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: pixtral

💔 This comment contains `run-slow`, but unknown error occurred and [the workflow run](https://github.com/huggingface/transformers/actions/runs/21665982086) aborted!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43731). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I don't think it's really needed because the fix is now also inside `model.forward`. So even if we accidentally "break" the pipe in the future, the test won't catch anything 🙃 

The whole pattern of casting inputs inside model to the correct `dtype` is a bit non-consistent and I don't know why it came to exist in the first place. We gotta figure out if we keep it and update all models, or ask users to always cast to dtype manually

Gotcha, no worries. Asking users to do something won't end well imo (90% ignore instructions like these)
Assume torch in certain files
Author: cyyever
Pull request: yes

# What does this PR do?

Always import torch in some source files that are PT only.

Comments:
This is definitely something that'll need a core maintainer, since it's part of the transition to being Torch-only. cc @arthurzucker @Cyrilvallez !

In general we don't want to assume torch, only in torch code! A lot of people just use the config or just the tokenizer etc