noamaanMulla-03/transformers-issues huggingface dataset:
Add FSDP plans to all models
Author: 3outeille
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47165)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47165)
<!-- ci-dashboard-badge:end -->

## Summary

- Adds `base_model_fsdp_plan` to **115 config/modular files** and `_fsdp_plan = {"lm_head": "keep_full_weight"}` to **94 ForCausalLM classes**, extracted from [PR #46269](https://github.com/huggingface/transformers/pull/46269) (`origin/distributed`).
- FSDP-only changes: no TP/SP/EP plan additions or style renames.
- Updates `FSDP_DISTRIBUTED_TEST_MODEL_TYPES` in `tests/test_fsdp_mixin.py` to run distributed FSDP tests on subset: `cohere2_moe`, `llama`, `mistral`, `mixtral`, `qwen3`, `qwen3_moe`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47165). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: afmoe, apertus, arcee, aria, bamba, bitnet, cohere, cohere2, cohere2_moe, csm, cwm, dbrx, deepseek_ocr2, deepseek_v2, deepseek_v3, deepseek_v32

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47165)
**Latest run:** [28985483396:1](https://github.com/huggingface/transformers/actions/runs/28985483396)
**Result:** `success` | **Jobs:** 16 | **Tests:** 173,316 | **Failures:** 0 | **Duration:** 20h 32m

<!-- ci-dashboard-recap:end -->
extend tests/quantization/gptq/test_gptq.py::GPTQTestCUDA and tests/q…
Author: sywangyi
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47166)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47166)
<!-- ci-dashboard-badge:end -->

uantization/mxfp4/test_mxfp4.py::Mxfp4QuantizerTest to xpu

- Intel XPU: @IlyasMoutawwakil

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gptq, mxfp4

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47166)
**Latest run:** [28928363564:2](https://github.com/huggingface/transformers/actions/runs/28989397743)
**Result:** `success` | **Jobs:** 2 | **Tests:** 24 | **Failures:** 0 | **Duration:** 1m 45s

<!-- ci-dashboard-recap:end -->
[Fix] Make DeepGEMM triton fallback more robust
Author: remi-or
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47126)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47126)
<!-- ci-dashboard-badge:end -->

This PR makes the triton fallback to DeepGEMM more robust:
- in case `CUDA_HOME` is not set, the current behavior is to fail with an `AssertionError` which is not caught by the `try` block because it specializes on `ImportError`. To avoid this, we call `get_mk_alignment()` before creating the `DeepGemm` object in a `try` block and pass the missing `CUDA_HOME` error as an `ImportError` -> triton fallback works
- in case `CUDA_HOME` is set but does not match the CUDA toolkit, the JIT compilation will fail but this is not caught at loading time, hence the problem manifests at Runtime and we hard-error instead of falling back to triton. To avoid this, we run a tiny JIT compiled kernel at load time (`transform_sf_into_required_layout`) and fallback to triton if the JIT compilation fails. 

Also, I think the `functools.cache` decorator never worked as intended because it does not catch error raised in the function, so any fail during `_load_deepgemm_kernel` was bound to happen all over again. To avoid this, instead of raising the expected `ImportError` in the cached function, we wrap it into a function that will raise by itself. 

Result: on a machine with no `CUDA_HOME` or a mismatching `CUDA_HOME`, we fall back to triton instead of erroring out. I have not tested on a machine with DeepGEMM but it should not change the behavior or a succesful load. Perhaps @IlyasMoutawwakil can confirm please? Thx!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47126). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Thanks for catching this ! we do have a flag to disable the deepgemm fp8 linear, we should probably mention it in the error message in finegrained_fp8's linear dispatch.

Not sure what you mean here, can you add a permalink? 

I feel like this is ready for a second round of review / approval, thanks! 

@remi-or 
https://github.com/huggingface/transformers/blob/d49affb382b9b9662ba1f103e67cf02402edf273/src/transformers/integrations/finegrained_fp8.py#L239
I have opened https://github.com/huggingface/transformers/pull/47154 based and into this branch, tell me what you think.

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47126)
**Latest run:** [28988683001:1](https://github.com/huggingface/transformers/actions/runs/28988683001)
**Result:** `success` | **Jobs:** 15 | **Tests:** 170,701 | **Failures:** 0 | **Duration:** 19h 43m

<!-- ci-dashboard-recap:end -->
Fix some failed test cases related with XPU Expectations
Author: kaixuanliu
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47173)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47173)
<!-- ci-dashboard-badge:end -->

Fixed following 6 failed XPU CI tests:
```
tests/models/llava_next/test_modeling_llava_next.py::LlavaNextForConditionalGenerationIntegrationTest::test_small_model_integration_test_batch_different_resolutions
tests/models/mimi/test_modeling_mimi.py::MimiIntegrationTest::test_integration_longform
tests/models/convnextv2/test_modeling_convnextv2.py::ConvNextV2ModelIntegrationTest::test_inference_image_classification_head
tests/models/cvt/test_modeling_cvt.py::CvtModelIntegrationTest::test_inference_image_classification_head
tests/models/exaone_moe/test_modeling_exaone_moe.py::ExaoneMoeIntegrationTest::test_model_logits
tests/models/internvl/test_modeling_internvl.py::InternVLQwen2IntegrationTest::test_qwen2_small_model_integration_generate
```
@ydshieh pls help review, thx!

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: convnextv2, cvt, exaone_moe, internvl, llava_next, mimi

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47173)
**Latest run:** [28942836158](https://github.com/huggingface/transformers/actions/runs/28942836158)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->
Add RTDetrForSegmentation
Author: ranykamel
Pull request: no
Labels: name: Feature request

### Feature request

Add an `RTDetrForSegmentation` and `RTDetrV2ForSegmentation` class that extends the object detection model class with a mask prediction head, similar to `DetrForSegmentation`.

### Motivation

RT-DETR in the transformers library currently only supports bounding box predictions through `RTDetrForObjectDetection` and `RTDetrV2ForObjectDetection`, with no segmentation counterpart unlike DETR. 
Adding this would also allow a more complete comparison of both RT-DETR and DETR models, as benchmarking on segmentation tasks is currently not possible within the library.

### Your contribution

I'll look into adding this and attempt to create a PR. This would be my first contribution to the repository. 😃 

Comments:
cc @nielsrogge @molbap

Hi! I’m interested in helping with this. Before I start, are you already actively working on a PR? If not, I can take an initial pass by adding the output class and segmentation head for RTDetr first, using DetrForSegmentation as the reference, then follow up with RTDetrV2 if the approach looks good.

Hi @liwd190019, thanks for the interest! I am actively working on a PR for this issue. I have an implementation on my local branch, currently focusing on testing. Once my PR is up, I would definitely appreciate it if you wanted to take a look, review my approach, or help test it out. Thanks!
Drop CPU/light docker image builder (moved to transformers-ci)
Author: tarekziade
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47202)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47202)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

Remove build-ci-docker-images.yml and the six light dockerfiles it built (quality, consistency, custom-tokenizers, torch-light, exotic-models, examples-torch). These CPU/light CI images are now built and pushed from the transformers-ci repo, which owns the PR test workflow; keeping a duplicate builder here pushed the same huggingface/transformers-* tags on the same cron, racing the transformers-ci build. The GPU image builder (build-docker-images.yml) and pipeline-torch.dockerfile (used by self-scheduled) stay.

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47202)
**Latest run:** [29004531854](https://github.com/huggingface/transformers/actions/runs/29004531854)
**Result:** `success` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47202). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove executorch from all-latest-gpu image + add torch smoke test
Author: tarekziade
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47196)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47196)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

executorch is only used for CPU tests, and it was breaking the GPU image because it pulled torch 2.13

this patch removes it but :
- keeps onnxruntime (gpu flavor)
- do a smoke test to verify it imports correctly


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47196). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47196)
**Latest run:** [29003426880:1](https://github.com/huggingface/transformers/actions/runs/29004103665)
**Result:** `success` | **Jobs:** 15 | **Tests:** 170,729 | **Failures:** 2 | **Duration:** 19h 21m

<!-- ci-dashboard-recap:end -->
Fix device_map computation when the no_split_modules have different sizes
Author: Cyrilvallez
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47203)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47203)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

As per the title. Fixes https://github.com/huggingface/transformers/issues/46823.
The problem is that in the case of Minimax for example, all layers have type `MiniMaxM3VLDecoderLayer`, but very very different sizes depending on internal conditional, i.e. if it uses dense or moe. The first few mlp layers have size ~0.6 GiB, while the others with moe have size ~13 GiB, which makes a huge diff.
Since the code was finding the 1st `MiniMaxM3VLDecoderLayer` it could find to infer the size, if it finds one of the first layers then the inferred size will be way too small. This PR fixes it by finding the maximum of all layers matching the class

Comments:
<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47203)
**Latest run:** [29005721146](https://github.com/huggingface/transformers/actions/runs/29005721146)
**Result:** `failure` | Grafana metrics are not available yet.

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47203). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Fix] Remove deprecated argument from `kernels` call
Author: remi-or
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47100)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47100)
<!-- ci-dashboard-badge:end -->

In version 0.14 the `kernels` package removed the `package_name` argument from `LocalLayerRepository.__init__` . Since we only support `kernels>=0.15.2,<0.16` it means we should remove it as well when creating this class.

No deprecation warning because `package_name` is inferred internally. 

Also added some stubs so that `uv` is happy.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47100). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47100)
**Latest run:** [28998601761:1](https://github.com/huggingface/transformers/actions/runs/28998601761)
**Result:** `success` | **Jobs:** 15 | **Tests:** 170,708 | **Failures:** 0 | **Duration:** 20h 22m

<!-- ci-dashboard-recap:end -->
torch-light: add executorch (+onnx) for CPU exporter tests
Author: tarekziade
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47204)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47204)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

Install onnxscript + onnxruntime and executorch==1.2.0 in the CPU test image so the CPU-only exporter tests (tests/exporters, executorch tests marked require_non_gpu) can run here. executorch 1.2.0 matches transformers' exporter API (targets 1.3.1) and requires torch>=2.11.0, satisfied by the torch<=2.11.0 pin already present.

Comments:
building this change in `push-image` as dev tag

https://github.com/huggingface/transformers/actions/runs/29010010840

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47204)
**Latest run:** [29009558561:1](https://github.com/huggingface/transformers/actions/runs/29009558561)
**Result:** `success` | **Jobs:** 2 | **Tests:** 24 | **Failures:** 0 | **Duration:** 1m 47s

<!-- ci-dashboard-recap:end -->

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47204). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Streaming Pipeline Regression in Transformers v5
Author: davidmezzetti
Pull request: no
Labels: name: bug

### System Info

Transformers 5+

### Who can help?

@Rocketknight1 @ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Since Transformers v5 there has been a regression with pipelines and generators. There is code that forces the generator to a list. When the generator is a large streaming dataset, it will force an OOM condition.

Simply excluding generators is a workaround but I'm not sure what the intent of this code was.

```diff
--- a/transformers/pipelines/base.py
+++ b/transformers/pipelines/base.py
@@ -1206,7 +1206,7 @@ class Pipeline(_ScikitCompat, PushToHubMixin):
             logger.warning(f"Ignoring args : {args}")
 
         # Detect if inputs are a chat-style input(s) and cast as `Chat` or list of `Chat`
-        container_types = (list, tuple, types.GeneratorType)
+        container_types = (list, tuple)#, types.GeneratorType)
         if is_torch_available():
             container_types = (*container_types, KeyDataset)
         if isinstance(inputs, container_types):
              if isinstance(inputs, types.GeneratorType):
                  # --> THE FOLLOWING LINE CAUSES THE ISSUE
                  inputs = list(inputs)
```

### Expected behavior

Pipeline iterates over generator incrementally and doesn't run out of memory.

Comments:
I would love to work on this if it's still open!

I looked into this and I think I have a fix ready, pending your go-ahead to open a PR.

Root cause: the `inputs = list(inputs)` in `Pipeline.__call__` was added in the v5 chat template change (#42326) so it could read `inputs[0]` to detect chat-style inputs. Generators are not subscriptable, so it materializes the whole thing just to peek at the first element, and on a large or streaming generator that forces the entire stream into memory and OOMs. In v4 there was no such conversion; generators would fall through to the existing lazy `get_iterator` / `PipelineIterator` path.

The fix i have peeks only the first item with `next()`. If it is a chat message, materialize just that single bounded conversation as a `Chat`. Otherwise it  reattach the peeked item to the generator and let it stream through the existing lazy iterator path, so a large generator is never fully materialized. Im avoiding just dropping `GeneratorType` from the container types which is the workaround in the issue, since that silently breaks single-chat-as-generator detection. I have a regression test asserting the generator input is not materialized, and it passes locally.


I also see @ojasvi24-coder offered to take this. Either way, I'm glad to help get this fixed. If you'd like me to open the PR I can do that right away,

Yes, I think this problem is real, and will get annoying on large inputs! I opened the PR myself since a lot of this is my code, and there were a few other vestigial bits in there that needed a refresh.

Thank you for the quick PR!
Cleanup pipelines, stop materializing generators
Author: Rocketknight1
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47142)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47142)
<!-- ci-dashboard-badge:end -->

This does some cleanups to pipelines, and how we handle complex inputs and generators. In particular, it avoids the situation where large generators were totally materialized, which caused OOMs in some cases.

Fixes #47116

Comments:
cc @zucchini-nlp I guess? It's a simple enough PR, so I don't think it needs a deep review as long as tests pass!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47142). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47142)
**Latest run:** [28943142987:1](https://github.com/huggingface/transformers/actions/runs/28962379444)
**Result:** `success` | **Jobs:** 15 | **Tests:** 172,750 | **Failures:** 1 | **Duration:** 6h 20m

<!-- ci-dashboard-recap:end -->
Fix tar-slip/zip-slip arbitrary file write vulnerabilities (CWE-22)
Author: ashusnapx
Pull request: yes
Labels: name: Code agent slop

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47183)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47183)
<!-- ci-dashboard-badge:end -->

## Summary

Fixes [#47177](https://github.com/huggingface/transformers/issues/47177).

Adds `safe_extract_tar()` and `safe_extract_zip()` utility functions to `src/transformers/utils/generic.py` that reject archive entries containing path traversal components (`..`), preventing arbitrary file writes outside the target directory.

## Vulnerable locations fixed

| File | Vulnerability |
|------|--------------|
| `models/parakeet/convert_nemo_to_hf.py` | `tar.extractall()` |
| `models/nemotron_asr_streaming/convert_nemotron_asr_streaming_to_hf.py` | `tar.extractall()` |
| `models/nemotron3_5_asr/convert_nemotron3_5_asr_to_hf.py` | `tar.extractall()` |
| `models/nemotron/convert_nemotron_nemo_to_hf.py` | `archive.extract()` (inline validation) |
| `models/marian/convert_marian_to_pytorch.py` | `zipObj.extractall()` |

## How it works

The new utilities use `os.path.realpath()` to resolve symlinks and `..` components, then verify the resolved path stays within the intended extraction directory. Malicious entries raise `ValueError`.

```python
safe_extract_tar(tar_handle, path="/output/dir")
safe_extract_zip(zip_handle, path="/output/dir")
```

## Testing

- `make fix-repo` — all 15 checks passed
- `make style` — all checks passed
- `make typing` — all checks passed
- Custom test verified that malicious archives with `../../` paths raise `ValueError`

Comments:
@huggingface/transformers-maintainers All CI checks pass. This is a security fix for #47177 (CWE-22 tar-slip/zip-slip). Happy to address any feedback.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: marian, nemotron, nemotron3_5_asr, nemotron_asr_streaming, parakeet

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47183)
**Latest run:** [28998765210:2](https://github.com/huggingface/transformers/actions/runs/28998765210)
**Result:** `failure` | **Jobs:** 15 | **Tests:** 170,729 | **Failures:** 1 | **Duration:** 18h 47m

<!-- ci-dashboard-recap:end -->