noamaanMulla-03/transformers-issues huggingface dataset:
🚨🚨🚨 Fully remove Tensorflow and Jax support library-wide
Author: Cyrilvallez
Pull request: yes
Labels: name: for_v5?

# What does this PR do?

Apart from obvious tf/jax support, I believe the following should be the only potential breaking changes to torch-only code:
- pipelines do not take `framework` argument anymore
- onnx config methods do not take `framework` argument anymore

It may break current torch code if users do `framework="pt"` explicitly, but it's a necessary change. It makes no sense to keep those arguments, as the only framework working for those objects is now `torch`. Would be weird to keep it only for BC, as we are breaking the support anyway.


Note: I did not remove traces of tensorflow/jax in docs `.md` (markdown) files for now, as this PR is already enormous. It's a very tedious task, and moreover a lot of doc is written in another alphabet that I cannot read at all. Will be done in a subsequent PR, hopefully with the help of AI (should be a perfect fit for that)


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40760). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

L
Mark test_prompt_lookup_decoding as flaky
Author: Rocketknight1
Pull request: yes

The CI seems a bit unstable and this test is the most common culprit! cc @ydshieh

Comments:
cc @ydshieh

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42184). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi, I think there have been some efforts putting to make it less flaky. Where did you see this being fail, could you share a job run link?



Hi @Rocketknight1 I am running inside CircleCI runners. Run each test 20 times and all pass

> root@4c11ad497105:~/project# python -m pytest -v --flake-finder --flake-runs=20 tests/models/ -k "test_prompt_lookup_decoding_matches_greedy_search"

```
============================================= 2565 passed, 600 skipped, 1484595 deselected, 5 warnings in 245.93s (0:04:05) ==============================================
```

I am not pro for putting `@is_flaky` to a test defined in `GenerationTesterMixin`, (putting it in a specific model test file is fine), especially there was already some work done before.

I don't mean non of these tests are flaky, it may happens if I specify `--flake-runs=1000` for example. But it's time consuming if I do so for all these tests `test_prompt_lookup_decoding_matches_greedy_search`. 

If you see in the future which specific model having its `test_prompt_lookup_decoding_matches_greedy_search` flaky, please ping me (or you can dive into it if you are motivated).

For this PR, I think we can close, or if you have any other idea to propose.


That's interesting, it was blocking me yesterday here: https://app.circleci.com/pipelines/github/huggingface/transformers/153284/workflows/feadf2bc-dc65-42f1-a693-8001f074db44/jobs/2015365

Great, I will check this one!

BTW, if you need to merge the mentioned PR, don't hesitate to ping me or core maintainers 🙏 no need to be blocked

@ydshieh this test was failing again in CI today! I still want to mark it as flaky

Example of a recent failing run: https://app.circleci.com/pipelines/github/huggingface/transformers/161525/workflows/1f5920ba-47ee-498d-b9b7-d88d60922cb9/jobs/2126196
add trackio to training notebooks
Author: merveenoyan
Pull request: yes

@stevhliu I also need to remove IDEFICS example (too old) and fix image captioning one (dataset was taken down, although it's hard to find an unbroken image captioning dataset), will do on a follow-up PR

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43442). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[kernels] Update cv_utils name
Author: MekkCyber
Pull request: yes

# What does this PR do?

We are standardizing kernel names in `kernels-community` to use `-` instead of `_`, this pr simply updates `cv_utils`
new kernel is here with the latest torch version 2.10: https://huggingface.co/kernels-community/cv-utils

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam3_video

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43529). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Tie the weights even if initializing from a config on meta device
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Fix https://github.com/huggingface/transformers/issues/43522.
TLDR we want to skip tying when inside `from_pretrained` (so we add the context manager), but always tie when initializing from config (even with meta device).

Consider the following:

```python
import torch
from transformers import AutoConfig,  PaliGemmaForConditionalGeneration


repo_id = "google/paligemma-3b-mix-224"
config = AutoConfig.from_pretrained(repo_id)
with torch.device("meta"):
    model = PaliGemmaForConditionalGeneration._from_config(config)

lm_head_in_params = "lm_head.weight" in [n for n, _ in model.named_parameters()]
print(f"Weights were tied correctly: {not lm_head_in_params}")
```

this was False before, True now

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43523). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Operation Green CI
Author: Rocketknight1
Pull request: yes

This is the PR where I just keep patching things until it's green and then I merge

Comments:
Marking PLBart tokenizer tests as flaky based on this run: https://app.circleci.com/pipelines/github/huggingface/transformers/161560/workflows/e1eb5a6b-d05b-4085-abfe-b8e3bd2a872b/jobs/2126677

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43530). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: plbart

:sob:
Fix for #43530
Author: Rocketknight1
Pull request: yes

#43530 fails because the test I wrote for verifying correct downloads fails on image files, which might have UTF-8 illegal bytes. Opening the file in `b` mode fixes it. This doesn't show up in the CI because the script is run when recreating docker workers!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43535). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`RoPE`] Make explicit inheritance
Author: vasqu
Pull request: yes

As per title, not every config needs this mixin - it should be only done for those that have it

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43410). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Moving too fast, need to check bit by bit for the last failures

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: afmoe, apertus, arcee, aria, bamba, bitnet, blt, chameleon, cohere, cohere2, csm, cwm, dbrx, deepseek_v2, deepseek_v3, dia
reduce_labels returns only one label instead of the whole array of labels in BeitImageProcessorFast
Author: sbucaille
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.0.1.dev0
- Platform: macOS-26.2-arm64-arm-64bit
- Python version: 3.12.11
- Huggingface_hub version: 1.3.1
- Safetensors version: 0.6.2
- Accelerate version: 1.11.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0 (NA)
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

https://github.com/huggingface/transformers/blob/4854dbf9da4086731256496cf4a8e4ea45d4d54e/src/transformers/models/beit/image_processing_beit_fast.py#L62-L70

I think the return statement is wrong and should be `labels` instead of `label`.

@yonigozlan 
will open a PR

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import torch

from transformers import AutoImageProcessor


model_name = "nvidia/mit-b3"
num_images = 5
images = [torch.rand(3, 256, 256) for _ in range(num_images)]
# 0 = background, 1 = class
labels = [torch.randint(0, 1, (1, 256, 256)) for _ in range(num_images)]
processor = AutoImageProcessor.from_pretrained(model_name, do_reduce_labels=True)
inputs = processor(images=images, segmentation_maps=labels, return_tensors="pt")
# after preprocessing, with reduce labels, background = 255 and class = 0
print(inputs["pixel_values"].shape)
print(inputs["labels"].shape)

```

Output : 
```cmd
torch.Size([5, 3, 512, 512])
torch.Size([1, 512, 512])
```

### Expected behavior

instead of :
```cmd
torch.Size([5, 3, 512, 512])
torch.Size([5, 1, 512, 512])
```

Comments:
fix: return labels instead of label in reduce_label method in BeitImageProcessorFast
Author: sbucaille
Pull request: yes

# What does this PR do?
Fixes #43526 

## Who can review?

@yonigozlan @molbap 


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: beit, dpt, mobilenet_v2, mobilevit, segformer

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43527). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix safe _file_ access in CLIP lazy module
Author: Olexandr88
Pull request: yes

Avoid potential KeyError when accessing _file_ in CLIP lazy module.


Comments:
`__file__` may indeed not be defined (I think when Python is running in an interpreter), but I suspect downstream code will probably crash when `_file = ""` , right? Does this actually fix any issues?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: clip

> `__file__` may indeed not be defined (I think when Python is running in an interpreter), but I suspect downstream code will probably crash when `_file = ""` , right? Does this actually fix any issues?


Updated to globals().get("__file__") to avoid KeyError without introducing a fake path
Fix tiles mixing for batched input, add tie_word_embeddings to LFM2VL config
Author: ankke
Pull request: yes

Fixes: For batched input with multiple images tiled input was mixed between batches.Now, using a util function.
Adds: `tie_word_embeddings` to Lfm2VlConfig. Tests for above bug + updated integration test for the newest lfm2vl model.

cc: @zucchini-nlp


Comments:
run-slow: lfm2_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/lfm2_vl"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43379). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21351857482)

✅ No failing test specific to this PR 🎉 !


I can't push to this branch apparently, can you update the docstring @ankke and add `tie_word_embeddings`?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: lfm2_vl

run-slow: lfm2_vl



This comment contains `run-slow`, running the specified jobs: 

models: ["models/lfm2_vl"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21431708726)

✅ No failing test specific to this PR 🎉 !
[CB] Minor perf improvements and ty compatibility
Author: remi-or
Pull request: yes

# Summary
This PR introduces minor changes to the continuous batching feature:

#### Performance
- optimize the `get_seqlens_k` function to avoid looping over all cache managers
- logits indexing is now done inside the cuda graph, as there was no apparent reason it should be done outside
- fuse the `get_read/write_indices` functions, which should also make the code more readable
#### Typing
- add a `ProtoPretrainedModel` class for typing purposes
- make the code `ty` compliant, using as little `# type:ignore` as possible
- add the `set_continuous_batching_context` to the `LogitsProcessorsList` class to stay coherent with typing 
#### Usability
- rename a variable in `copy_cache`
- we now add a temporary token to the request that get scheduled, which is replaced when a new real token is produced. This is not very usefull as of now but will be when we add CPU / GPU overlapping scheduling, which is soon™

## Performances
| Arguments                                          | Throughput on main | Throughput with PR | Delta    |
|----------------------------------------------------|--------------------|--------------------|----------|
| --samples 10                                       | 530.45             | 531.02             | +0.57    |
| --samples 20 --num-blocks 16/20                    | 100.14             | 126.91             | N/A*     |
| --samples 50                                       | 1477.42            | 1478.66            | +1.24    |
| --samples 100                                      | 2266.81            | 2243.32            | -23.49   |
| --samples 100 --attn flash_attention_2             | 2076.81            | 2050.43            | -26.38   |
| --samples 100 --attn sdpa                          | 840.72             | 831.78             | -8.94    |
| --samples 500                                      | 5281.98            | 5287.99            | +6.01    |
| --samples 500 --add-prefix --compile               | 6743.21            | 6732.60            | -10.61   |
| --samples 50 --num-return-sequences 8 --do-sample  | 5362.52            | 5359.59            | -2.93    |
| --samples 100 --num-return-sequences 4 --do-sample | 4674.91            | 4697.05            | +22.14   |

## Tests
Tests pass, including the tests added for this PR.

## Sanity check
Looks good.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43521). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Quantization] Add cutlass kernel for FP8
Author: MekkCyber
Pull request: yes

# What does this PR do?

adds the cutlass kernel for scaled matmul, the performance is much better than triton for the specific block size : (128, 128):
```
================================================================================
COMPARISON: CUTLASS vs Triton Speedup
================================================================================

     M x      N x      K |  Triton (ms) | CUTLASS (ms) |    Speedup
----------------------------------------------------------------------
     1 x   4096 x   4096 |       0.0571 |       0.0317 |       1.80x
     4 x   4096 x   4096 |       0.0571 |       0.0516 |       1.11x
    16 x   4096 x   4096 |       0.0563 |       0.0511 |       1.10x
    32 x   4096 x   4096 |       0.0573 |       0.0509 |       1.13x
    64 x   4096 x   4096 |       0.0581 |       0.0509 |       1.14x
   128 x   4096 x   4096 |       0.0807 |       0.0509 |       1.59x
   256 x   4096 x   4096 |       0.0798 |       0.0511 |       1.56x
   512 x   4096 x   4096 |       0.0826 |       0.0509 |       1.62x
  1024 x   4096 x   4096 |       0.1069 |       0.0510 |       2.10x
  2048 x   4096 x   4096 |       0.2187 |       0.0660 |       3.31x
  4096 x   4096 x   4096 |       0.4310 |       0.1256 |       3.43x
```

All FP8 tests passing !

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43304). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43304&sha=cfd4b9

@ArthurZucker if you can merge this ! 

nice work and perfs!
[Quantization] Fix triton_kernels name after being renamed to gpt-oss-triton-kernels
Author: MekkCyber
Pull request: yes

# What does this PR do?

change `triton_kernels` name to `gpt-oss-triton-kernels`, no new failling tests related to this change

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mxfp4

The latest version don't feel compatible with our code, I just added the kernel as is to https://github.com/huggingface/kernels-community/tree/main/gpt-oss-triton-kernels, and renamed it for more clarity

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43528). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

perfect thanks for confirming ! 

please merge @ArthurZucker
[kernels] exception handling for fa kernels
Author: MekkCyber
Pull request: yes

# What does this PR do?

Before we were just silently skipping parameters that are passed by the user like `s_aux` in case they are not supported by the attention backend specified, it would be better to raise an exception instead.

cc @danieldk 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43549). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Increase timeout when preparing CI
Author: Rocketknight1
Pull request: yes

We switched from `requests`, with no timeout by default, to `httpx`, which does have a read timeout by default. This causes some timeout errors in the CI, so this PR increases the timeout length by following the snippet [here](https://github.com/encode/httpx/discussions/3356)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43560). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[don't merge] check bot permission
Author: ydshieh2
Pull request: yes

[don't merge] check bot permission

Comments:
@bot /repo

@bot /style

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/21440579078).

@bot /style

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/21440828681).

@bot /style

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/21441033875).

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/21441034195).

@bot /style

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: vit

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/21443503914).
check/fix repo. check bot workflow
Author: ydshieh
Pull request: yes

# What does this PR do?

Try if the permission is correctly set.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43565). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
add an option to disable Sam3VideoModel progress bar
Author: ndeybach
Pull request: yes

# What does this PR do?

Avoids flooding the tty when using Sam3VideoModel ( when using propagate_in_video_iterator() )

harmonize argument with other propagate_in_video_iterator in other similar classes (EdgeTamVideoModel, Sam2VideoModel, Sam3TrackerVideoModel). (⚠️ breaking change: disables progress by default contrary to current behavior)

\+ docs mention of the option to enable a progress bar


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@yonigozlan

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam3_video

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43564). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Modular`] Allow to add new bases that are not present in the inherited class
Author: vasqu
Pull request: yes

As per title, it is an edge case for lasr only at the moment but #41212 will also need it. 

I think this is the easiest solution (and fastest) because there are several edge cases with additional bases. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43556). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: edgetam, edgetam_video, emu3, glm_ocr, lasr, qwen2_moe, sam2, sam2_video, sam3_tracker, sam3_tracker_video

Merging since it unblocks another model PR and CI is green :exploding_head:
Fix Wav2vec and a few others
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

https://github.com/huggingface/transformers/pull/43523 broke Wav2Vec and a few others as they have their own `tie_weights`, which is actually not tying the weights... Who would have thought that changing the purpose of a method in a few models could create a lot of problems later on?? 🥲🥲😅😅

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: hubert, sew, sew_d, unispeech, unispeech_sat, wav2vec2, wavlm

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43566). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

CI still not triggering... Merging this one!
Fix repo. consistency bot (push permission issue)
Author: ydshieh
Pull request: yes

# What does this PR do?

Use git checkout command instead github api (otherwise we can't push the changes)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43570). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Debug] [Draft] Investigating red CI
Author: remi-or
Pull request: yes

This is a draft / debug PR to check why the CI is red. No need to review. 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: kosmos2_5

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43562). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.