noamaanMulla-03/transformers-issues huggingface dataset:
Move the Mi355 to regular docker
Author: remi-or
Pull request: yes

Currently, the Mi355 workflow depends on an old docker container `huggingface/testing-rocm7.0-preview`
With recent changes (#41792) the regular AMD docker should work on both Mi325 and Mi355. 
To avoid bugs, this PR changes the workflow to use the regular AMD docker. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41989). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
More data in benchmarking
Author: remi-or
Pull request: yes

This PR changes the benchmarking workflow to gather more data per benchmark. cc. @McPatate I only changed one workflow file, not sure I did not miss others.
It also adds a check on the benchmark config to disable FA when it is not installed to avoid an exception. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41848). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix 3 failed test cases for video_llama_3 model on Intel XPU
Author: kaixuanliu
Pull request: yes

This PR fixes 3 failed test cases on Intel XPU:
```
1.tests/models/video_llama_3/test_modeling_video_llama_3.py::VideoLlama3IntegrationTest::test_small_model_integration_test
2.tests/models/video_llama_3/test_modeling_video_llama_3.py::VideoLlama3IntegrationTest::test_small_model_integration_test_batch_wo_ima
ge
3.tests/models/video_llama_3/test_modeling_video_llama_3.py::VideoLlama3ModelTest::test_generate_with_quant_cache 
```


Comments:
Thank you @kaixuanliu 

cc @yao-matrix for a first look 🙏 



@kaixuanliu @yao-matrix Thank you.

I have currently some trouble to get a CI runner to check the `_is_package_available` issue. From [our scheduled CI runs](https://github.com/huggingface/transformers/actions/runs/18961133099/job/54148721758), there is no issue. Of course, it's likely depending on the sys. environment, but I don't want to merge this PR without being verified more deeply myself.

I am happy to merge the PR if you can revert this part of change, and open an issue for this `_is_package_available`. Don't forget to  provide the env. details (`transformers env` and `pip freeze`). WDYT?

run-slow: vit, clip

This comment contains `run-slow`, running the specified jobs: 

models: ["models/clip", "models/vit"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/19001092563)

✅ No failing test specific to this PR 🎉 !


**[For maintainers]** Suggested jobs to run (before merge)

run-slow: video_llama_3

@SunMarc , Hi, have reverted the code and submitted issue here: https://github.com/huggingface/transformers/issues/41981
Add MobileNetV1 FastImageProcessor
Author: Priyanshjain10
Pull request: yes

Implements GPU-accelerated FastImageProcessor for MobileNetV1 using BaseImageProcessorFast.

Fixes #36978

### Changes:
- Added `image_processing_mobilenet_v1_fast.py` with MobileNetV1FastImageProcessor class
- Updated `__init__.py` to export the new processor
- Inherits from BaseImageProcessor for compatibility
- Tested and working in Google Colab

### Testing:
- ✅ Successfully imported and instantiated
- ✅ Verified inheritance structure
- ✅ Basic functionality confirmed

This is a starting implementation ready for review and enhancement based on maintainer feedback.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mobilenet_v1

cc @yonigozlan

Not sure what this PR is meant to achieve, as mobilenetv1 already has a fast image processor. Closing.
fix to get output of intermediate output of dinov3 for more use case
Author: SangbumChoi
Pull request: yes

# What does this PR do?

@qubvel @yonigozlan Can anyone review this simple PR?
since DINOv3 has alot of use case the output format should be enables to return intermediate hidden_states.

Fixes # (issue)


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface, @SunMarc and @qgallouedec
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: dinov3_vit
🚨 Bump to Python 3.10 and rework how we check 3rd-party libraries existence
Author: Cyrilvallez
Pull request: yes
Labels: name: for_v5?

# What does this PR do?

Python 3.9 is officially EOL starting this month, so let's bump it! Took the opportunity of the new features available in 3.10 to simplify quite a bit our library checks

Note that it is breaking BC a bit as I removed some old and unused library checks that were public members

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41268). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yeah, the exact date was not super clear to me @ydshieh 🥲, do you have more infos from experience? Official website only says October, without a given day, so I guess it's already EOL? Or at the end of the month? (but very unlikely they would have a last release in the last month anyway I guess)

> do you have more infos from experience?

No. From [previous python versions](https://devguide.python.org/versions/#unsupported-versions), it is definitely not something predictable (i.e. not always the end of the month):


```
3.8 2024-10-07
3.7 2023-06-27
3.6 2021-12-23
3.5 2020-09-30
``` 

Not _strictly_ required, but as the output of those functions cannot change anyway, it's an easy and nice way to cache the result without having to define 10k global variables as was done previously. About one third/half of them had it previously already
It's basically a tiny performance optimization

Merging as we released last v4 version already!

(note that `transformers@main` can still be installed on 3.9 since it's not completely dropped in setup.py and raises `AttributeError: module 'importlib.metadata' has no attribute 'packages_distributions'`)

Indeed, I forgot it! It was fixed in https://github.com/huggingface/transformers/pull/41410 already, thanks for checking-in!
Is there plan to integrate ColQwen2.5 into Transformers?
Author: rebel-thkim
Pull request: no
Labels: name: New model

### Model description

Is ColQwen2ForRetrieval integrated into the transformers library, and are there plans to add [ColQwen2.5](https://github.com/illuin-tech/colpali/blob/main/colpali_engine/models/qwen2_5/colqwen2_5/modeling_colqwen2_5.py) in the future?

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

https://github.com/illuin-tech/colpali/blob/main/colpali_engine/models/qwen2_5/colqwen2_5/modeling_colqwen2_5.py

https://github.com/huggingface/transformers/pull/38391

Comments:
Add GLPNImageProcessorFast with enhanced 4-channel support for #36978 
Author: akacmazz
Pull request: yes

# What does this PR do?

This commit introduces GLPNImageProcessorFast, a PyTorch-optimized image processor for GLPN models with enhanced multi-channel support.

Key improvements:
- Added GLPNImageProcessorFast class with native PyTorch tensor processing
- Enhanced support for 1, 3, and 4-channel images (including RGBA)
- Optimized preprocessing pipeline using torchvision transforms
- Updated GLPNImageProcessor to support 4-channel inference
- Added comprehensive tests for multi-channel image processing
 - Added proper documentation for the new processor

The fast processor leverages PyTorch tensors throughout the processing pipeline, providing better performance and memory efficiency compared to the PIL-based approach. Both processors now support variable channel dimensions for improved flexibility.

Technical details:

- Uses torchvision.transforms for efficient tensor-based preprocessing
- Implements proper channel dimension handling with infer_channel_dimension_format(num_channels=(1,3, 4))
- Maintains API compatibility with existing GLPNImageProcessor
- Provides significant performance improvements for PyTorch workflows

This enhancement enables GLPN models to work seamlessly with RGBA images and other multi-channel  inputs, which is particularly useful for computer vision applications involving images with transparency channels.

Fixes  #36978 https://github.com/huggingface/transformers/issues/36978

  - [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/
  CONTRIBUTING.md#create-a-pull-request),
        Pull Request section?
  - [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)?
   Please add a link to it if that's the case. 
  - [x] Did you make sure to update the documentation with your changes? Here are the
        [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
        [here are tips on formatting
  docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
  - [x] Did you write any new necessary tests?


## Who can review?

@yonigozlan, @amyeroberts, @qubvel

This PR focuses on vision models (GLPN) and adds a new fast image processor with enhanced channel support. The changes include both the core implementation and comprehensive testing.



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, glpn

Closing this PR as https://github.com/huggingface/transformers/pull/41725 is about to be merged
Finetune `gpt-oss-20b` with `mxfp4` quantization
Author: eliotjones1
Pull request: no

Apologies if this is the wrong issue format -- I am not confident enough to say that this is for sure a bug and not just user error. I am currently unable to finetune (using peft/trl) the new oss openai model with quantization. 

The relevant packages and their versions are:

```
transformers                             4.56.0.dev0              /{installed from gh repo}/training/transformers
triton                                   3.4.0
triton-kernels                           1.0.0
```

With these, I am able to train in bf16; `MXFP4 quantization requires triton >= 3.4.0 and kernels installed, we will default to dequantizing the model to bf16`

So, what do I do? I run: `uv pip install kernels` 

Then, when training, I get the following error:

```
ValueError: The model you are trying to fine-tune is quantized with QuantizationMethod.MXFP4 but that quantization method do not support training. Please open an issue on GitHub: https://github.com/huggingface/transformers to request the support for training support for QuantizationMethod.MXFP4
```

I'm operating on a node of H100s for reference, and more or less following the recipe [here](https://cookbook.openai.com/articles/gpt-oss/fine-tune-transfomers) on a custom dataset.

Has anyone else had success here? 

Comments:
Hi did you resolve the issue?


`mxfp4` support was only added very recently for inference, to support `gpt-oss`. We'll need additional kernels to support the backward mxfp4 path. Hopefully there'll be some progress here soon, but for now you need to dequantize to `bf16` to train the models.
Fix `run slow v2`: empty report when there is only one model
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix Run slow v2 (#41914):

when there is only one model specified (in particular, new model addition PR), the `setup` job's step in `.github/workflows/self-scheduled.yml`:

> echo "slice_ids=$(python3 -c 'd = list(range(${{ env.NUM_SLICES }})); print(d)')" >> $GITHUB_OUTPUT

will still generate a list `[0,1]` (because `NUM_SLICES: 2`) and the `run_models_gpu` will have a matrix `slice_id: [0, 1]` but one of it would get an empty list of models to run.

The workflow still runs without problem, but this causes `${{ needs.model-ci.outputs.report }}` in `.github/workflows/self-comment-ci.yml` being empty , see [this run](https://github.com/huggingface/transformers/actions/runs/19057114226/job/54430404760).
We got `Annotations 3 errors` if we click any job which shows

> Error when evaluating 'strategy' for job 'run_models_gpu'. huggingface/transformers/.github/workflows/model_jobs.yml@f6837a4cf5b173c38c992531685a4a9c2fe43259 (Line: 54, Col: 18): Matrix vector 'folders' does not contain any values


This PR fixes this issue by using the actual length of `folder_slices` instead.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42002). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Avoid adding space when decoding tokenization
Author: cikay
Pull request: no
Labels: name: Feature request

### Feature request

Hi, I trained a tokenizers. Tokens contain spaces as well. When I decode the decode method add space between tokens and it makes it wrong I need to avoid them. How to do that?

from transformers import PreTrainedTokenizerFast

tokenizer = PreTrainedTokenizerFast.from_pretrained("muzaffercky/kurdish-kurmanji-tokenizer", revision="v1.0")

test_text = """
Ez ê di vê gotarê da qala ên ku ez guhdar û temaşe dikim bikim
"""

tokens = tokenizer.tokenize(test_text)

print(f"Tokens: {tokens}")

output: Tokens: ['\n', 'Ez ê ', 'di vê ', 'got', 'arê ', 'da ', 'qala ', 'ên ku ', 'ez ', 'guh', 'dar û ', 'temaşe ', 'dikim ', 'bikim', '\n']


ids = tokenizer.encode(test_text)
print(f"IDs: {ids}")

output: IDs: [6, 6271, 1323, 452, 462, 396, 2409, 566, 654, 1204, 3278, 4543, 7880, 7595, 6]

text = tokenizer.decode(ids)

print(f"text: {text}")
output: 
text: 
Ez ê  di vê  got arê  da  qala  ên ku  ez  guh dar û  temaşe  dikim  bikim

As you can see it add extra space between tokens when decoding. I know I can make something like below but I am curious if transformer support something like this built-in

individual_tokens = [tokenizer.decode([id]) for id in ids]

"".join(individual_tokens)


### Motivation

Not writing custom code to avoid adding space between tokens

### Your contribution

No

Comments:
I seemly encounter same problem，look this：
'''
from transformers import AutoTokenizer 

model_name = 'llmlingua-2-bert-base-multilingual-cased-meetingbank'
text = '你好'
obj = tokenizer.encode(text)
print(f"使用encode后的结果为：{obj}") # 使用encode后的结果为：[101, 2262, 3240, 102]
result = tokenizer.decode(obj)
print(f"使用decode后的结果为：{result}") # 使用decode后的结果为：[CLS] 你 好 [SEP]

'''
[kernels] change import time in KernelConfig
Author: MekkCyber
Pull request: yes

# What does this PR do?

Importing `kernels` when not installed was causing issues in the CI. Since we have a `kernels` folder `_is_package_availabe` assumes the package `kernels` is available when it's not. This PR simply fixes that by moving the import inside the kernel function that are only executed when we are sure `kernels` is installed

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42004). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
DOC: Fix typo in argument name: pseudoquant
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

The correct argument name is `pseudoquantization`. Since there is no error on passing wrong argument names (which is arguably an anti-pattern), this is difficult for users to debug.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41994). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Wrong loss computation for VisionEncoderDecoderModel
Author: tcnguyen
Pull request: no
Labels: name: bug

### System Info

Hi team, in this:

https://github.com/huggingface/transformers/blob/main/src/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py#L550

The `decoder_input_ids` is already shifted right w.r.t the labels. but then in the ForCausalLMLoss loss, the labels is also shifted left, which caused the problem.

This happened since 4.50.0 version with this commit: https://github.com/huggingface/transformers/pull/36753/files#diff-d6b5547bc8ece59fd6c96645e48cce4f32d0995a6621b57f896ad800168017bdL641

thank you

cc @qubvel

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When training a TrOCR model following this https://huggingface.co/docs/transformers/v4.53.3/en/model_doc/vision-encoder-decoder#transformers.VisionEncoderDecoderModel.forward.example, the loss is not converging well for transformers==4.50, while it did with transformers==4.49

### Expected behavior

we can set the `shift_labels = labels `in the case `decoder_input_ids `is None and pass it to the loss function

Comments:
Hi, I'd like to take up this issue in case it's still available

Hi @tcnguyen, thanks for opening an issue! We would appreciate your PR to fix it, otherwise, please let @Flakes342 know if they can take this up 🤗

Thank you for your replies. @Flakes342 yes please take this. Thank you!

@Flakes342 are you working on  this? If not, I can take it up

Hi, I am working on #40185, have submitted the PR, waiting for review and further changes now :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This issue can be closed now

> This issue can be closed now

sorry but I saw your pr is dropped, is this issue already fixed?

https://github.com/huggingface/transformers/pull/40185#discussion_r2284610909 please refer this once

/assign

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I'm not sure when, but I believe this issue was fixed, and I can't see evidence of double-shifting in the current code. If you think it's still present, feel free to comment and reopen!

@Rocketknight1 yes this issues is fixed in this PR https://github.com/huggingface/transformers/pull/40863
Fix `torch+deepspeed` docker file
Author: ydshieh
Pull request: yes

# What does this PR do?

Similar to #41978 and #41975, but with extra issue of `--global-option=`. All issues are caused by the new pip version `25.3`.

Verified the docker file could be built and the `deepspeed` CI job could run.



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41985). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

change it to draft as there are more things to fix


In the deepspeed docker, we also have `torchcodec`

> RUN python3 -m pip uninstall -y torch torchvision torchaudio && python3 -m pip install --no-cache-dir -U torch==$PYTORCH torchvision torchaudio torchcodec --extra-index-url https://download.pytorch.org/whl/$CUDA
Build a CI docker image with FA3
Author: ydshieh
Pull request: yes

# What does this PR do?

Build a CI docker image with FA3.

It is available as 

huggingface/transformers-all-latest-gpu:fa3

a build is trigged at [this run](https://github.com/huggingface/transformers/actions/runs/18347217144/job/52257062261)

Let's see how it ends.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41451). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Unfortunately, the build is still failing. Will wait until FA release the wheel file then.

Change to draft for now

Close this PR for now.

Already ping FA people [here](https://github.com/Dao-AILab/flash-attention/issues/1967#issuecomment-3481252755)

Hope we have some news soon.
Reduce the number of benchmark in the CI
Author: remi-or
Pull request: yes

In recent changes, we increased the number of benchmarks being run in the benchmarking CI. This has led to a relatively long CI runtime ~1hr on A10 runners. This might be wasteful. 
This PR changes how the `run_benchmark.py` generates the benchmark configs, favoring a benchmarking "level" system instead of ad hoc heuristic. The system has 5 levels of coverage:

0. Only one cfg is benchmarked, the most efficient until now for generate
1. A few well-performing config are benchmarked. Default option
2. For each attention implementation, we benhcmark one config, and we add 2 more on top to test `kernelize`. This leads to a total of 6 configs, and it is the option used in the CI.
3. We benchmark all possible config, but the compile mode is either `None` (no compile) or `default`
4. Same but with all possible compile modes.

This simplifies the script and makes it usable for both CI and users that want to run a fast benchmark suite as well as those who want more coverage.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42008). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.