noamaanMulla-03/transformers-issues huggingface dataset:
Add Fast Image Processor for mobileViT
Author: MinJu-Ha
Pull request: yes

related to #36978
cc: @yonigozlan

I added Fast image processor for mobileViT and I noticed a noticeable difference between the outputs after preprocessing.

Here’s the code I used to compare them:

```
diff = (encoding_slow.pixel_values - encoding_fast.pixel_values).abs()
print(f"\n📊 Difference statistics:")
print(f"  Max difference: {diff.max().item():.10f}")
print(f"  Mean difference: {diff.mean().item():.10f}")
print(f"  Slow min/max: {encoding_slow.pixel_values.min().item():.10f} ~ {encoding_slow.pixel_values.max().item():.10f}")
print(f"  Fast min/max: {encoding_fast.pixel_values.min().item():.10f} ~ {encoding_fast.pixel_values.max().item():.10f}")
print(f"Slow implementation dtype: {encoding_slow.pixel_values.dtype}")
print(f"Fast implementation dtype: {encoding_fast.pixel_values.dtype}")
```
results:
📊 Difference statistics: 
  Max difference: 0.3411765397
  Mean difference: 0.1117687449
  Slow min/max: 0.0313725509 ~ 0.9764705896
  Fast min/max: 0.0313725509 ~ 0.9764706492
Slow implementation dtype: torch.float32
Fast implementation dtype: torch.float32
```

Even though the size configs look the same ({'shortest_edge': 20}), and both use torch.float32, the output difference seems quite significant for a slow/fast equivalence test.

Comments:
It really was—thank you so much for the advice! I'm still working on this issue and have finally passed 18 tests. This week, I’ll be focusing on the remaining 2 skipped ones, which are related to batched image processing and CUDA.

@yonigozlan, waiting for your review!

@yonigozlan , I'm still waiting for your review!

and just to clarify:
The test failures in `tests/models/mistral3/test_processor_mistral3.py` seem to be caused by external image download issues (e.g., PIL.UnidentifiedImageError when loading from https://www.ilankelman.org/stopsigns/australia.jpg).
Since my changes don’t touch this file or its logic, I believe these failures are unrelated to my PR.

Let me know if you’d like me to skip the tests or mock the image loading as a workaround!

<img width="1039" alt="image" src="https://github.com/user-attachments/assets/9ed4c553-06a5-4272-8caa-ef52cbc75b4e" />


@yonigozlan, Thank you for your kind comments and advice! I’ve completed the changes — could you please review them? Many thanks in advance for your time and help!

Hello @MinJu-Ha , don't hesitate to ping me if you need help or if the PR is ready for review!

Hi @yonigozlan ! Sorry for the delay — I've been busy recently due to my U.S. visa process.  
Thanks so much for your patience!  
I've just finished reflecting your suggestions in the code.  
Please have a look when you have time, and let me know if any further updates are needed!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37143). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
guard torch distributed check
Author: tvukovic-amd
Pull request: yes

This PR fixes issue where transformer pipelines invoke `torch.distributed.is_initialized()` in `src/transformers/pipelines/base.py`.

We need to check if `torch.distributed.is_available()` before checking if `torch.distributed.is_initialized()` otherwise we will face 
```
File "C:\develop\v\Lib\site-packages\transformers\pipelines\base.py", line 1036, in __init__
    if is_torch_available() and torch.distributed.is_initialized():
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'torch.distributed' has no attribute 'is_initialized'
```
Similar issue but in different file is mentioned on [Regression: Need to guard torch.distributed.is_initialized with torch.distributed.is_available #26039](https://github.com/huggingface/transformers/issues/26039) and solved with [safeguard torch distributed check #26056](https://github.com/huggingface/transformers/pull/26056).

Comments:
Made one suggestion, going to apply it and merge. Thank you for the PR!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39057). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix: unprotected import of tp plugin
Author: S1ro1
Pull request: yes
Labels: name: for patch

Fixes #39077. We only use `TensorParallelPlugin` [here](https://github.com/huggingface/transformers/blob/9c8d3a70b8bf359150c960c4281aaa853498fe8c/src/transformers/trainer.py#L5204) and that is version checked, the matching import that is also version checked is [here](https://github.com/huggingface/transformers/blob/9c8d3a70b8bf359150c960c4281aaa853498fe8c/src/transformers/trainer.py#L242). The import I deleted is non-version checked and not needed.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39083). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
TST: Fix PEFT integration test bitsandbytes config
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

The PEFT integration tests still used load_in_{4,8}_bit, which is deprecated, moving to properly setting BitsAndBytesConfig. For 4bit, also ensure that nf4 is being used to prevent

> RuntimeError: quant_type must be nf4 on CPU, got fp4

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. <= internal slack
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?  <= already there


## Who can review?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39082). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Please review @SunMarc @ydshieh


> Thanks. Would love to wait for another approval, but if you want to merge, works for me

Marc seems to be OoO at the moment. It's not super urgent, so I guess we can wait. Otherwise, someone from bnb could take a look, tentatively pinging @matthewdouglas. 

@ydshieh With Matt's review, I think it was safe to merge. As it's only about PEFT tests anyway, I think we should be good.
[fix] Add FastSpeech2ConformerWithHifiGan
Author: stevhliu
Pull request: yes

Implements fix in this [comment](https://github.com/huggingface/transformers/pull/37377#discussion_r2070820456) to enable `FastSpeech2ConformerWithHiFiGan`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38207). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Is it ok to merge despite the failing test? The error seems to come from `fastspeech2_conformer_with_hifigan` not being defined in `CONFIG_MAPPING_NAMES`, but when I try adding it there, a ton of other tests fail as well 😅
Sandeepyadav1478/2025 06 19 deberta v2 model card update
Author: sandeepyadav1478
Pull request: yes

# What does this PR do?

Updates the DeBERTa-v2 card according to the description in [#36979](https://github.com/huggingface/transformers/issues/36979) to standardize all model cards' look.
The pr includes:
-  Model description
-  Examples for Pipeline, AutoModel, and command line usage

Note: AttentionMaskVisualizer is not applicable either, as DeBERTa-v2is currently not supported

- [x] This PR improves the docs

Who can review?

@stevhliu Please have a look and let me know if any further work is needed. thank you


Comments:
Done with changes you pointed out @stevhliu, Thanks for letting me work on it.
Lmk, if I can help with any other issue. 
As, I'm new to this repo., I might take little extra time but really curious to work.

Hey @stevhliu,

Done with changes, lmk for else.
Thanks

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_38895). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu Done with :arrow_up: changes.
Can you assign me new task please?

Wanna join any technical task :relaxed:
skip some `test_sdpa_can_dispatch_on_flash`
Author: ydshieh
Pull request: yes

# What does this PR do?

Let's skip, and if the community ask for the support of this for some models, we can allocate the time then, or let the community to open the PRs.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39092). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
MODEL_FOR_MASK_GENERATION_MAPPING_NAMES is broken for SAM
Author: afaenger
Pull request: no
Labels: name: bug

### System Info

The new entry for `sam_hq` overwrites the existing entry for regular `sam`.

```
MODEL_FOR_MASK_GENERATION_MAPPING_NAMES = OrderedDict(
    [
        ("sam", "SamModel"),
    ]
)

MODEL_FOR_MASK_GENERATION_MAPPING_NAMES = OrderedDict(
    [
        ("sam_hq", "SamHQModel"),
    ]
)
```

See
https://github.com/huggingface/transformers/blob/d34e21e7dd392c76e7852d836b6f30ba1a2c5d62/src/transformers/models/auto/modeling_auto.py#L1521

https://github.com/huggingface/transformers/blob/d34e21e7dd392c76e7852d836b6f30ba1a2c5d62/src/transformers/models/auto/modeling_auto.py#L1527

Both entries should be merged into one

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

generator = pipeline("mask-generation", model="Zigeng/SlimSAM-uniform-77", device=device, points_per_batch=128)


### Expected behavior

sam should work again, not only sam_hq

Comments:
Yup, this is a bug for sure!

@afaenger would you be willing to make a PR to fix it?

Ah, opened a quick fix as I saw "sam hq" since I helped review it. Thanks for flagging @afaenger ! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Cant Load example from IP Adapters
Author: AeroDEmi
Pull request: no
Labels: name: bug

### System Info

I'm trying to run the following snippet with Diffusers 0.33.1 and Transformers 4.51.3

```
import  torch
from diffusers import FluxPipeline
from diffusers.utils import load_image

pipe = FluxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16
).to("cuda")

image = load_image("[https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/flux_ip_adapter_input.jpg").resize((1024](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/flux_ip_adapter_input.jpg%22).resize((1024), 1024))

pipe.load_ip_adapter(
"XLabs-AI/flux-ip-adapter",
weight_name="ip_adapter.safetensors",
image_encoder_pretrained_model_name_or_path="openai/clip-vit-large-patch14"
)
pipe.set_ip_adapter_scale(1.0)
```

It breaks when loading the ip adapter:
TypeError: CLIPVisionModelWithProjection.__init__() got an unexpected keyword argument 'dtype'

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the posted code

### Expected behavior

No error from the CLIPVisionModelWithProjection.__init__()

Comments:
I had to do this to fix it:

```
import torch
from diffusers import FluxPipeline, LCMScheduler
from transformers import CLIPVisionModelWithProjection

image_encoder = CLIPVisionModelWithProjection.from_pretrained(
    "openai/clip-vit-large-patch14",
    torch_dtype=torch.bfloat16
)

pipe: FluxPipeline = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    image_encoder=image_encoder,
    torch_dtype=torch.bfloat16,
).to("cuda")

pipe.load_ip_adapter("XLabs-AI/flux-ip-adapter", weight_name="ip_adapter.safetensors")
```

This rather seems to be a diffusers issue imo since their pipeline passes an incorrect kwarg on model init.

I've found this PR https://github.com/huggingface/diffusers/pull/11346 - does that fix your issue? Otherwise, I'd advise going to the diffusers repo directly instead.

I was running into the same issue and that PR https://github.com/huggingface/diffusers/pull/11346 fixed the issue.

I had to change `diffusers` in my requirments.txt to `git+https://github.com/huggingface/diffusers.git` as it's not released yet.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fix: Fixed `pydantic` required version in Dockerfiles to make it compatible with DeepSpeed
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
The `DeepSpeed` module now requires `pydantic vesrion >=2.0.0`. 
https://github.com/microsoft/DeepSpeed/blob/e2654bfd1ab431bde088a7501ed01b503daa5ab1/requirements/requirements.txt#L6

This is why the following docker builds are failing
https://github.com/huggingface/transformers/actions/runs/10534132397/job/29191315906#
https://github.com/huggingface/transformers/actions/runs/10534132397/job/29191315906

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ydshieh @amyeroberts 

Comments:
Joy okay, sounds good. Thanks for the flag. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33105). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix: rename 'eval_strategy' to 'evaluation_strategy' in TrainingArgum…
Author: trevorhughdavis
Pull request: yes

### 🐛 Bugfix: Rename `eval_strategy` → `evaluation_strategy` in `TrainingArguments`

This PR corrects an API mismatch in the `TrainingArguments` dataclass where the constructor expected `eval_strategy`, but `Trainer`, config files, and CLI use `evaluation_strategy`.

### ✅ Changes
- Renames the field `eval_strategy` → `evaluation_strategy`
- Updates all internal logic and references
- Corrects docstrings and argument descriptions
- Validated via `inspect.signature(TrainingArguments.__init__)`
- Tested with full reinstallation and cache invalidation

### 🧪 Reproduction (Before Fix)
```python
TrainingArguments(evaluation_strategy="epoch")
# ❌ TypeError: got an unexpected keyword argument 'evaluation_strategy'


Comments:
fix a bunch of XPU UT failures on stock PyTorch 2.7 and 2.8
Author: yao-matrix
Pull request: yes

@ydshieh, pls help review, thx very much.

Comments:
> Thank you!
> 
> For Aria, I consider to skip the tests as they don't fit into T4 (and likely A10 too). Don't know if your team have strong opinion about that (skip all integration tests for this particular model)

no, i don't have as of now, will follow your way,:).
Will Gemma 3n be added to transformers?
Author: TheMrCodes
Pull request: no
Labels: name: New model

### Model description

Question: Are there plans from Google or Huggingface to implement Gemma 3n in other frameworks?

I've seen the LiteRT weights and Android App Link on Huggingface, and was wandering if it would be possible to convert the model architecture in the *.task file to a transformer pytorch Module?

Personally I'll really interested in the Per-Layer Embeddings and MatFormer implementation they used, but do not have any experience with Tensorflow Lite

### Open source status

- [ ] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

https://huggingface.co/google/gemma-3n-E4B-it-litert-preview

Comments:
@TheMrCodes hey! Yes, adding Gemma3n is planned, we're working on it

Thanks for all your contributions. Do we have any progress or plans? Really appreciate for it.

We are almost done with the PR, it is in review process and will be shipped soon :)

Model is released, closing the issue
fix `kosmos2` tests
Author: ydshieh
Pull request: yes

# What does this PR do?

[VLMs]  support attention backends (#37576) actually breaks `kosmos2` as `KosmosTextAttention` is used in the decoder (`Kosmos2TextBlock`) as well as `Kosmos2ImageToTextProjection` (which should attend to all image places).

But without `is_causal`, the `sdpa_attention_forward` will treat it as `causal` due to

```
    if torch.jit.is_tracing() and isinstance(is_causal, torch.Tensor):
        is_causal = is_causal.item()
```

Maybe there is a better way to handle this. But I would not spend too much time but just adding `is_causal` arugment and pass it.

All tests pass on A10 now

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_39037). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I love the suggestion, unfortunately there is a super nit issue.

```
class Kosmos2TextBlock

        if config.add_cross_attention:
            self.encoder_attn = KosmosTextAttention(
                config,
                embed_dim=self.embed_dim,
                num_heads=config.attention_heads,
                dropout=config.attention_dropout,
                is_decoder=True,
                add_inner_attn_layernorm=False,
            )
```

At the time back, I added this block (so a decoder could do self-attention as well as cross-attention to an encoder's output, just like the original `Attention is all you need`).

Although, this is never used for Kosmos2 ( microsoft uses fairseq to contain a lot of code in the library, not all the paths are used for a particular model).

So if a user have `config.add_cross_attention`, it will add `add_cross_attention` to a `Kosmos2TextBlock` which is a decoder, but it should still pass `is_causal = False.` 

`Kosmos2ImageToTextProjection` is however has `is_decoder = False`, so would work with your suggestion.

I would say no hub repository of kosmos2 use `config.add_cross_attention=True` however. So do you want me to remove that confusing part of code and go with your suggeestion?



> I would say no hub repository of kosmos2 use config.add_cross_attention=True however. So do you want me to remove that confusing part of code and go with your suggeestion?

Ah I see, so we have a not-used code path. I think we can remove it with a minor deprecation cycle, which is in line with the whole "unbloating" cycle we have currently. And then we can safely assume that non-decoder uses non-causal mask

OK, will do that. Thanks
[NEW MODEL] MViTV2
Author: camilla-deckard
Pull request: yes
Labels: name: New model

# What does this PR do?

This PR introduces MViT, which is the basis for MeMViT, which is a video transformer that got requested in the issue below. I'm currently doing research in fine-grained video understanding and MeMViT happens to be able to process long video sequences. Unfortunately it's only available from the oficial FAIR repo which can be hard to integrate. That's why I decided to implement it here, but to make things cleaner and more modular I started with MViTv2 which is also a great vision transformer (mainly for image-related tasks)

Fixes [20545](https://github.com/huggingface/transformers/issues/20545)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@amyeroberts @qubvel 


Comments:
@alaradirik maybe you could take a look at this PR, I can see your comments in the original issue and it's been a while since i posted here with no replies :) 

cc @zucchini-nlp

Hi @zucchini-nlp, 
I totally understand that, in fact i had the same doubts recently 😃 I’m quite new to the field and didn’t fully realize how quickly it was moving. Closing this PR and hoping to contribute more meaningfully in the future!