noamaanMulla-03/transformers-issues huggingface dataset:
chore: update model card for SigLIP
Author: saswatmeher
Pull request: yes

# What does this PR do?

Update the model card for SigLIP to handle #36979 

Fixes #36979 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@stevhliu

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @eustlb
- graph models: @clefourrier

Library:

- flax: @gante and @Rocketknight1
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @zach-huggingface and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @SunMarc @zach-huggingface
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @Rocketknight1
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37585). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[cleanup] remove `/model_cards` 🧹 🧹 
Author: gante
Pull request: yes

# What does this PR do?

As also discussed with @LysandreJik -- `/model_cards` holds a README regarding the old model cards, which used to live in `transformers`. We have long moved away from them :)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37685). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
tokenizer.convert_tokens_to_ids inconsistent with tokenizer forward in CLIPTokenizer
Author: ysig
Pull request: no
Labels: name: bug

Only `openai/clip-vit-base-patch32` was downloaded `14,408,028` over the last month!
Yet the clip tokenizer has a straightout bug:

```
>> from transformers import CLIPTokenizer
>> tokenizer = CLIPTokenizer.from_pretrained('openai/clip-vit-base-patch32')
>> token = 'my'
>> print(
        tokenizer.convert_tokens_to_ids([token])[0],
        tokenizer(
            [token],
            padding=True,
            return_tensors="pt",
        ).input_ids[0][1].item()
)
>> 1152,   607
```

Even if there is an internal reason why this happens, it should be fixed or warned!

### Expected behavior

```
>> 1152, 1152
```

Comments:
This behaviour is common to many tokenizers! In the CLIP tokenizer token `1152` is `"my"` and `607` is `"my</w>"`. CLIP tokenizes words using `</w>` as the 'end of word' marker, so the correct tokenization for the string "my" is `607`. When you use `convert_tokens_to_ids` you bypass this behaviour, and you get the token that represents "my" as an incomplete word.
Add maintainers for ROCm/Intel XPU/Ascend NPU
Author: Rocketknight1
Pull request: yes

We have a list of maintainers for various parts of the library in the issue template; this PR adds maintainers for backends like ROCm, XPU and Ascend NPU.

cc @ivarflakstad @IlyasMoutawwakil - let me know if you're okay with this!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37678). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@IlyasMoutawwakil changes accepted!
[CI] add back `sacrebleu` (and document why)
Author: gante
Pull request: yes

# What does this PR do?

Reverts the removed dep in #37676 , and documents why

(TL;DR: `sacrebleu` is not in present `transformers`. However, calling `evaluate.load("sacrebleu")` requires it, and our `Trainer` tests use it under the hood, through the `run_translation.py` example)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37700). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

image pushed in https://github.com/huggingface/transformers/pull/37703, merging
[CI image] add back sacrebleu
Author: gante
Pull request: yes

# What does this PR do?

DO NOT MERGE

CI image builder for #37700 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37703). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Skip red Internvl tests
Author: Rocketknight1
Pull request: yes

Right now, a couple of InternVL tests are failing on `main`. I'm going to skip them for now to stop blocking other PRs, but it would be good to fix and un-skip them! cc @yonigozlan @zucchini-nlp

Comments:
Alright, will close this PR for now. Thanks @zucchini-nlp!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37711). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix: remove classmethod from `Qwen2_5OmniConfig.get_text_config`
Author: shahruk10
Pull request: yes

# What does this PR do?

  - Since the `get_text_config` references an instance variable within the class (`self.thinker_config`), the `get_text_config` method should not be a classmethod.

  - Before this fix, users were getting the following error:

    ''' AttributeError: type object 'Qwen2_5OmniConfig' has no attribute 'thinker_config' '''

  - This commit simply removes the `classmethod` wrapper from the `get_text_config` method

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37690). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks
[pipeline] allow image-text-to-text generate from text inputs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, passing empty list of images triggers `image_processor` call which in turn fails. It's because we usually check for `Noneness` in processors 


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37696). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hello @zucchini-nlp ! I have this PR that I completely forgot about https://github.com/huggingface/transformers/pull/35616, but it looks like it solves this issue. Added your tests to it, sorry about that!

@yonigozlan oh oke, cool. Can it be merged soon? Gemma3 team was asking why it doesn't work so would be nice to include in next release

Yes I asked for core maintainer approval, should be merged soon :)
[internvl] fix chat template
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, loading video with custom sampling has been changed recently and InternVL didn't have the update. Also PR modifies `content_image_token -> image_token`

@yonigozlan ideally we would have the same `fake_image_token` and `image_token`, which is used in vLLM to construct dummy text inputs. If we change it now, it won't be breaking as there was no release yet, WDYT? It will need updates in hub configs and chat template probably


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37656). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[tests, `qwen2_5_omni`] fix flaky tests
Author: gante
Pull request: yes

# What does this PR do?

cc @zucchini-nlp 

Fixes two `qwen2_5_omni` flaky issues:
1. Typo in the model name in `VLM_CLASS_NAMES` (used to skip tests) -> model was not being skipped -> flaky tests
2. `qwen2_5_omni` has a new config attribute corresponding to a token that can't be used in the output

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37721). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add Fast PVT Processor
Author: keetrap
Pull request: yes

Related #36978
cc @yonigozlan

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37204). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yonigozlan This test is failing here but passing locally.
`PvtImageProcessingTest::test_slow_fast_equivalence_batched - AssertionError: 0.0011574935633689165 not less than or equal to 0.001`
enable cpu offloading for Bark on xpu
Author: yao-matrix
Pull request: yes

**command**

pytest -rA tests/models/bark/test_modeling_bark.py::BarkModelIntegrationTests::test_generate_end_to_end_with_offload

**after this PR**
PASSED

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@ydshieh , added, pls help review, thx

Thanks. I updated with 2 commits [1a4557c](https://github.com/huggingface/transformers/pull/37599/commits/1a4557cfad65cefda36bfcd859d954f20bdb7e67) and [0911c14](https://github.com/huggingface/transformers/pull/37599/commits/0911c14dd4abe28dcdfc39f8ad43edc6f4582888)

> Thanks. I updated with 2 commits [1a4557c](https://github.com/huggingface/transformers/pull/37599/commits/1a4557cfad65cefda36bfcd859d954f20bdb7e67) and [0911c14](https://github.com/huggingface/transformers/pull/37599/commits/0911c14dd4abe28dcdfc39f8ad43edc6f4582888)

thx, i know how to do in next time for the deprecations cases. and you usage of `getattr` is simpler, great to learn along the journey, thx.
enable 4 test_trainer cases on XPU
Author: yao-matrix
Pull request: yes

below 4 cases passed
tests/trainer/test_trainer.py::TrainerIntegrationTest::test_galore_lr_display_with_scheduler
tests/trainer/test_trainer.py::TrainerIntegrationTest::test_galore_lr_display_without_scheduler
tests/trainer/test_trainer.py::TrainerIntegrationTest::test_schedulefree_radam
tests/trainer/test_trainer.py::TrainerIntegrationTest::test_use_liger_kernel_trainer


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @ydshieh

Actually, cc @IlyasMoutawwakil! We discussed internally and Ilyas will take on reviewing XPU PRs

For PRs like this, I can review too. It's just mainly about `require_torch_gpu` --> `require_torch_accelerator` and change some expected values most of the time.

If @IlyasMoutawwakil can share the tasks, more than welcome 🤗
SmolLM is ExecuTorch Compatible
Author: guangy10
Pull request: no
Labels: name: Feature request

name: ExecuTorch

### Feature request

### Feature request

Enable SmolLM to ["Export to ExecuTorch"](https://github.com/huggingface/transformers/issues/32253) workflow.

#### Instructions
Instructions of how to enable this model for ExecuTorch:
 1. Export the model to ExportIR. For LLM, to run with performance, typically you will need to export the model with cache. #34101 is a reference of how to export and validate the model. Note that you may run into some export issue and it may require fixes in the modeling code.
 2. Lower the model to ExecuTorch (to generate a .pte file). You will need to clone the github repo and create a recipe to lower the model. For example lowering the to XNNPACK is the simplest way. See the example code here: https://github.com/pytorch/executorch/blob/release/0.4/extension/export_util/export_hf_model.py#L89L106
 3. Run the model with ExecuTorch. You can follow these instructions to build and run the executor runtime for llama: https://github.com/pytorch/executorch/tree/release/0.4/examples/models/llama2#step-4-run-on-your-computer-to-validate

(Optional) Congrats! Once you complete step 1-3, you will be able to run the model on a host machine. Now if you would to go further like making the model faster, smaller, cheaper for your model use-case, you can create more complicated recipes with quantizations and delegations for different HW accelerators. You can find more tutorials on our website, for example to optimize and run the model with Core ML on Apple’s platform: https://pytorch.org/executorch/stable/build-run-coreml.html

### Motivation

See details in #32253

### Your contribution

TBD

Comments:
Hi,

I’m eager to contribute to this issue. As a beginner, I’d appreciate the opportunity to get involved.


> Hi,
> 
> I’m eager to contribute to this issue. As a beginner, I’d appreciate the opportunity to get involved.

@merronmuche I updated this task to include the instructions. Let me know if it’s helpful for you to get started.

> > Hi,
> > I’m eager to contribute to this issue. As a beginner, I’d appreciate the opportunity to get involved.
> 
> @merronmuche I updated this task to include the instructions. Let me know if it’s helpful for you to get started.

@guangy10,
Thank you for the quick response! I’ll let you know after reviewing the instructions you provided.

Done
Phi3 is ExecuTorch compatible
Author: guangy10
Pull request: no
Labels: name: Feature request

name: ExecuTorch

### Feature request

Enable Phi3-mini to ["Export to ExecuTorch"](https://github.com/huggingface/transformers/issues/32253) workflow

### Motivation

See details in #32253

### Your contribution

model enablement

Comments:
Done
[predict or eval error] There are cases where an error occurs when `preprocess_logits_for_metrics` and `gather_function` are executed.
Author: YooSungHyun
Pull request: no
Labels: name: bug

### System Info

<img width="619" alt="Image" src="https://github.com/user-attachments/assets/ca71a30e-6fde-4488-ad6f-f98e83247681" />

If I move the logits to the CPU in `preprocess_logits_for_metrics` due to GPU memory issues, it can cause an error in `gather_function`.

This happens because the tensors are no longer distributed across CUDA devices.  
I think there should be a condition like `if logits.is_cuda:` to handle this properly.  
Should I contribute a fix for this?

You can reproduce the issue by using the following code and running `predict()`:

```python
def preprocess_logits_for_metrics(logits, labels):
    return logits.to("cpu")
```

### Who can help?

@zach-huggingface @SunMarc 

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
def preprocess_logits_for_metrics(logits, labels):
    return logits.to("cpu")
```

run trainer.predict()

### Expected behavior

predict success

Comments:
my fault sorry!
IdeficsProcessor cannot handle multiple images in one text
Author: Kamichanw
Pull request: no
Labels: name: bug

### System Info

```
- `transformers` version: 4.48.1
- Platform: Linux-5.4.0-148-generic-x86_64-with-glibc2.31
- Python version: 3.11.11
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.2
- Accelerate version: 1.3.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA RTX A6000
```

### Who can help?

@zucchini-nlp @stevhliu 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi, there. I identified a critical bug on `IdeficsProcessor`. It is unable to process multiple images in a single text, i.e., `text="<image> <image>"` and `images=[[img], [img]]`. You can produce this bug with the following code:
```python
from transformers import AutoProcessor
processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-9b")
text = [
    "user:<image> What is in this image?\n assistant:",
    "user:<image> What is in this image?\n assistant:",
]
image = [
    ["https://hf-mirror.com/datasets/hf-internal-testing/fixtures_nlvr2/raw/main/image1.jpeg"],
    ["https://hf-mirror.com/datasets/hf-internal-testing/fixtures_nlvr2/raw/main/image1.jpeg"],
]
processor(text=text, images=image) 
```
It is not fixed in the latest version (4.49.0). Besides, due to change of processors, the [example](https://huggingface.co/docs/transformers/main/en/model_doc/idefics#transformers.IdeficsForVisionText2Text) in document should be updated .

### Expected behavior

Should work.

Comments:
@Kamichanw you have to batch the images correctly. So if the text has more than one image like `text=["<image> <image> what is here?", "<image> what color is cat?"]`, the images list will be smth like `[[img1, img2], [img3]]`

LMK if that works, and yeah, maybe we need to be explicit about this in the docs, if it is not there already, Feel free to open a PR in that case :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix torchao doc examples
Author: MekkCyber
Pull request: yes

# What does this PR do?
Fixes torchao docs examples after https://github.com/huggingface/transformers/pull/37592


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37697). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.