noamaanMulla-03/transformers-issues huggingface dataset:
Add AutoRound quantization support
Author: wenhuach21
Pull request: yes

This PR is intended to add support for AutoRound quantizer to the transformers library.
Paper https://arxiv.org/abs/2309.05516
Git https://github.com/intel/auto-round


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@IlyasMoutawwakil  Thanks for the review. I’ll address the remaining issues in the following commits.

> Thanks for the PR ! I'll review asap but please fix the CI first

CI passed. Please help review. Thanks. @SunMarc 

Thanks @wenhuach21 for this huge work 🔥 left some questions and nits above

> Thanks @wenhuach21 for this huge work 🔥 left some questions and nits above

Thank you for the review. I’ll address the issues after I return to work tomorrow.

Thanks for iterating @wenhuach21 and @hshen14, can you please add the `auto-round` lib to the ci docker for the tests here : https://github.com/huggingface/transformers/blob/main/docker/transformers-quantization-latest-gpu/Dockerfile

> Thanks for iterating @wenhuach21 and @hshen14, can you please add the `auto-round` lib to the ci docker for the tests here : https://github.com/huggingface/transformers/blob/main/docker/transformers-quantization-latest-gpu/Dockerfile

Done! AutoRound 0.5.0 will be released soon, just finishing up a bit more testing on our side.

Thanks a lot for your patience 🤗 

@SunMarc @MekkCyber @IlyasMoutawwakil  Thanks for the kind review! AutoRound v0.5.0 has been released. This PR should be good to merge if there are no further comments and all related tests pass. The current failure doesn't appear to be related to my changes.
Fixing quantization tests
Author: MekkCyber
Pull request: yes

# What does this PR do?

Fixes many quantization failling tests

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37650). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
add fromjson to jinja environments
Author: yhg0112
Pull request: yes

# What does this PR do?

Adds a `fromjson` filter to the Jinja environments in `chat_template_util`.  
This is particularly useful for handling tool use data that is stored as JSON strings.

## Who can review?

@Rocketknight1



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hi @yhg0112, can you give me an example of when this is useful? We generally recommend that tools are [passed as dicts, not JSON strings](https://huggingface.co/docs/transformers/main/chat_extras)!

Here are my useful cases:
1. directly applying  apply_chat_template to 'function calling' datasets (e.g. https://huggingface.co/datasets/Team-ACE/ToolACE, https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k, or https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2)
    * datasets that include a lot cases of JSON strings because we cannot save mixed struct of various function descriptions as dict, but only with JSON strings.
    * we can easily handle those with jinja template if it supports 'from_json'
        * e.g. loop over list of tools and putting some tags for each
2. real-world users may insert JSON strings into their inputs since a lot of APIs are communicating with JSON strings.


Hmm, I guess it might be useful! cc @Narsil @xenova since we might need support in `@huggingface/jinja` and `minijinja` as well, WDYT?

Adding support in `@huggingface/jinja` will be very easy since it's simply a call to `JSON.parse`. 👍 

In terms of standardization, is `fromjson` used in other engines, or will this be specific to HF transformers?

It isn't used in other engines - I'm a little hesitant to add it as a result, especially because it contradicts our preferred API for tools a bit, but I thought I should get some second opinions just in case!

> It isn't used in other engines - I'm a little hesitant to add it as a result, especially because it contradicts our preferred API for tools a bit, but I thought I should get some second opinions just in case!

I understand. I needed it for my convenience as a LLM trainer & data engineer.
This may cause a confusion to other users (but still it helps a lot).

I think for the moment we'd prefer not to add it (especially since it's not a standard Jinja filter and there are several other Jinja implementations we want to keep in sync), but we may change our minds in future if it becomes a popular request!
Fix autoround docs 
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR adds autoround docs in the _toctree file 

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37675). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix no_split_modules for Llama4 pretrained models
Author: astefanutti
Pull request: yes

# What does this PR do?

This PR moves the definition of `_no_split_modules` for Llama4 pre-trained models from the base `Llama4PreTrainedModel` class to the subclasses so each model has the correct set of modules.

Otherwise loading the `Llama4ForCausalLM` currently fails because it doesn't have the `Llama4VisionEncoderLayer` module.

Fixes #37672

## Who can review?

@ArthurZucker @amyeroberts @qubvel

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@zucchini-nlp thanks, I've just rebased and re-pushed.

The CI error seems related to #37675.

Hey @astefanutti, should be fixed now ! you can just update the branch

@MekkCyber thanks for the update, I've just rebased on it.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37673). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Refactor bitsandbytes doc
Author: MekkCyber
Pull request: yes

# What does this PR do?
Refactor bitsandbytes docs for better readability

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37668). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
This PR causes Transformers to error out when a model is using Tensorflow and the environment does not provide `torch` in any way
Author: rishic3
Pull request: no

              This PR causes Transformers to error out when a model is using Tensorflow and the environment does not provide `torch` in any way

transformers/src/transformers/pipelines/base.py:

```
        if torch.distributed.is_initialized():
```

[Source](https://github.com/huggingface/transformers/blame/530322ccb6d109ba1383e852b17729a2185d816b/src/transformers/pipelines/base.py#L984)

_Originally posted by @radoslav-dimitrov-indeavr in https://github.com/huggingface/transformers/issues/37307#issuecomment-2786494331_
            

Comments:
[cleanup] remove old scripts in `/scripts` 🧹 🧹 
Author: gante
Pull request: yes

# What does this PR do?

Removes old scripts in `/scripts` that have no cross-references in other parts of the code base nor recent updates, as chatted with @LysandreJik 

Scripts removed:
- `scripts/benchmark/trainer-benchmark.py`: many trainer flags were added since last relevant commit (>3 years ago)
- `scripts/deberta_scrtipt.py`: timing benchmark script related to old model, deberta v3 (`torch.jit`)
- `scripts/fsmt`: contains scripts with commands that no longer exist (`transformers-cli upload`) or tasks that we've automates since (tiny model creation)
- `scripts/pegasus/build_test_sample_spm_no_bos.py`: script to create a fixture file that we haven't touched in 5 years
- `scripts/tatoeba`: script to convert [Tatoeba-Challenge](https://github.com/Helsinki-NLP/Tatoeba-Challenge) models (challenge that has long finished)

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37676). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[docs] only build `en` docs in push CI
Author: gante
Pull request: yes

# What does this PR do?

In our push CI, restrict the doc builder to `en`
- All languages: ~26 mins ([data point](https://github.com/huggingface/transformers/actions/runs/14597021526/job/40945447156))
- `en`: ~12 mins (this PR)

Pros:
- `en` has the full doc reference to class/methods (in other languages it is a copy)
- source of truth is tested at a fraction of the time
- `main` still builds all languages (different CI workflow)

Cons:
- PRs to other languages won’t build the corresponding docs at push time

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37677). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Cyrilvallez we can create a script in `utils` that, from the diff, detects the changes languages in `/docs` and prepares the CI job accordingly. Somewhat similar to our test fetcher.

(But that's a task that I wouldn't like to commit time until we realize it's needed :D Perhaps a `v2` of this PR, if we realize other languages are causing issues?)

I'm eagerly merging to speed up the merge cycle, happy to revert/improve if we decide we need a more elaborate mechanism (see comment above)
[Docs] Move models to appropriate section
Author: NielsRogge
Pull request: yes

# What does this PR do?

I noticed several multimodal models are in the text-only section, hence this PR moves them.

cc @stevhliu


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37338). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I'm not sure why the test keeps failing, the `toctree` looks properly sorted to me 🤔 

Me neither, @ydshieh do you have an insight here? `Make style` passes locally for me

Not sure, but 

> python  utils/check_doc_toc.py  --fix_and_overwrite

give me

https://github.com/huggingface/transformers/pull/37338/commits/f32d39a2a60cc36d6e8756124cb3da7057db1a93

will merge once CI ok
🌐 [i18n-KO] Translated `siglip.md` to Korean
Author: devxaitist
Pull request: yes

# What does this PR do?

Translated the `siglip.md` file of the documentation to Korean.
Thank you in advance for your review.

Part of https://github.com/huggingface/transformers/issues/20179

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 본인의 조 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요!-->
May you please review this PR?
@cjfghk5697, @yijun-lee, @rlaalsrl0922 , @MinJu-Ha

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. N조 팀원들과 리뷰가 끝난 후에 아래 주석을 노출해주세요! -->
<!-- @stevhliu May you please review this PR? -->

Comments:
cc @stevhliu 

> Thanks, we can merge once a Korean reader has reviewed!

Thank you! I've updated the title to kebab-case format as suggested. Looking forward to having it merged after a Korean reader's review. I appreciate your guidance on my first open source contribution.

Hey @devxaitist 👋
Thanks for the translation. I'll review it tonight!

In my opinion, only the two parts above need to be revised!

> In my opinion, only the two parts above need to be revised!

Thank you for your feedback. I have made changes to reflect your feedback, please review.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37145). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Merging since the requested changes have been applied. Thanks again everyone! 🤗
[docs] Model docs
Author: stevhliu
Pull request: yes

🚧 WIP 🚧

Explores a new design for several of the most commonly visited model docs. It is pretty stripped down at the moment, but we can add back the essentials as needed (or not). The main idea is to only show developers what they need to know to use the model.

- A badge indicating whether a model supports certain features/frameworks (PyTorch, SDPA, FlashAttention, etc.) in the upper right side of the doc.
- A brief description of the model and what makes it different. Some of these abstracts just don't tell you that much about the model itself or what makes it unique, while others are very technical and the details probably don't matter that much if you aren't a researcher. The description aims for somewhere in between.
- Code snippets demonstrating inference with all the optimizations included.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36469). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
enable mllama cases on xpu
Author: yao-matrix
Pull request: yes

5 cases pass on both XPU and A100.

tests/models/mllama/test_modeling_mllama.py::MllamaForConditionalGenerationIntegrationTest::test_11b_model_integration_batched_generate
tests/models/mllama/test_modeling_mllama.py::MllamaForConditionalGenerationIntegrationTest::test_11b_model_integration_forward
tests/models/mllama/test_modeling_mllama.py::MllamaForConditionalGenerationIntegrationTest::test_11b_model_integration_generate
tests/models/mllama/test_modeling_mllama.py::MllamaForConditionalGenerationIntegrationTest::test_11b_model_integration_generate_text_only
tests/models/mllama/test_modeling_mllama.py::MllamaForConditionalGenerationIntegrationTest::test_11b_model_integration_multi_image_generate


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

@ydshieh , pls help review, thx.

cc @ydshieh
enable 6 granite cases on xpu
Author: yao-matrix
Pull request: yes

**cases**

tests/models/granitemoe/test_modeling_granitemoe.py::GraniteMoeIntegrationTest::test_model_3b_generation
tests/models/granitemoeshared/test_modeling_granitemoeshared.py::GraniteMoeSharedIntegrationTest::test_model_3b_generation
tests/models/granite/test_modeling_granite.py::GraniteIntegrationTest::test_model_3b_logits
tests/models/granitemoe/test_modeling_granitemoe.py::GraniteMoeIntegrationTest::test_model_3b_logits
tests/models/granitemoeshared/test_modeling_granitemoeshared.py::GraniteMoeSharedIntegrationTest::test_model_3b_logits
tests/models/granite/test_modeling_granite.py::GraniteIntegrationTest::test_model_3b_logits_bf16

**results**
All pass on XPU, XPU has similar numerical behavior w/ A100, so we can see A100 has similar ground-truth as XPU.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

cc @ydshieh

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37569). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ydshieh, done, thx.

Thank you.

(In the future, let's try to put the format like https://github.com/huggingface/transformers/pull/37569/commits/c83a02aabb17d4a1c16bed0d15de80927c33f26f)

> Thank you.
> 
> (In the future, let's try to put the format like [c83a02a](https://github.com/huggingface/transformers/commit/c83a02aabb17d4a1c16bed0d15de80927c33f26f))

sure, will follow your style
enable blip2 and emu3 cases on XPU
Author: yao-matrix
Pull request: yes

5 cases:
FAILED tests/models/emu3/test_modeling_emu3.py::Emu3IntegrationTest::test_model_generate_images
PASSED tests/models/emu3/test_modeling_emu3.py::Emu3IntegrationTest::test_model_generation_batched
PASSED tests/models/emu3/test_modeling_emu3.py::Emu3IntegrationTest::test_model_generation_multi_image
PASSED tests/models/blip_2/test_modeling_blip_2.py::Blip2TextRetrievalModelTest::test_model_from_pretrained
PASSED tests/models/blip_2/test_modeling_blip_2.py::Blip2VisionModelWithProjectionTest::test_model_from_pretrained

will fix 1 failed case in separate PR

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.
Add Fast Image Processor for VideoMAE 
Author: miguelscarv
Pull request: yes

# What does this PR do?

This PR implements a FastImageProcessor for the the VideoMAE model using BaseImageProcessorFast. Related #36978 

Note:
1. I was not able to run `test_can_compile_fast_image_processor` as I'm using a M1 and don't have access to a NVIDIA GPU.
2. The test implemented in the method `test_slow_fast_equivalence_batched` is only able to pass when all images in the batch have a fixed width and height (by setting `equal_resolution=True` in the `self.image_processor_tester.prepare_image_inputs`). If the width and height vary across videos, then there is a low chance of passing this test, i.e. it doesn't fail every time.

Other than that, all tests passed.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@yonigozlan 

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Hey @yonigozlan 

Saw in #37611 that fast processors for video models aren't being merged, so guessing this PR won’t go through either.

That said, would a fast image processor make sense for SmolVLM? It handles high-res images, so it could help.
qwen2_5_omni initialize bug.
Author: kobenaxie
Pull request: no

As menthioned asd https://github.com/huggingface/transformers/pull/37690
`model = Qwen2_5OmniForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-Omni-7B")` get error `AttributeError: type object 'Qwen2_5OmniConfig' has no attribute 'thinker_config'`
https://github.com/huggingface/transformers/blob/1d9743edc2030e8444d5bdffa910a3d6822bcf2d/src/transformers/models/qwen2_5_omni/configuration_qwen2_5_omni.py#L1061

`pip install git+https://github.com/huggingface/transformers@1d9743edc2030e8444d5bdffa910a3d6822bcf2d`

Comments:
PR merged, closing as resolved! We'll add the fix in the model preview tag