noamaanMulla-03/transformers-issues huggingface dataset:
Make SigLip implementation less weird
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/42759. WIP, I will fix more models that copied this pattern from siglip

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: siglip

iirc @yonigozlan started a few refactors but that was before the weight converter. Might be smart to sync

Clip definitely has a similar issue 😢 

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42841&sha=74e9f9

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42841). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Nice cleanup! As @vasqu mentioned, I started some refactors for vision models but paused them for now. I'll get back to them once the v5 refactoring is more stable.

Oh I remember, in that case we can close it and wait for the big refactor imo. The bug mentioned isn't affecting many users, not many reactions under the issue
No need to gather all parameters to save model when training with DeepSpeed Zero3 and LoRA
Author: Taise228
Pull request: no
Labels: name: Feature request

### Feature request

Gather only adapter weights to save model weights when training with LoRA and DeepSpeed Zero3.

### Motivation

When we use `Trainer` and train models with LoRA, trainer only saves adapter weights. However, when we use DeepSpeed Zero3, which distributes model weights on multi GPUs, and set `stage3_gather_16bit_weights_on_model_save: true`, current implementation tries to gather the whole model weights to rank0 in `self.accelerator.get_state_dict(self.deepspeed)`.

It is not optimal and also it can lead to OOM when training large models even if trained with LoRA.

### Your contribution

I think I can fix it by adding a few lines to `Trainer.save_model` as below:

before
```
elif self.is_deepspeed_enabled:
    try:
        state_dict = self.accelerator.get_state_dict(self.deepspeed)
```

after
```
elif self.is_deepspeed_enabled:
    try:
        accept_exclude_frozen_parameters = "exclude_frozen_parameters" in set(
            inspect.signature(self.model_wrapped.save_checkpoint).parameters.keys()
        )
        zero3_sharding = self.deepspeed.config.get("zero_optimization", {}).get("stage", None) == 3
        if accept_exclude_frozen_parameters and _is_peft_model(self.model) and zero3_sharding:
            # When using PEFT with DeepSpeed ZeRO Stage 3, 
            # we do not need to load the frozen parameters
            state_dict = self.deepspeed._zero3_consolidated_16bit_state_dict(exclude_frozen_parameters=True)
        else:
            state_dict = self.accelerator.get_state_dict(self.deepspeed)
```

If it is okay, I can take responsibility to file PR.

Comments:
cc @sunmarc @mekkcyber

Happy to have this @Taise228 ! Please submit a PR for that
[v5] Increase default shard size
Author: Wauplin
Pull request: no
Labels: name: Feature request

name: for_v5?

### TL;DR

It'd be greatly beneficial to increase the default shard size to align better with the new Hub recommendations + avoid repos with hundreds of shards BUT let's be careful on the implications in terms of I/O and optimizations.

---

### Feature request

Mentioned in https://github.com/huggingface/transformers/pull/42391 so I'm creating a separate issue to keep track of it.

_Original comment from @Wauplin:_

> currently max_shard_size defaults to 5GB. [Recommendations are now to use <50GB chunks](https://huggingface.co/docs/hub/storage-limits#recommendations) (the recommendation increased a lot since Xet is now the default). Should we increase the default value to e.g. 48GB? (not 50, to avoid having files slightly larger than 50).

Then from @LysandreJik (https://github.com/huggingface/transformers/pull/42391#issuecomment-3575540857):

> This is the only one I'm not sure about -> have smaller shards means that it's easier to parallelize load from checkpoints. Will let @ArthurZucker complete as he's been working with this quite a bit lately

from @ArthurZucker (https://github.com/huggingface/transformers/pull/42391#pullrequestreview-3505013594):

> have not really seen the exact line, but its something we do need to study: smaller shards -> better i/o file lock potentially?
>
> I am probably wrong as we can parallel read a file (which we did not do before) so should be fine with main but let's test it out please

finally from @Cyrilvallez (https://github.com/huggingface/transformers/pull/42391#issuecomment-3597973672):

> Agreed with @ArthurZucker here! In general it would indeed be very good to increase the size of the shards to a much larger value, to avoid having hundreds of files on big repos, and it's something that was raised previously. But we need to study carefully the implications on loading time and if it's going to impact us positively/negatively!

Comments:
Thanks @Wauplin for opening the feature request, and copy pasting the history! I will benchmark everything this week, then we can follow-up accordingly! But indeed, quite excited to (hopefully) be able to do this!
Simplify tie weights logic
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title! Originally, the logic should have tied both weights, but because there was a name mix-up, only the primary target was tied. This is however still logically correct, as then the original source will be tied later when reaching its turn in the mapping. So this PR removes the double tying when both are missing, as it's not needed logically! 
It is still correct to only check forward `for target_backup, source_backup in tied_keys[i + 1 :]:`, as if both are missing at this point, it means that we did not yet reach the actual weight, so it necessarily lives on a future pair.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42895). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove duplicated processor class from config
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, adding/adjusting tests and then ready for review

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42806). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`IGNORE`] Testing something
Author: vasqu
Pull request: yes

Dont merge

Comments:
run-slow: bart

This comment contains run-slow, running the specified jobs: 

models: ['models/bart']
quantizations: [] ...

run-slow: bart

This comment contains run-slow, running the specified jobs: 

models: ['models/bart']
quantizations: [] ...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41828). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: bart

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bart

This comment contains run-slow, running the specified jobs: 

models: ['models/bart']
quantizations: [] ...

run-slow: bart

This comment contains run-slow, running the specified jobs: 

models: ['models/bart']
quantizations: [] ...
refactor more tokenizers - v5 guide update
Author: itazap
Pull request: yes

following https://github.com/huggingface/transformers/pull/42563

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42768). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix gguf tokenizers
Author: SunMarc
Pull request: yes

# What does this PR do?

We need to fix how tokenizers are handled with gguf ...

### Run tests
`RUN_SLOW=True CUDA_VISIBLE_DEVICES=0 pytest -rA tests/quantization/ggml/test_ggml.py`



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42908&sha=2167b9

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42908). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix `Dtensor` and `tensor` mismatch
Author: 3outeille
Pull request: yes

# Bug

`local_rowise` or `local_colwise` is calling `RowiseParallel(use_dtensor=False)` (resp. `ColwiseParallel(use_dtensor=False)`. Issue was first noticed in https://github.com/huggingface/transformers/pull/42356 , quoting 

> we would like to not have Dtensor logic in the modeling. For example, sinks are supposed to use local_rowwise (cf [main/src/transformers/models/gpt_oss/configuration_gpt_oss.py#L41](https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt_oss/configuration_gpt_oss.py?rgh-link-date=2025-11-24T12%3A36%3A48Z#L41)) which is supposed to not return a Dtensor (cf [main/src/transformers/integrations/tensor_parallel.py#L1171](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/tensor_parallel.py?rgh-link-date=2025-11-24T12%3A36%3A48Z#L1171)) but somehow doesnt work

# Fix

```python
def convert_and_load_state_dict_in_model(
	#...
	tp_layer = ALL_PARALLEL_STYLES[model.tp_plan[matched_tp_pattern]].__class__
```

was creating the bug because `.__class__` will reuse the class default `use_dtensor` value. Thus, overwritting the value we specifed in `local_rowise/colwise`.

The fix makes sure to properly use the `use_dtensor` value and thus no more `Dtensor` and `tensor` mismatch 

 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42906). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42906&sha=12ff9a
Sam: Perception Encoder Audiovisual
Author: eustlb
Pull request: yes
Labels: name: New model

name: Audio

# What does this PR do?



Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, gemma3n, pe_audio, pe_audio_video, pe_video, timm_wrapper

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42905). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix add_dates script: Fetch github repo from url to check if model is new
Author: yonigozlan
Pull request: yes

As the title says, we used to fetch the repo with upstream/main, which doesn't work if the repo is not forked (?).
This way should be more robust

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42878). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
dynamic_module_utils: fix false imports from conditional blocks (#28459)
Author: ssmanjunath
Pull request: yes

# What does this PR do?

Fixes an issue where 'get_imports' incorrectly reported imports that only appear inside conditional blocks as mandaroty dependencies

This replaces brittle condition specific logic with a general AST-based control

flow check:
-- imports are collected only from code paths that are guarenteed to execute at module import time (top-level or 'if True:' blocks)
-- imports inside conditional or runtime dependent blocks are ignored
-- existing behaviour of skipping 'try' blocks is preserved

Fixes # 28459


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
A lot of this is cleaned up now and I don't think the problem still exists! Please be careful when pointing your code agent at old issues; they might just be accidentally left open.

Understood . Thanks for taking the time to clarify.
Support having multiple sub-processors (of any kind) in the same processor
Author: yonigozlan
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/41816.

Also standardizes the few remaining processors left to standardize.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42667). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: audioflamingo3, auto, fuyu, lasr, phi4_multimodal, pix2struct, pixtral

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42667&sha=1c7335
How about adding local kernel loading to `transformers.KernelConfig()`
Author: zheliuyu
Pull request: no
Labels: name: Feature request

### Feature request

As title.

### Motivation

Currently, the class `KernelConfig()` creates the `kernel_mapping` through the `LayerRepository` provided by `huggingface/kernels`. The `LayerRepository` downloads and loads kernel from the hub. I think adding the ability for it to load kernel locally should be very helpful for the debugging process.

### Your contribution

`huggingface/kernels` already has `LocalLayerRepository` built in. Maybe we should consider adding it to `KernelConfig()`.

Comments:
cc @MekkCyber 

Hi @zheliuyu ! Yes it totally makes sense, we can have a `use_local_kernels` flag inside `KernelConfig` to do that, do you want to open a PR for that ?

> Hi [@zheliuyu](https://github.com/zheliuyu) ! Yes it totally makes sense, we can have a `use_local_kernels` flag inside `KernelConfig` to do that, do you want to open a PR for that ?

Of course. Let's get started.

WIP on https://github.com/zheliuyu/transformers-kernels

https://github.com/huggingface/transformers/pull/42800 PR merged. Documentation will be updated accordingly, but this issue can be closed. 

Thanks to everyone who followed this issue.❤
Refactor: Improve clarity and robustness in route_tokens_to_experts
Author: danielquintas8
Pull request: yes

# What does this PR do?
This PR includes two small improvements for `route_tokens_to_experts` in MoE models:
- Improves Readability: Renames the function's input argument from `hidden_states` to `router_logits`. This more accurately reflects that the function receives the output of the gating network.
- Improves Robustness: Changes the `softmax` dimension from `dim=1` to `dim=-1`. While functionally identical in the current implementation, `dim=-1` is more explicit and ensures the logic remains correct if the tensor shape ever changes.
## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ArthurZucker 
@Cyrilvallez 


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: dbrx, ernie4_5_moe, flex_olmo, hunyuan_v1_moe, jamba, olmoe, qwen2_moe, qwen3_moe, qwen3_next, qwen3_omni_moe
fix(Qwen3VLProcessor): fix incorrect handling of return_metadata
Author: yiguanxianyu
Pull request: yes

This PR fixes the following:

- ensures video metadata is correctly returned in `Qwen3VLProcessor` when `return_metadata=True` during `processor.apply_chat_template()`.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_vl

cc @zucchini-nlp @molbap @yonigozlan

@yonigozlan recently added a cool feature for `BatchFeature` to skip converting inputs to tensors. He'll work on adding support for non array-like inputs next :)

Thanks for pointing out the issue @yiguanxianyu 

> @yonigozlan recently added a cool feature for `BatchFeature` to skip converting inputs to tensors. He'll work on adding support for non array-like inputs next :)
> 
> Thanks for pointing out the issue @yiguanxianyu

Thanks!
Fix PEFT gradient checkpointing with frozen input embeddings
Author: Aaraviitkgp
Pull request: yes

error: #42489
I have added a fallback check in gradient_checkpointing_enable, if the PEFT flag is missing, the code now checks if the input embeddings are frozen. If they are, it explicitly calls enable_input_require_grads().

@qgallouedec 

Comments:
@qgallouedec Th test_processors and test_tokenization error seem to have nothing to do with pr, is there a issue in ci pipeline, it happened before in my other pr aswell.
Issue 42197 resolve
Author: Aaraviitkgp
Pull request: yes

Fix offline model loading with subprocess cache warming

Improve cache resolution in cached_files() to properly locate models
downloaded in subprocess when loading offline. 

Changes:
- Enhanced cache directory lookup to check HF_HOME, TRANSFORMERS_CACHE, and HF_HUB_CACHE
- Added logic to search through snapshot directories when ref file is missing
- Return cached files early to avoid unnecessary download attempts

Tests:
- Test AutoModel/AutoTokenizer offline loading after subprocess download
- Test pipeline API offline loading with socket blocking
- Verify no network access attempts when cache is warm



Comments:
Rewrite for loop in get_image_features with torch ops for export
Author: jackzhxng
Pull request: yes

# What does this PR do?

Make Ministral 3, Llava, etc. torch exportable by rewriting for loop with torch ops.

## Who can review?

@molbap @yonigozlan @zucchini-nlp 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42822). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp should be good now 👍🏻 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llava, mistral3
[WIP] Video support in vLLM backend
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Enables https://github.com/vllm-project/vllm/pull/30680 from transformers side. I had this in my local draft for a very long time and some models still don't fit neatly, such as InternVL uses image token-id per video-frame processing or SmolVLM adds timestamps between each frame

I think we will support models that are easy first, in progress for now. 

fyi @hmellor 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: blip_2, got_ocr2, instructblip, instructblipvideo, internvl, llava_next_video, llava_onevision, perception_lm, qwen2_5_omni, qwen2_5_vl, qwen2_vl, smolvlm, video_llava

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42919&sha=ce0dfb

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42919). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Overwrite `get_decoder()` in AudioLLMs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, audio models need to call LM's `get_decoder()` because the model has no `base model`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42896). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@bot /style

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/20270111687).

Yes, also there is Qwen2-Audio but it already has the method overriden 

@bot /style

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/20299787341).

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: granite_speech

Voxtral had it overriden already 🤦🏻  I misread the GH comment, so it's only GraniteSpeech
Preprocessing fixes and more tests for LFM2-VL
Author: ankke
Pull request: yes

Fixes: 
* no image tokens for small images when `use_thumbnail` set to False
* repeated text part of the prompt for multi-image and multi-turn input

Adds several missing tests to catch bugs like above earlier.

Comments:
cc @zucchini-nlp @molbap @yonigozlan

run-slow: lfm2_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/lfm2_vl"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42784). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20165906930)

✅ No failing test specific to this PR 🎉 !


Tests are failing on main branch so I can't yet merge until they are fixed, let's wait

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: lfm2_vl
[`Tokenizers`] Change treatment of special tokens
Author: vasqu
Pull request: yes

Related to #39585 where we try to make some previously made special tokens to non-special tokens 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42903). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.