noamaanMulla-03/transformers-issues huggingface dataset:
[loading] Correctly load params during offloading & careful memory considerations
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

During offloading, we are under heavy cpu memory constraints: we have disk offloading because the cpu/devices cannot hold all parameters. Currently, all parameters would be loaded on cpu async, then converted, then resaved if needed, which results in killing the cpu ram and crashing.

Loading asynchronously means we assume that we can hold all parameters in memory, which is true in general if we want to be able to load the model anyway. But with offloading, this assumption does not hold anymore, as we *know we don't have enough memory* so we offload to disk. For this reason, threading is deactivated when offloading is needed, as we cannot control the memory otherwise.

Moreover, this PR changes the logic when we do not use threading from gathering all tensors on cpu before, then converting them all, to loading them on cpu only when entering the Converter. This is essential, as during offloading, as explained above, if we load everything first, we kill the cpu RAM. 
In terms of logic, everything is sequential in both cases, so it has no impact. We just move from:
- loading everything sequentially -> converting everything sequentially with everything in RAM
to
- for each Converter: load all related params sequentially -> convert -> move on to next Converter

This allows to lift all memory issues when offloading, as only the required tensors are loaded, then we convert, resave the converted param to disk, and move on to the next one.
It's basically only a smart reordering but has not other impact on speed etc

Also, this PR makes sure that we never use more memory than necesary during conversion ops, especially if several are chained, by not keeping any reference to the internal `self.collected_tensors` attribute, and deleting refs to collected tensors in the loading loop.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42632). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Use hfh's is_offline_mode helper
Author: Wauplin
Pull request: yes

Following up on https://github.com/huggingface/huggingface_hub/pull/3593 (released in `v1.2.1`). 

Nothing changes, just moves the helper to huggingface_hub for consistency.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42657). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Let transformers know when a model is being traced via jax.jit (torchax)
Author: qihqi
Pull request: yes

# What does this PR do?

transformers current recognizes when the PyTorch model is 
being traced instead of executed, so it knows to not use untraceable ops (data-dependent ops). 
However, currently it recognizes torchscript, fx trace, dynamo but does not recognize `jax.jit` via torchax.

This PR adds detecting jax.jit.  

Fixes https://github.com/google/torchax/issues/56


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@vasqu @ArthurZucker @CyrilVallez


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42611). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

CI issues are unrelated and seem to be transient connection timeouts - @cyril do you want to force merge, or will we just wait and rebase?

> LGTM! Would just like to add it to the docstring as well if you don't mind! 🤗

done.

Thanks! Merged it!
Loading a checkpoint with tied weights results in `ValueError: This checkpoint seem corrupted.`
Author: tomaarsen
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.0.0.dev0
- Platform: Windows-10-10.0.26100-SP0
- Python version: 3.11.13
- Huggingface_hub version: 1.1.5
- Safetensors version: 0.6.2
- Accelerate version: 1.11.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.0+cu126 (CUDA)
- Using distributed or parallel set-up in script?: No
- Using GPU in script?: No
- GPU type: NVIDIA GeForce RTX 3090

### Who can help?

@Cyrilvallez @molbap 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction
```python
from transformers import AutoModel

model_name = "sentence-transformers/sentence-t5-base"
model = AutoModel.from_pretrained("sentence-transformers/sentence-t5-base")
print(type(model))
```
Currently, the behaviour on `main` is:
```
Traceback (most recent call last):
  File "c:\code\transformers\demo_sentence_t5.py", line 5, in <module>
    model = AutoModel.from_pretrained("sentence-transformers/sentence-t5-base")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\transformers\src\transformers\models\auto\auto_factory.py", line 373, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\transformers\src\transformers\modeling_utils.py", line 275, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\transformers\src\transformers\modeling_utils.py", line 3977, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys, offload_index, error_msgs = cls._load_pretrained_model(
                                                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\transformers\src\transformers\modeling_utils.py", line 4163, in _load_pretrained_model
    model.tie_weights(missing_keys=missing_keys, recompute_mapping=False)
  File "C:\code\transformers\src\transformers\modeling_utils.py", line 2337, in tie_weights
    raise ValueError(
ValueError: This checkpoint seem corrupted. The tied weights mapping for this model specifies to tie shared.weight (which should be present and is not), to encoder.embed_tokens.weight (which is present).
```

### Expected behavior

I expect the model to load normally, as it had been doing for <v5.0.


- Tom Aarsen

Comments:
Hey @tomaarsen 👋 

this is expected for now but @Cyrilvallez wanted to add a PR to fix this. The core issue is that while the tied mapping says x should exist, it's the other way around. 

Related PRs #42377, #42355

Should be fixed by #42507

Can confirm, fixed!
Parallelize the model download with the weight loading
Author: LysandreJik
Pull request: no
Labels: name: Core: Modeling

name: Feature request

### Feature request

It could be cool to load each individual shard as it finishes being downloaded instead of waiting for all the shards to be downloaded before loading them.

Right now, with the dynamic weight loading, we see a significant improvement in model loading time; this could help push this further.

cc @Cyrilvallez and @ArthurZucker probably

### Motivation

Speed!

### Your contribution

I can test it out 😉 

Comments:
Hi @LysandreJik, I'd like to take a shot at implementing this. A few questions before I start:

1. Which file/module handles the shard downloading and loading logic currently?
2. How does the existing "dynamic weight loading" work? Any specific files I should study first?
3. Are there any edge cases to consider (.g., failed downloads, memory constraints, dependency between shards) ?
4. Would you prefer an async-based approach or threading for parallel download + load?

Thanx!

Hey @Aznix07! For this kind of core features, we are usually extremely picky and prefer to do it ourselves as it requires deep knowledge and understanding of the whole Transformers loading mechanisms... This is not intended to discourage you, but rather a friendly warning so that you do not waste your time in trying to work on this! It's definitely not a good one to start working on if you do not have any prior knowledge of the loading pipeline, as it's too involved between many different aspects of the library!
[`mRope`] Fix warnings
Author: vasqu
Pull request: yes

As per title, `mrope_section` was the keyword, not `mrope` leading to incorrect warnings

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4v, glm4v_moe, qwen2_5_omni, qwen2_5_vl, qwen2_vl

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42660). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
fix: skip weight initialization for quantized models
Author: arrdel
Pull request: yes

# What does this PR do?

Fixes #39366 

This PR resolves a `RuntimeError` when loading llmcompressor W8A8 quantized models by skipping weight initialization for quantized models.

## Problem

When loading quantized models with int8/uint8 weights, the model initialization fails with:

RuntimeError: expected a floating-point or complex dtype, but got dtype=torch.int8

This occurs because `_initialize_missing_keys()` unconditionally calls `initialize_weights()`, which attempts to use `normal_()` on int8 tensors. PyTorch's `normal_()` only supports floating-point dtypes.

## Solution

Added a `not is_quantized` check to the else branch in `_initialize_missing_keys()` method (line 4448 of `modeling_utils.py`):

```python
elif not is_quantized:
    self.initialize_weights()

Comments:
That's quite an old issue, have you checked that it still exists?

A lot of refactor happened since the issue was posted. I don't think this is relevant anymore, so i'm closing this !
Fix: Skip weight initialization for quantized int8 models
Author: AnjaliiD
Pull request: yes

This PR fixes the RuntimeError encountered when loading quantized models (int8 weights). 

The issue occurs because PyTorch’s `normal_()` cannot operate on integer tensors. 

Changes:
- Skip weight initialization for quantized models in `_load_pretrained_model()`
- Print a small info message when skipping

Fixes #42574

- @SunMarc
- @MekkCyber

Comments:
A lot of refactor happened since the issue was posted. I don't think this is relevant anymore, so i'm closing this !
Deprecate NeptuneCallback
Author: tomaarsen
Pull request: no

Hello!

## Overview
* We should deprecate the [NeptuneCallback](https://github.com/huggingface/transformers/blob/fccb04998af44166524c2a7cfe0d946b99aef497/src/transformers/integrations/integration_utils.py#L1455) as they'll shut down by March 5, 2026

## Details
I've been informed via email that [Neptune](https://neptune.ai/) is shutting down starting March 5, 2026. Perhaps we can add a soft deprecation to the callback for the time being, and fully remove it in early 2026? We can also fully remove it with the v5.0 release, as that's a natural moment to move forward with deprecations.

- Tom Aarsen

Comments:
Remove Neptune integration references and deprecate `NeptuneCallback`
Author: qgallouedec
Pull request: yes

<img width="607" height="455" alt="image" src="https://github.com/user-attachments/assets/b61dfc69-3d69-4ba6-b17b-7dc5d89d3b4c" />


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42666). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add VideoMAEImageProcessorFast
Author: vasanthrpjan1-boop
Pull request: no

Feature Request

### Description

Add a fast image processor for VideoMAE (`VideoMAEImageProcessorFast`) that uses `torch` and `torchvision` instead of PIL/numpy for better inference performance.

### Motivation

- VideoMAE currently only has a slow image processor (`VideoMAEImageProcessor`) using PIL/numpy
- Adding a fast processor using torch/torchvision will significantly improve inference speed
- This aligns with the ongoing effort to add fast processors to vision models (referenced in #36978)
- Other similar models (ViT, DETR, etc.) already have fast processors

### Proposed Implementation

1. Create `src/transformers/models/videomae/image_processing_videomae_fast.py`
   - Inherit from `BaseImageProcessorFast`
   - Match the functionality of the existing slow processor
   
2. Update `src/transformers/models/videomae/__init__.py` to export the new class

3. Add tests to verify equivalence between slow and fast processors

### Example (based on ViTImageProcessorFast pattern)

from ...image_processing_utils_fast import BaseImageProcessorFast
from ...image_utils import IMAGENET_STANDARD_MEAN, IMAGENET_STANDARD_STD, PILImageResampling
from ...utils import auto_docstring

@auto_docstring
class VideoMAEImageProcessorFast(BaseImageProcessorFast):
    resample = PILImageResampling.BILINEAR
    image_mean = IMAGENET_STANDARD_MEAN
    image_std = IMAGENET_STANDARD_STD
    size = {"shortest_edge": 224}
    crop_size = {"height": 224, "width": 224}
    do_resize = True
    do_center_crop = True
    do_rescale = True
    do_normalize = True### Willingness to Contribute

I am willing to submit a PR for this feature.



Comments:
@vasanthrpjan1-boop VideoMAE already has a fast processing way with a video processor class (added in https://github.com/huggingface/transformers/pull/41534). For video models we will not be adding a FastImageProcessor and instead we'll slowly move to using VideoProcessors. Note that a video processor has no slow version and is always "fast", i.e. relies on torchvision for transforms



Thanks for the clarification! I wasn't aware that video models use `VideoProcessor` instead of the `ImageProcessorFast` pattern.

I see that #41534 already added the video processor for VideoMAE. Closing this issue as it's not needed.
Fix parallelism_config being overwritten in TP-only training
Author: arrdel
Pull request: yes
Labels: name: Code agent slop

Fixes #42661

## Problem
When using TP with user-provided parallelism_config, the Trainer overwrites the entire config object, discarding user settings.

## Solution  
- Update only tp_size attribute when config is provided
- Create new config only when none is provided
- Preserve all user-provided parallelism settings

## Changes
- Fixed logic in trainer.py lines 5074-5087
- Removed redundant nested checks
- User configs (dp_size, pp_size, cp_backend, etc.) now preserved

Comments:
Transformers 4.57.3 needs older huggingface-hub
Author: dlin95123
Pull request: no
Labels: name: Feature request

### Feature request

support latest huggingface-hub releases

### Motivation

I'd like to update my huggingface-hub from 0.36 to 1.2.1. However, transformers 4.57.3 needs huggingface-hub<1.0. So I cannot update the huggingface-hub. Can you please consider support newer version of huggingface-hub?

### Your contribution

not capable... sorry

Comments:
Transformers versions are usually tied to specific huggingface-hub versions! It generally won't work if you try to install them separately. Instead, just hang on for the next release - you can install the release candidate `v5` or the latest version from `main` if you really need the updated `huggingface-hub` right now!
FIX Error when trying to load non-LoRA PEFT
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

This PR fixes a bug that prevented non-LoRA PEFT adapters to be loaded into a transformers model. It was just a very simple logical error where a check was performed generally when it actually only is relevant for hotswapping. A test for this has been added.

Additionally, also testing if a non-LoRA adapter can be added to a transformers model. This was not broken but still lacked test coverage.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42663). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Not sure who would be best suited to review this from among the transformers devs, pinging @Cyrilvallez because you reviewed the original hotswap PR.

Personally, I'd consider this a regression introduced by #41297 so ideally it could be part of the v5 release :)

> Alright, LGTM then! Will merge as the failing test is unrelated

Fantastic, thanks. Anything I should do to ensure that this is included in v5?

Nop, will be automatically in the next rc1 release!
Fix PEFT integration with new weight loader
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

As per the title. The `VLMS` list should not be duplicated in `peft.py` at it's not getting updated! Also fix it when it's not coming from `from_pretrained`!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42701). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42701&sha=9dba7f
Add is_flash_attn_4_available() detection functionAdd is_flash_attn_4_available() detection function
Author: vasanthrpjan1-boop
Pull request: yes
Labels: name: Code agent slop

## What does this PR do?

Partial contribution for #42405

This PR adds the `is_flash_attn_4_available()` detection function for Flash Attention 4 (Blackwell/Hopper GPUs).

## Changes

Added to `src/transformers/utils/import_utils.py`:

@lru_cache
def is_flash_attn_4_available() -> bool:
    """
    Check if Flash Attention 4 is available.

    FA4 requires:
    - CUDA availability
    - SM 8.0+ GPU (Hopper: H100/H200, or Blackwell architecture)
    - flash_attn package with the cute module
    """Also exported in `src/transformers/utils/__init__.py`.

## Scope

This is a **partial contribution** - just the detection function. The full FA4 integration requires:
- [ ] API compatibility layer for FA4 vs FA2/FA3
- [ ] Auto-selection priority logic
- [ ] Registration in AttentionInterface
- [ ] Testing (requires H100/H200/Blackwell hardware)

I don't have access to H100/H200 hardware, so I'm contributing what I can. The remaining integration can be done by someone with the appropriate hardware.

## Before submitting

- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case)
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md)?
- [x] Did you make sure to update the documentation with your changes?
- [ ] Did you write any new necessary tests? *(N/A - detection function only, needs hardware to test)*

cc @sambhavnoobcoder (issue author)

Comments:
Docs: clarify testing dependencies installation in AGENTS.md
Author: OliverZhangA
Pull request: yes

This PR improves clarity in the `AGENTS.md` documentation regarding how to install the necessary testing dependencies.

- Clearly separates the installation of testing extras.
- Adds explicit instructions for installing PyTorch and Accelerate when they are not already available.
- Improves readability and aligns the section with the documentation style used across the project.

This is a minor documentation improvement and does not affect any code.

Comments:
Remove duplicated imports
Author: AgainstEntropy
Pull request: yes


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: smolvlm

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42689). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
update gemma3 exepectations and switch to dynamic cache
Author: Abdennacer-Badaoui
Pull request: yes

When using the ROCm platform, some tests that process multimodal inputs with a static cache produce incomplete outputs. They generate only one token and then immediately output the EOS token, because the logits become NaN.

It appears that the issue comes from `index_copy_` operations in `cache_utils.py`, which fail silently. Replacing these operations with slice assignment, which is more reliable, makes some of these tests pass. It may also be related to the model being compiled while using a static cache. However, changing the cache system for a single model failing on a single hardware platform is probably not a good idea.

For now, I switched to the dynamic cache, which produces correct outputs.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42688). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3

run-slow: gemma3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/gemma3"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20030320819)

✅ No failing test specific to this PR 🎉 !