noamaanMulla-03/transformers-issues huggingface dataset:
Warning: You are using a model of type sam3_video to instantiate a model of type sam3_tracker
Author: gboeer
Pull request: no
Labels: name: bug

### System Info

When using the sample code to create a SAM3 Tracker Model like:
```python
from transformers import Sam3TrackerProcessor, Sam3TrackerModel

model = Sam3TrackerModel.from_pretrained("facebook/sam3")
processor = Sam3TrackerProcessor.from_pretrained("facebook/sam3")
```

The following warning will be displayed:
```ba  
You are using a model of type sam3_video to instantiate a model of type sam3_tracker. This is not supported for all configurations of models and can yield errors.
```

This happens with version `5.0.0.dev0`


### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import Sam3TrackerProcessor, Sam3TrackerModel

model = Sam3TrackerModel.from_pretrained("facebook/sam3")
processor = Sam3TrackerProcessor.from_pretrained("facebook/sam3")
```

### Expected behavior

Idealy the model should be instantiated without a Warning.

Comments:
cc @nielsrogge @yonigozlan is this actually supported? I think this error might just be legitimate.

Yes that's a known issue. The problem is that there is only one sam3 checkpoint for 4 different model architectures. We might want to  add a way to detect these cases and skip the warning. Thanks @gboeer for raising the issue!

So for a fix can we add a parameter in the config like `compatible_model_types` and then check it before the warning ? 
If so I can submit a PR for it 

Is there a known workaround for this issue?


@Nailujj 
It's not really a serious issue, besides the warning popping up. There also is a PR currently in review which will omit the warning.
Remove SDPA workarounds for torch 2.4+
Author: cyyever
Pull request: yes

# What does this PR do?
https://github.com/pytorch/pytorch/issues/112577 was fixed in torch 2.2. Let's remove the workarounds and see what happen in the tests.
 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: diffllama, falcon, kyutai_speech_to_text, mimi, moshi, nemotron

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43754). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix EP post merge
Author: ArthurZucker
Pull request: yes

# What does this PR do?

## Fix GPT-OSS ???
EP sharding works, the forward is broken for eager (does not work) and output giberissh for grouped_mm. 
Will work on a fix to have in the patch! 

## Fix any TP model 
- fix get shard tensor 
- fix get packed shard weights
- fix `tensor_idx`


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43730). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: glm4_moe_lite, gpt_oss, minimax_m2, solar_open, fbgemm_fp8

lgtm nice job !
[Trainer] Move sort and rotate checkpoints to standalone functions
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR do the following:

- Extract `sorted_checkpoints` and `rotate_checkpoints` from Trainer class to standalone functions in `trainer_utils.py`
- Rename `sorted_checkpoints` to `sort_checkpoints`
- Simplify implementations for better readability
- Add and update tests 

Part of PRs to unbloat Trainer

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43736). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@bot /style 

Style fix bot fixed some files and pushed the changes.

cc @winglian
Activation offloading for Trainer
Author: qgallouedec
Pull request: no
Labels: name: Feature request

### Feature request

Activation offloading is implemented in TRL. It's common to all trainers, so it could be nice to upstream this feature.

https://github.com/huggingface/trl/blob/43fb8d310633448a0c4c731a2efe9c1ca55e6184/trl/trainer/sft_trainer.py#L898

(cc @kashif, IDK how extensively it's tested)


### Motivation

Activation offloading is a memory efficiency technique that reduces GPU VRAM usage by temporarily moving activation tensors to CPU RAM

### Your contribution

Not on this one, but @kashif is probably able to help

Comments:
Hi, I'd like to do this. I’ve submitted a PR that addresses it: #43774.
Please let me know if any further adjustments are needed or if I missed something.
[docs] Update torch version
Author: stevhliu
Pull request: yes

Addresses #41960 to update to torch 2.3

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42135). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @Rocketknight1, one more torch 2.4 update! 🙏
[docs] deploying
Author: stevhliu
Pull request: yes

adds ecosystem integration docs for deploying with Candle, ExecuTorch, and MLX

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43241). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove CompressedLinear support for compressed-tensors > 0.13
Author: colldata79
Pull request: yes

Title: Remove CompressedLinear support for compressed-tensors > 0.13

Body:


## What does this PR do?

Prepares transformers for the removal of `CompressedLinear` from compressed-tensors (v0.14+). Users should now call `model.dequantize()` after loading to decompress quantized models.

**Changes:**
- Stop passing `run_compressed` to `apply_quantization_config`
- Always decompress models after loading for compressed-tensors > 0.13
- Add `_dequantize` method to `CompressedTensorsHfQuantizer`
- Remove tests that reference deleted `CompressedLinear` class (`test_default_run_compressed__True`, `test_default_run_compressed__False`)

Backwards compatibility is maintained for compressed-tensors ≤ 0.13.

Related to vllm-project/llm-compressor#2279
Companion PR: vllm-project/compressed-tensors#564

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case. → vllm-project/llm-compressor#2279
- [x] Did you make sure to update the documentation with your changes?
- [x] Did you write any new necessary tests?

## Who can review?

@SunMarc @MekkCyber (quantization)

Comments:
CI failure in test_processing_layoutlmv2.py is unrelated to this PR - no tokenizer/processor code was modified

can a  maintainer please  verify

cc @dsikka 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: compressed_tensors_integration

@kylesayrs ,Fix is in place . please go ahead with the end to end test
[Trainer] Move NEFTune impl to standalone functions
Author: SunMarc
Pull request: yes

# What does this PR do ?

This PR refactor NEFTune implementation from Trainer methods to standalone functions. Part of PRs to unbloat Trainer. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43714). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @winglian

@bot /style 

Style fix bot fixed some files and pushed the changes.
Improved agents
Author: tarekziade
Pull request: yes

# What does this PR do?

Tweaks to `AGENTS.md` to make AI tools more efficient


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43763). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: diffllama, edgetam, falcon, kyutai_speech_to_text, mimi, moshi, mt5, nemotron, sam2, sam3, sam3_tracker, t5, youtu
Fix-release-ubild
Author: ArthurZucker
Pull request: yes

# What does this PR do?
We had to do this twice already! 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: sam, sam_hq

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43773). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix `convert_rope_params_to_dict` so it uses `rope_theta` from the config
Author: hmellor
Pull request: yes

`convert_rope_params_to_dict` assumes that `rope_theta` and `partial_rotary_factor` will be present in `kwargs`. This is only true if these parameters are not explicit arguments of the config class's `__init__` method.

i.e. `convert_rope_params_to_dict` will fail for a config which looks like:

```py
class MyConfig(PreTrainedConfig):
    def __init__(self, ..., rope_theta, ..., **kwargs):
        ...
        super().__init__(**kwargs)
```

All custom configs to date will look like this so this is not a safe assumption to make.

This PR updates `convert_rope_params_to_dict` so that it checks for the rope parameters in the following locations in order of priority:

1. Values in `rope_parameters` dict (where they should be after standardization)
2. Values in `kwargs` (i.e. it's in config.json but not MyConfig.__init__'s args)
3. Values in the config's attributes (i.e. it's in MyConfig.__init__'s args)
4. Default values (i.e. not present at all but other RoPE parameters are present)

It also simplifies the fallback to `{}` for `rope_parameters`.

Comments:
Looks like the branch is a few commits behind main? Best to merge/rebase re CI 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43766). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
:rotating_light: Modify ModernBERT's default attention implementation to stop using FA
Author: YangKai0616
Pull request: yes

Background: PR https://github.com/huggingface/transformers/pull/43672

This PR modifies the default attention implementation in `modernbert`, switching away from `FlashAttention`. It also restores the related tests that were previously overridden.

@vasqu , please help review again, thank you very much!

Comments:
Also check modern_bert decoder, it inherited from modernbert (should be a simple removal of the function there in modular) + should we add a small note in the docs / update them for the padding free example

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: modernbert, modernbert_decoder, pe_audio, pe_video

> Also check modern_bert decoder, it inherited from modernbert (should be a simple removal of the function there in modular) + should we add a small note in the docs / update them for the padding free example

Done.

> Much better, can you add 🚨 to the title since it is breaking (in a good way)

It seems I don’t have permission to edit the title. The `Edit` button next to the title is gone.
Could you please help me change the title? Thank you!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43764). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
unpin torch for CircleCI
Author: ydshieh
Pull request: yes

# What does this PR do?

So CircleCI will run with torch 2.10

CI is ✅  see [here](https://app.circleci.com/pipelines/github/huggingface/transformers/163136/workflows/58ba2617-481b-4b4e-a456-d327806af088)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43790). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Modular Dependencies`] Fixup qwen rms norms
Author: vasqu
Pull request: yes

As per title, this led to weird dependencies where modeling files used direct imports 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43772). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: afmoe, aimv2, apertus, arcee, aria, bamba, bitnet, blt, chameleon, clvp, csm, cwm, deepseek_v2, deepseek_v3, dia, diffllama
Support to measure TTFT & TPOT
Author: Jereshea
Pull request: yes
Labels: name: Code agent slop

# What does this PR do?
This PR introduces a `token_latency` feature to the generation. When enabled, it allows users to capture the execution time of each generation step, facilitating the measurement of key performance metrics like **Time to First Token (TTFT) and Time Per Output Token (TPOT)**

# Motivation
Currently, measuring TTFT and inter-token latency requires custom wrappers or external profilers. By integrating this into `GenerationMixin`, users can benchmark model performance directly through the standard `.generate()` API.

# Changes 
- Added `token_latency` tracking logic to `_sample`, `_beam_search` and `_assisted_decoding`.
- Modified the return signature of `.generate()` to return a tuple `(output, latency_list)` when `model.config.token_latency = True`.
- `latency_list[0]` captures the prefill + first token time (TTFT).
- Subsequent elements in `latency_list` capture individual decoding steps.

Comments:
Clarify GPU memory expectations when using Adafactor with DeepSpeed
Author: hemanth678901-stack
Pull request: yes
Labels: name: Code agent slop

This PR clarifies a common misconception around GPU memory usage during training, particularly when using the Adafactor optimizer with DeepSpeed.

Several users expect Adafactor to always reduce peak GPU memory due to reduced optimizer states. In practice, overall memory depends on multiple factors such as activations, distributed configuration, and training stages.

This update improves user-facing documentation without modifying optimizer behavior or implementation.

Comments:
Add new Trainer utilities and documentation
Author: harshang03
Pull request: yes
Labels: name: Code agent slop

# What does this PR do?

This PR makes remove_unused_columns=True customizable for trainers that transform dataset columns before calling the model. It introduces an optional signature_columns argument on Trainer to control which dataset columns are preserved, avoiding fragile signature hacks in custom trainers (e.g., TRL DPO/GRPO). It also adds a regression test and documents the option in the Trainer guide.

<!-- Remove if not applicable -->


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

issue #43598 

Comments:
[`Repo Consistency`] Fix rms norm
Author: vasqu
Pull request: yes

As per title, I merged #43772 after #43400 which caused this mismatch - causes red CI 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: vibevoice_acoustic_tokenizer

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43803). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
preventing I/O errors on closed streams in the `cli` helper
Author: tarekziade
Pull request: yes

Let's see if we can fix that flaky

Comments:
(this branch inherits from `tarekziade-flaky-test_generate` ) 

only the last commit matters here

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: donut, esm, mimi, minimax_m2, mixtral, videomae

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43797). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

merged back in https://github.com/huggingface/transformers/pull/43794
Fix resume_from_checkpoint ignoring user's batch size configuration
Author: lordaarush
Pull request: yes

# Fix Trainer resume_from_checkpoint incorrectly calculating max_steps when changing batch size

## What does this PR do?

When resuming training from a checkpoint with a different `per_device_train_batch_size` but the same global batch size (by adjusting `gradient_accumulation_steps`), the Trainer was incorrectly recalculating `max_steps`.

**Root cause:** `self._train_batch_size` was unconditionally restored from the checkpoint state at line 2150-2151, causing the dataloader to be created with the OLD batch size. This led to incorrect `len_dataloader` and ultimately wrong `max_steps` calculation.

**Example:**
- Script 1: `per_device_train_batch_size=1`, `gradient_accumulation_steps=4` → `max_steps=8`
- Script 2 (resume): `per_device_train_batch_size=4`, `gradient_accumulation_steps=1` → `max_steps=32` ❌ (should be 8)

**The fix:** Only restore the checkpoint's `train_batch_size` when `auto_find_batch_size` is enabled, since that feature specifically needs the auto-discovered batch size to be preserved. Otherwise, use the current args batch size.

Fixes #43708

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case. → #43708
- [ ] Did you make sure to update the documentation with your changes? (N/A - no doc changes needed)
- [ ] Did you write any new necessary tests? (Bugfix with minimal code change)

## Who can review?

@SunMarc (already engaged on the issue)

Comments:
Chase TokenizersBackend issue
Author: Rocketknight1
Pull request: yes

One of the recurring issues we see in the CI is tokenizers occasionally being initialized as instances of `TokenizersBackend` rather than the actual tokenizer class. The error crops up all over, and is annoyingly hard to track down because I can't reproduce it locally. This PR is an attempt to flush it out and run some debug so I can see what's happening in the CI itself.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43735). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

We finally tracked it down and the fix is here: https://github.com/huggingface/transformers/pull/43791
Refactor trainer data_collator and callbacks tests
Author: SunMarc
Pull request: yes

# What does this PR do?

This PR updates and simplifies tests for trainer. We never really had any issues with those tests, just cleaning a bit. 

1) `tests/trainer/test_data_collator.py` 
- Restructured from 4 large classes (PyTorch/NumPy × Integration/Immutability) into 10
  focused classes by collator type
- Each test class now contains PyTorch, NumPy, and immutability tests together
- Added missing tests (attention_mask generation, flattening immutability)

2) `tests/trainer/test_trainer_callback.py`
  - Split monolithic tests into ~30 focused single-purpose tests
  - Added 4 new test classes: `TrainerStateTest`, `TrainerControlTest`, `CallbackHandlerTest`, `EarlyStoppingCallbackTest`
  - Tests now cover: callback registration, event firing, control flow, state,  persistence, and early stopping logic


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43776). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Is there a specific part I should be review? (there is a loooooot )

Not really haha. Just look at the structure of the tests I would say. But this is not really important as those tests never really failed until now.