noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-<languageCode>] Translating docs to <languageName>
Author: Pediboi666
Pull request: no
Labels: name: WIP

<!--Deutsch 
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [x] [index.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.md) https://github.com/huggingface/transformers/pull/20180
- [x] [quicktour.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.md) (waiting for initial PR to go through)
- [ ] [installation.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.md).

## Tutorial section
- [ ] [pipeline_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.md)
- [ ]  [autoclass_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/autoclass_tutorial.md)
- [ ]  [preprocessing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.md)
- [ ]  [training.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.md)
- [ ]  [accelerate.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.md)
- [ ]  [model_sharing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.md)
- [ ]  [multilingual.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.md)

<!--
Keep on adding more as you go 🔥
-->


Comments:
SwanLab integration uses outdated swanlab.init() signature
Author: TianjiaoTsao
Pull request: no
Labels: name: Good First Issue

name: Feature request

### Feature request

https://github.com/huggingface/transformers/blob/aefa23ad1c52de9c115f3d762fe1a1eda643275a/src/transformers/integrations/integration_utils.py#L2305

In `src/transformers/integrations/integration_utils.py`, the SwanLab integration calls `swanlab.init(**init_args)` but the current implementation does not expose important init options like `id` and `resume`.

As a result, when resuming training with `Trainer.train(resume_from_checkpoint=...)`, SwanLab cannot resume the same experiment/run and instead creates a new one, because there is no way to pass `id`/`resume` through the callback.

SwanLab docs for `init` (showing `id` / `resume` support):
https://docs.swanlab.cn/en/api/py-init.html

Request:
Please update the SwanLab integration to support passing `id` and `resume`, then forwarded into `swanlab.init(...)`).

Thanks.

### Motivation

When using `Trainer` with the SwanLab integration, resuming training (`trainer.train(resume_from_checkpoint=...)`) starts a new SwanLab experiment instead of continuing the existing one. This breaks experiment continuity: metrics/history are split across multiple runs, comparisons become confusing, and automation workflows that expect a single run per training job (e.g., restart on preemption/SLURM requeue) are harder to manage.

The root cause is that the current integration calls `swanlab.init()` without exposing `id` and `resume`, even though SwanLab supports them. Adding support for these parameters would allow seamless run continuation across restarts.

### Your contribution

None

Comments:
cc @sunmarc @MekkCyber 

Thanks for the report @TianjiaoTsao ! Would you like to submit a PR to fix it ? Otherwise, we can let the community fix it !
fix(qwen3_vl): Use temporal_patch_size for timestamp calculation
Author: Mr-Neutr0n
Pull request: yes

## What does this PR do?

Fixes incorrect timestamp calculation in Qwen3VL Processor by using `temporal_patch_size` instead of `merge_size`.

## Problem

The `_calculate_timestamps()` method was being called with `self.video_processor.merge_size` when it should use `self.video_processor.temporal_patch_size`. 

The `temporal_patch_size` represents the temporal stride of video patches (how many frames are grouped into a single temporal patch), which is the correct value for calculating frame timestamps. The `merge_size` is used for spatial merging and is unrelated to temporal calculations.

## Changes

Updated both `processing_qwen3_vl.py` and `modular_qwen3_vl.py` to use `temporal_patch_size` instead of `merge_size`.

Fixes #43519

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_vl

Duplicate of https://github.com/huggingface/transformers/pull/43659
SwanLab: Add support for id and resume arguments in SwanLabCallback
Author: surya10602
Pull request: yes

# Summary

This PR updates `SwanLabCallback` to accept `**kwargs` in its `__init__` method and passes them to `swanlab.init()` during setup.

Previously, the SwanLab integration did not expose important initialization arguments like `experiment_id`, `id`, or `resume`. This made it impossible to properly resume training runs (e.g., when using `Trainer.train(resume_from_checkpoint=...)`) because SwanLab would create a new experiment instead of continuing the existing one. 

By allowing `**kwargs` to be passed through, users can now maintain experiment continuity across restarts by explicitly passing the run ID.

**Changes:**
- Updated `SwanLabCallback.__init__` to store `**kwargs`.
- Updated `SwanLabCallback.setup` to merge these user-provided arguments into `init_args` before calling `swanlab.init`.

Fixes #43698 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed.


Comments:
fix: add id and resume parameters to SwanLab integration
Author: thecaptain789
Pull request: yes

Fixes #43698

Adds support for run resumption in the SwanLab integration via two new environment variables:

- `SWANLAB_RUN_ID`: Unique identifier for the run
- `SWANLAB_RESUME`: Controls resumption behavior (`must`, `allow`, `never`, `auto`)

This allows users to resume training runs when using `Trainer.train(resume_from_checkpoint=...)` without SwanLab creating a new experiment.

Example usage:
```bash
export SWANLAB_RUN_ID="my-run-id"
export SWANLAB_RESUME="allow"
python train.py --resume_from_checkpoint=./checkpoint
```

Comments:
Fix incorrect timestamp calculation in Qwen3VL Processor
Author: jonathan-fulton
Pull request: yes

## What does this PR do?

Fixes #43519

### Problem

The Qwen3VL processor was incorrectly using `merge_size` (spatial merging) instead of `temporal_patch_size` (temporal grouping) when calculating video frame timestamps.

```python
# Before (incorrect)
curr_timestamp = self._calculate_timestamps(
    metadata.frames_indices,
    metadata.fps,
    self.video_processor.merge_size,  # Wrong! This is for spatial merging
)
```

### Root Cause

The video processor has two different parameters:
- `temporal_patch_size`: Groups consecutive frames together temporally (default 2)
- `merge_size`: Groups pixels spatially (default 2)

When calculating timestamps, we're grouping frames temporally to compute their average timestamp. Using `merge_size` instead of `temporal_patch_size` produces incorrect timestamp values because:
1. The grouping factor could be different between spatial and temporal dimensions
2. Conceptually, we're averaging timestamps across time, not space

### Solution

Use `temporal_patch_size` for timestamp calculation:

```python
# After (correct)
curr_timestamp = self._calculate_timestamps(
    metadata.frames_indices,
    metadata.fps,
    self.video_processor.temporal_patch_size,  # Correct!
)
```

The fix is applied to both `processing_qwen3_vl.py` and `modular_qwen3_vl.py`.

## Who can review?

VLM maintainers

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_vl

cc @molbap @zucchini-nlp

run-slow: qwen3_vl

This comment contains `run-slow`, running the specified jobs: 

models: ["models/qwen3_vl"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43659). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/21818507657)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [93d0b7e3](https://github.com/huggingface/transformers/commit/93d0b7e3e811bb73d00c71e2ba629dbef25c834e) | merge commit |
| PR | [a6e8b2eb](https://github.com/jonathan-fulton/transformers/commit/a6e8b2ebabe7e0f83f06f8ebdc65a61806076525) | branch commit |
| main | [9e5c7d16](https://github.com/huggingface/transformers/commit/9e5c7d16aa13cf4c160076d23ec6d4d2ef5a8fd0) | base commit |

✅ No failing test specific to this PR 🎉 👏 !
Incorrect timestamp calculation in Qwen3VL Processor
Author: 666zz666
Pull request: no
Labels: name: bug

### System Info

Should temporal_patch_size be used instead of merge_size for the timestamp calculation in this context?
https://github.com/huggingface/transformers/blob/dfe30827b8ebdd974eb7ce69c7d5d8cf8e6cf852/src/transformers/models/qwen3_vl/processing_qwen3_vl.py#L155

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

None

### Expected behavior

curr_timestamp = self._calculate_timestamps(
                        metadata.frames_indices,
                        metadata.fps,
                        self.video_processor.temporal_patch_size,
                    )

Comments:
Since both, merge and temporal sizes are `2`, we don't see any difference in inference. I am not sure what is the intention behind though so let's tag Qwen team who contributed the model (@JJJYmmm). 

https://github.com/huggingface/transformers/blob/dfe30827b8ebdd974eb7ce69c7d5d8cf8e6cf852/src/transformers/models/qwen3_vl/processing_qwen3_vl.py#L261-L265

Thanks for the ping! 🤗 Currently, the video processor sets `merge_size` and `temporal_patch_size` to the same value. 

Specifically, `temporal_patch_size` determines the temporal receptive field of the vision 3dconv, while `merge_size` controls the temporal compression ratio (initially, it's called `temporal_merge_size`). We may assign them different values in the future.

https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct/blob/main/video_preprocessor_config.json#L7-L8

Thanks for your reply!
It would be beneficial to introduce a separate temporal_merge_size parameter. When using this Processor, I encounter different compression ratios for the spatial and temporal dimensions, which leads to a length mismatch between the video frames and the timestamps.
In the current codebase, merge_size effectively acts as spatial_merge_size. Since temporal_patch_size matches the temporal_stride of the Vision Encoder's first Conv layer, it essentially serves the purpose of temporal_merge_size.

> I encounter different compression ratios for the spatial and temporal dimensions, which leads to a length mismatch between the video frames and the timestamps.

Sry, could you clarify why spatial compression matters here? The frame count and timestamps depend only on `temporal_patch_size` or `video_processor.merge_size`, which are currently equal. Spatial compression affects frame resolution only.
Remove GPU tracking from TrackioCallback and remove env var support
Author: qgallouedec
Pull request: yes

Trackio now natively supports GPU tracking, see https://github.com/gradio-app/trackio/releases/tag/trackio%400.14.0, so I suggest that we remove it from this callback.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_43371). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43371&sha=0efebc

I'm not sure why the CI is red, but I'm quite sure it's not related to the changes in this PR, so I'll go ahead and merge. If I'm wrong don't be mad and forgive me <3 

Yes the CI is broken these days! But in any case if you are not a core maintainer you can't merge with red CI 😅 , auto-merge only works in case the CI is green
Fix InternVL models
Author: hmellor
Pull request: yes
Labels: name: for patch

Since https://github.com/huggingface/transformers/pull/45078 (first released in v5.6.0), the fallback from `TokenizersBackend` to ` tokenizer_config.tokenizer_class` has been removed.

`internvl_chat` had been added to `MODELS_WITH_INCORRECT_HUB_TOKENIZER_CLASS` so that it would explicitly use `TokenizersBackend` instead of the class specified in its `tokenizer_config`. However, it appears that `internvl_chat` was actually using this fallback and therefore was actually loading the class specified by `tokenizer_config`.

This PR removes `internvl_chat` from `MODELS_WITH_INCORRECT_HUB_TOKENIZER_CLASS` so that the model can be loaded again with the same behaviour as in <v5.6.0.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46524). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46524)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46524&sha=d8569a
Whisper generation fails on empty transcription after align_special_tokens
Author: ronansgd
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 5.6.0.dev0
- Platform: Linux-6.17.0-1009-gcp-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 1.11.0
- Safetensors version: 0.7.0
- Accelerate version: 1.13.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.11.0+cu130 (CUDA)
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: no
- GPU type: NVIDIA A100-SXM4-40GB

### Who can help?

@eustlb 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

Basic reproduction script
```
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from transformers.trainer_utils import align_special_tokens


def main() -> None:
    model_name = "openai/whisper-tiny"
    processor = WhisperProcessor.from_pretrained(model_name)
    model = WhisperForConditionalGeneration.from_pretrained(model_name)

    # clear suppress tokens so the model can freely produce EOS when there is nothing to transcribe.
    model.generation_config.begin_suppress_tokens = None
    model.generation_config.suppress_tokens = [
        i for i in range(model.config.vocab_size) if i != model.config.eos_token_id
    ]

    # --- Without align_special_tokens: works fine ---
    features = processor(torch.zeros(16_000 * 10).numpy(), sampling_rate=16_000, return_tensors="pt").input_features

    out = model.generate(features)
    decoded = processor.batch_decode(out, skip_special_tokens=True)
    print(f"Before align_special_tokens: generate() on silence → {decoded!r}  (empty transcription, as expected)")

    # --- After align_special_tokens (called e.g. by Trainer.train()): crashes ---
    align_special_tokens(model, processor)

    print("\nAfter align_special_tokens: generate() on silence →", end=" ")
    try:
        model.generate(features)
        print("no crash")
    except IndexError as e:
        print(f"IndexError: {e}")


if __name__ == "__main__":
    main()
```

### Expected behavior

The generation should not fail (and return an empty transcription)

Comments:
```
Could not load model openai/whisper-large-v3-turbo with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCTC'>, <class `'transformers.models.auto.modeling_auto.AutoModelForSpeechSe
```

have you faced error of such kind recently? 
@ronansgd 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix missing Gemma4Processor._compute_audio_num_tokens
Author: csantosbh
Pull request: yes

# What does this PR do?
`Gemma4Processor._get_num_multimodal_tokens` calls `self._compute_audio_num_tokens` to size audio placeholder spans, but the method was never defined on the class. Any audio input therefore raises `AttributeError: 'Gemma4Processor' object has no attribute '_compute_audio_num_tokens'`.

Add the method, replicating the audio encoder's sequence-length arithmetic so the placeholder count exactly matches the audio tower output: `Gemma4AudioFeatureExtractor` mel framing (semicausal pad + unfold) followed by the two stride-2 Conv2d subsampling layers in `Gemma4AudioSubSampleConvProjection`, capped at `audio_seq_length`.

Validated against the real model: feeding the feature-extractor mel frames through `Gemma4AudioSubSampleConvProjection` and counting surviving positions matches this function across all sample lengths in [100, 160000].


## Code Agent Policy
- [X] I confirm that this is not a pure code agent PR.

## Before submitting
- [X] make fix-repo -> All 15 checks passed
- [X] utils/tests_fetcher.py -> No relevant tests found

## Who can review?
@RyanMullins and @zucchini-nlp (tagged as top contributors to the Gemma 4 implementation)

Comments:
Oh, can you add a test in processing gemma4 for this pls? For image tokens and audio tokens

@zucchini-nlp Done! I didn't include tests for the vision encoder since there are two tests already covering it - please let me know if you had something else in mind.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma4, gemma4_unified

@zucchini-nlp Having the issues addressed, what are the next steps for this PR? I see CI has 3 workflows awaiting approval, but I'm unsure who/how these are approved. Is there any further action required from my side?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46416). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix the offsets in processing
Author: zucchini-nlp
Pull request: yes
Labels: name: for patch

# What does this PR do?

Helps vllm to bump v5.10

Before: spans were computed only from the original regex positions, so after the first replacement every image after it became misaligned and `new_span` was wrong.

Now: we track a running `offset` that accumulates length changes from each replacement, and use it to adjust spans so every match is mapped correctly into the expanded text


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46525). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: florence2, fuyu, idefics3

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46525)
[docs] 4D attention masks
Author: stevhliu
Pull request: yes

fixes #46458 by adding docs for how to create and pass a custom 4D attention mask

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46507). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46507)
[docs] torchvision lanczos
Author: stevhliu
Pull request: yes

adds docs for #46496

updates a stale note about models that always default to PIL because torchvision doesn't support lanczos and update backend auto-selection behavior docs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46528). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46528)
Only download hub cache for pipelines_torch job
Author: ydshieh
Pull request: yes

## Summary

- The "download and unzip hub cache" step in `.circleci/create_circleci_config.py` was running for all CircleCI jobs
- Restricted it to only run when `job_name == "pipelines_torch"`, avoiding unnecessary downloads in other jobs

We get more download failures recently. This PR might avoid that until we migrate to GH.



Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46544)

> One q: iirc we have few other tests that download artifacts from `hf-testing`, like processor with tiny videos/images. Are those downloaded in a different process?

That are in one step above that I didn't remove in this PR

```python
            {
                "run": {
                    "name": "fetch hub objects before pytest",
                    "command": "cp -r /test_data/* . 2>/dev/null || true; python3 utils/fetch_hub_objects_for_ci.py",
                }
            },
```

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46544&sha=d173dc