noamaanMulla-03/transformers-issues huggingface dataset:
[DiffusionGemma] better docs and links
Author: gante
Pull request: yes

# What does this PR do?

[waiting for the doc builder preview]

- Fixes broken internal references
- Fixes multi-line diagrams
- Adds missing checkpoint links
- More examples


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: diffusion_gemma

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46569)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46569). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
a11 for checkers
Author: tarekziade
Pull request: yes

# What does this PR do?

Adds OTEL traces for checkers

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46599)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46599). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
WIP add minimax_m2_7
Author: itazap
Pull request: yes

checkpoint: https://huggingface.co/MiniMaxAI/MiniMax-M2.7/tree/main

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46411). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46411)

> ⚠️ **Code quality check failed** — all test jobs were skipped. Fix the code quality issues and push again to run tests.
Add inference time layer fusion optimisations via `PreTrainedModel.from_pretrained(fuse_layers=True)`
Author: hmellor
Pull request: yes


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_44942). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llama

Closing as, while it does work, the current implementation is too invasive to be practically used across all architectures
Fix `_set_model_specific_special_tokens` to accept list-format `extra_special_tokens`
Author: bensons
Pull request: yes

# What does this PR do?

Some model repos provide `extra_special_tokens` as a list in their tokenizer_config.json, which caused an `AttributeError: 'list' object has no attribute 'keys'`. This converts list inputs to a dict mapping each token to itself before processing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

@Cyrilvallez 
@itazap 
@ArthurZucker



Comments:
Please provide the excat model where this happens before we review. Also, we do not want new test files

Example that caused me to find this behavior: 
https://huggingface.co/jedisct1/Qwen3-VL-Embedding-8B-mlx/blob/main/tokenizer_config.json

I'm able to load `AutoTokenizer.from_pretrained("jedisct1/Qwen3-VL-Embedding-8B-mlx")` without any errors. Can you please pull on main and share the full reproducer and error you have?
[CI] capture checkers output in OTEL
Author: tarekziade
Pull request: yes

# What does this PR do?

make sure we dont lose checkers output in traces

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46601). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46601)
[CI] don't export OTELs within the tests
Author: tarekziade
Pull request: yes

# What does this PR do?

Filter out otel from tests in prod otel in CI :) 

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46602)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46602). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
BAN
Author: Priyabhunia
Pull request: no

sorry for creating this issue but this is little unrelated

Hey any maintainer here or mod here from discord , or if you know someone to connect .My account is banned from the discord server because my account was hacked and spammed ... is there anyone who can help me to unban .

I am not able to find anyone

Comments:
from hugging face discord server 

here is my username at discord @laxis_op



Hi @Priyabhunia we have forwarded this internally to the appropriate team, so I'll close this issue now. Good luck!

<img width="632" height="452" alt="Image" src="https://github.com/user-attachments/assets/d49b2e57-58b3-4059-a4e5-a1aac2adc564" />

hey @pcuenca why I am still facing this issue 

hey @pcuenca  still not solved yet
Update bug report list
Author: molbap
Pull request: yes

# What does this PR do?

Updating the bug report after recent shuffle :grin: 


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46607)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46607). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update post release
Author: vasqu
Pull request: yes

Way too many releases we were several dev versions back + syncing with proper kernel path on minimax (which is in the release)

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: minimax_m3_vl

cc @ArthurZucker @Cyrilvallez merging this one since it's minor and should be done before we forget about it :joy: 

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46608)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46608). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46608&sha=f86edd
Fix `peft` lower bound
Author: hmellor
Pull request: yes

As of https://github.com/huggingface/transformers/pull/45155, Transformers started importing `from peft.utils.save_and_load import _maybe_shard_state_dict_for_tp` in the peft integration. This import did not exist until https://github.com/huggingface/peft/pull/3096, which was first released in `peft==0.19.0`.

This PR corrects the `peft` lower bound.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46605)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46605). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46605&sha=a04868

Can you merge the PR @vasqu ? 

Lemme retry running the CI job but will merge afterwards
Disable PR CI workflow for PRs from forked repo. during the weekend
Author: ydshieh
Pull request: yes

# What does this PR do?

Disable PR CI workflow for PRs from forked repo. during the weekend

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46609). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix hub kernels compatibility with kernels>=0.15.0
Author: jshn9515
Pull request: yes

# What does this PR do?

Fixes #46291

This PR fixes the [hub_kernels.py](https://github.com/huggingface/transformers/blob/main/src/transformers/integrations/hub_kernels.py) version error introduced by `kernels>=0.15.0`.

Starting from kernels v0.15.0, Hub-backed kernel repositories must specify either a version or a revision. This PR adds explicit versions to the affected kernel repository definitions. Since this only adds version metadata, it should remain compatible with earlier kernels versions.

This PR also updates two kernel repository names. According to the documentation, the previous names are marked for removal.

cc @drbh 


Comments:
[CB] Fix seqlens and use TypedDict
Author: remi-or
Pull request: yes

# Summary

This PR fixes a race condition where the `cumulative_seqlens` tensors would be padded on the device during their transfer. It seems the bug did not happen often in a non-distributed setting, but it appeared when running the continuous batching overall script in DP 8. It is now fixed: the padding happens before the tensor is constructed and moved to the device.

This PR also removes the `
@dataclass` decorator that was used to define `PagedAttentionArgs` in favor of a `TypedDict`, which is cleaner.

## Performance ✅ 

No performance changes 

## Tests ✅ 

All tests pass.

## AI Review ✅ 

Reviewed by claude

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46593). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46593)
Fix dynamic module symlinked cache on trust_remote_code models
Author: ldkhang1201
Pull request: yes

## What this fixes
Loading a `trust_remote_code` model from a **local, symlinked cache** crashes with:

```
FileNotFoundError: [Errno 2] No such file or directory:
'.../blobs/transformers_4_44_2__modeling_rope_utils.py'
```

A standard `huggingface_hub`  cache stores each file twice:

```
models--org--repo/
  blobs/<sha>                  # real content, content-addressed, hash-named, no .py extension
  snapshots/<rev>/modeling.py  # symlink -> ../../blobs/<sha>
```
read: https://huggingface.co/docs/huggingface_hub/en/guides/manage-cache

`_compute_local_source_files_hash` did `Path(resolved_module_file).resolve()` before calling
`get_relative_import_files`. That follows the snapshot symlink into `blobs/`, and
`get_relative_import_files` then derives sibling import paths from `Path(module_file).parent`
(now `blobs/`). The named `.py` siblings only exist in `snapshots/<rev>/`, not in `blobs/`, so the
lookup raises `FileNotFoundError`.

## Fix

- Run relative-import discovery on the **unresolved** snapshot path, so traversal stays inside
  `snapshots/<rev>/` where the named `*.py` symlinks live.
- When building the hash key, canonicalize only the **parent directory** (to normalize symlinked
  path components such as macOS `/var`→`/private/var`) and keep the file's logical name.
  `read_bytes()` still follows the symlink, so content is hashed correctly. For plain
  non-symlinked local dirs the produced hash is unchanged (no cache churn).

## Tests

Added a regression test in `tests/utils/test_dynamic_module_utils.py` that build the real
`blobs/` + `snapshots/` symlink layout:

- `test_get_cached_module_file_local_handles_symlinked_hub_cache` — loads a module with a transitive
  relative-import chain (A→B→C) from a symlinked cache; fails on `main` with the exact
  `FileNotFoundError`, passes with the fix, and asserts the transitive deps are copied with correct
  content.

```
$ python -m pytest tests/utils/test_dynamic_module_utils.py -q
16 passed
```

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46613)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46613&sha=4048db
cannot import name 'is_offline_mode' from 'huggingface_hub'
Author: dollarser
Pull request: no
Labels: name: bug

### System Info

- transformers-5.0.0
- huggingface_hub-1.2.1
```
ImportError: cannot import name 'is_offline_mode' from 'huggingface_hub' (/root/miniconda3/envs/transformers/lib/python3.10/site-packages/huggingface_hub/__init__.py)
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModel, AutoProcessor, AutoTokenizer

### Expected behavior

how to fix ?

Comments:
the same question

bruh he closed without explaining how to fix it?


Same question here with 
```
huggingface_hub                          0.36.2
transformers                             5.3.0
```
Revert "Disable PR CI workflow for PRs from forked repo. during the weekend"
Author: ydshieh
Pull request: yes

Reverts huggingface/transformers#46609

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46652). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Mirko Privitera
Author: mirko772
Pull request: no
Labels: name: New model

### Model description

30 settembre 1990 Palermo ma abito a sustinenza vicino casaleone provincia di Verona 3247929218

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Ciaoo
Fix typo in test_utils.py: 'that that' → 'that'
Author: Rohan5commit
Pull request: yes
Labels: name: Code agent slop

Fixes a duplicate word typo in `tests/generation/test_utils.py`.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46636)
Fix typo in modeling_mask2former.py: 'from from' → 'from'
Author: Rohan5commit
Pull request: yes
Labels: name: Code agent slop

Fixes a duplicate word typo in `src/transformers/models/mask2former/modeling_mask2former.py`.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46637)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mask2former
Fix typo in processing_utils.py: 'overriden' → 'overridden'
Author: Rohan5commit
Pull request: yes
Labels: name: Code agent slop

Fixes a misspelling in `src/transformers/processing_utils.py`.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46646)
Fix typo in modeling_mobilebert.py: 'embedings' → 'embeddings'
Author: Rohan5commit
Pull request: yes
Labels: name: Code agent slop

Fixes a misspelling in `src/transformers/models/mobilebert/modeling_mobilebert.py`.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46648)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mobilebert

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46648&sha=6c6434
Fix typo in input_outputs.py: 'ouputs' → 'outputs'
Author: Rohan5commit
Pull request: yes
Labels: name: Code agent slop

Fixes a misspelling in `src/transformers/generation/continuous_batching/input_outputs.py`.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46649)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: mobilebert

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46649&sha=3b1010
Fix typo in modeling_rope_utils.py: 'funcitonality' → 'functionality'
Author: Rohan5commit
Pull request: yes

Fixes a misspelling in `src/transformers/modeling_rope_utils.py`.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46647)
Fix typo in configuration_utils.py: 'we we' → 'we'
Author: Rohan5commit
Pull request: yes
Labels: name: Code agent slop

Fixes a duplicate word typo in `src/transformers/generation/configuration_utils.py`.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46635)
Fix beam search cache reorder for non-standard caches
Author: cjc0013
Pull request: yes
Labels: name: Code agent slop

Fixes #46612

## What changed

Beam search now reorders the active generation cache by scanning the supported cache names in `ALL_CACHE_NAMES` instead of only checking `past_key_values`.

The reorder path still preserves the legacy `_reorder_cache` hook for models that define it, and otherwise calls `reorder_cache` on the active cache object.

## Why

Models that store generation cache under names such as `cache_params` or `state` could otherwise keep stale beam ordering between decoding steps.

## Validation

- `PYTHONPATH=src python -m pytest tests/generation/test_utils.py::UtilsFunctionsTest -q`
- `PYTHONPATH=src python -m pytest tests/models/rwkv/test_modeling_rwkv.py::RwkvModelTest::test_beam_search_generate tests/models/rwkv/test_modeling_rwkv.py::RwkvModelTest::test_beam_sample_generate tests/models/rwkv/test_modeling_rwkv.py::RwkvModelTest::test_generate_from_inputs_embeds_1_beam_search tests/models/xlstm/test_modeling_xlstm.py::xLSTMModelTest::test_beam_search_generate tests/models/xlstm/test_modeling_xlstm.py::xLSTMModelTest::test_beam_sample_generate -q`
- `PYTHONPATH=src python -m compileall -q src/transformers/generation/utils.py tests/generation/test_utils.py`
- `python -m ruff check --select I src/transformers/generation/utils.py tests/generation/test_utils.py`
- `git diff --check`


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46615)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46615&sha=13b98d
Fix stale phonemizer_lang in Wav2Vec2PhonemeCTCTokenizer.phonemize
Author: kpal002
Pull request: yes
Labels: name: Code agent slop

Closes #46614.

## Root cause

`phonemize()` re-creates the espeak backend when the requested language differs from `self.phonemizer_lang`, but never updates `self.phonemizer_lang`:

```python
# before
if phonemizer_lang is not None and phonemizer_lang != self.phonemizer_lang:
    self.init_backend(phonemizer_lang)   # backend now uses the new lang
# self.phonemizer_lang still holds the old value
```

So the next call with the *original* lang compares equal (`"en-us" != "en-us"` → False), skips `init_backend`, and runs the cached wrong-language backend silently.

`prepare_for_tokenization` (L226-228) already does this correctly — `phonemize` just missed the assignment.

## Fix

Add `self.phonemizer_lang = phonemizer_lang` inside the re-init branch, mirroring `prepare_for_tokenization`:

```python
if phonemizer_lang is not None and phonemizer_lang != self.phonemizer_lang:
    self.phonemizer_lang = phonemizer_lang
    self.init_backend(phonemizer_lang)
```

## Test

Added `test_phonemize_lang_switch_roundtrip`: interleaves English → French → English through one tokenizer instance and asserts the first call's output equals the third call's output.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46616)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: wav2vec2_phoneme
SmolVLM: Inconsistent meaning of num_frames in video processor
Author: andreasgoulas
Pull request: no
Labels: name: bug

### System Info

transformers: 5.12.0

### Who can help?

@zucchini-nlp 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Compare the implementation of `SmolVLMVideoProcessor`'s sample_frames method with the default implementation in `video_processing_utils.py`.

### Expected behavior

The expected behaviour of `num_frames` would be to sample exactly N frames with uniform sampling (see the base class implementation for reference). However, the current SmolVLM implementation treats this value as the maximum number of frames to sample, which can cause confusion. Note that changing this behaviour would be a breaking change.

Comments:
Hey @andreasgoulas , some models were trained in certain way and the sampling strategy is chosen as the best for given model. In case of SmolVLM, if model performs better when sampling `num_frames` at max, that is how the sampling should be implemented. We can't force all models to follow `uniform_sampling` rule 

I see that the docstring also explains it 🤗  

Alright, that makes sense, thanks! @zucchini-nlp
fix(generation): reorder all cache types during beam search
Author: KirtiRamchandani
Pull request: yes
Labels: name: Code agent slop

Fixes #46612

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46639)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46639&sha=f0b20a