noamaanMulla-03/transformers-issues huggingface dataset:
Fix hf_hub_download not placing file in current dir for url_to_local_path
Author: ydshieh
Pull request: yes

## Summary

`hf_hub_download` places the file in a a subdirectory, not directly in `.`, so `url_to_local_path` (which checks `os.path.exists(filename)` in the current dir) would never find it. This copies the downloaded file into the current directory after the download.

For example, for

> https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/australia.jpg

the downloaded path is

> /root/project/transformers/tasks/australia.jpg

and since `url_to_local_path` doesn't find it, in the tests, we fetch the image and lead to some connection error from time to time.

Comments:
cc @zucchini-nlp Fixed!

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46545)
[CI] fix and make less flaky
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

as per title, deletes duplicated docstring and fixes flaky tests

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46543). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hmm, CI is flaky today :cry: 

~Should we fix it properly here? I think it was just a mismatch between the new kwarg introduced? Wdyt?~ Nvm I'm blind lol

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: diffusion_gemma

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46543)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46543&sha=e5268c

Merging as it's flaky and the failing tests are unrelated
docs: Update NeMo Automodel getting-started docs for updated user API
Author: akoumpa
Pull request: yes

## What
Updates the two NeMo AutoModel getting-started pages to match the current NeMo AutoModel user API. Both code samples are currently broken against recent releases.

## Why
Fixes [NVIDIA-NeMo/Automodel#2507](https://github.com/NVIDIA-NeMo/Automodel/issues/2507).

**Pretraining page** — the Python sample:
- imported `nemo_automodel.recipes._dist_setup.setup_distributed`, which no longer exists; and
- passed `device_mesh`/`moe_mesh`/`distributed_config`/`moe_config` to `from_pretrained`, which now raises `TypeError` — distributed settings must be passed through a single `distributed_setup` object.

  It now builds a `DistributedSetup` with `DistributedSetup.build(...)` and passes it as `distributed_setup=`.

**Finetuning page** — the launch command used the deprecated `torchrun ... examples/llm_finetune/finetune.py -c <yaml>` path (that script now emits a `DeprecationWarning`). It now uses the `automodel <config.yaml> --nproc-per-node N` CLI, and the YAML snippet gains the top-level `recipe:` key that the CLI requires.

## Notes
Opened as a **draft**. The YAML `_target_` paths were verified against the live `examples/llm_finetune/nemotron/nemotron_nano_v3_hellaswag_peft.yaml` and are unchanged.


Comments:
[CB] [Minor] Add data-parallel to overall script
Author: remi-or
Pull request: yes

This PR adds a data-parallel mode to the `continuous_batching_overall.py` script that we use to benchmark the performance of CB across a range of use cases.
Until now, the script did not allow to leverage all the GPUs on a node because it only supported TP: if a user wanted to run 16 benchmarks on a 8 GPUs node with the model on a single GPU, the 16 benchmarks ran on the same GPU, in order. Now each GPU will get 16 / 2 = 8 benchmarks if `--dp-size 8` is passed (must be run with `torchrun`). This drastically decreases the time needed to run the full benchmark suite ofc.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46490). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46490)
Wav2Vec2CTCTokenizer in v5 treats the word delimiter as a special token, which leaks into get_special_tokens_mask and convert_ids_to_tokens
Author: SHAI-ishan-katoch
Pull request: no


### System Info

- transformers 5.10.0.dev0 (current main), also reproduces on 5.9.0
- last good: 4.57.6
- Python 3.12, macOS arm64 (CPU only)

### Who can help?

@eustlb

### Reproduction

Runs offline in a few seconds:

```python
import json, tempfile, os
from transformers import Wav2Vec2CTCTokenizer

d = tempfile.mkdtemp()
vocab = {c: i for i, c in enumerate(["<pad>", "
", "
", "<unk>", "|"] + list("abcdefghijklmnopqrstuvwxyz"))}
with open(os.path.join(d, "vocab.json"), "w") as f:
    json.dump(vocab, f)

tok = Wav2Vec2CTCTokenizer(os.path.join(d, "vocab.json"))
ids = tok("hello world")["input_ids"]
print(tok.get_special_tokens_mask(ids, already_has_special_tokens=True))
```

On 4.57.6 this prints all zeros. On v5 the `|` position (id 4) is marked as special:

```
v4.57.6:  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
v5:       [0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]
```

`tok.all_special_tokens` shows the difference directly: v4 gives `['
', '
', '<unk>', '<pad>']`, v5 gives `['
', '
', '<unk>', '<pad>', '|']`.

### What's going on

In v5 the word delimiter ends up registered as a special token. `_decode` knows about this and exempts it explicitly (tokenization_wav2vec2.py line 430: `... and token != self.word_delimiter_token`), so plain `tokenizer.decode()` still looks fine. But the other special-token-aware APIs got no such exemption:

- `get_special_tokens_mask(..., already_has_special_tokens=True)` marks every `|` as special (output above)
- `convert_ids_to_tokens(ids, skip_special_tokens=True)` drops `|` entirely (line 286 filters on `all_special_ids` with no delimiter exemption), so word boundaries vanish: v4 keeps the delimiter, v5 returns the tokens with all `|` removed

Anything that uses the mask or skip_special_tokens to strip specials (label masking in ASR fine-tuning, WER computation that filters special ids) now eats word boundaries silently. No warning, output just changes.

### Expected behavior

Same as v4: the word delimiter is not a special token for masking/skipping purposes, only for CTC string assembly.

Two ways to fix it, depending on what you prefer: either stop registering the delimiter as special and drop the `_decode` exemption, or keep it special and add the same exemption to `get_special_tokens_mask` and `convert_ids_to_tokens`. Happy to open a PR for either.


Comments:
Closing in favor of #46552 (reposted under my personal account). Same content.
[CI] Add 60s delay in security gate for flow observation
Author: ydshieh
Pull request: yes

Temporary: adds a `sleep 60` in the security check job so there is time to observe the full flow before the approval API call fires.

To be reverted before the real security checks are added.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46555). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[TBC] [CI] Auto-approve PR CI for fork PRs via security gate
Author: ydshieh
Pull request: yes

## Summary

- Add `pr-ci-security-gate.yml` triggered by `pull_request_target` to automatically approve the pending **PR CI** workflow run for fork PRs once a security check passes
- Security check is a stub (always passes) for now — real checks will be added after the flow is validated end-to-end
- `pr-ci-caller.yml` is unchanged; the `ydshieh2` allowlist remains in place for testing

## How it works

1. Fork PR opened → `pull_request_target` fires (no approval needed, runs in base repo context)
2. `security-check` job runs (currently a no-op pass)
3. `approve-pr-ci` job polls the Actions API for the pending **PR CI** run on that commit SHA and approves it
4. The unblocked `pull_request` workflow proceeds normally

## Test plan

- [ ] Open a PR from `ydshieh2` fork and verify the security gate triggers and approves the PR CI run automatically
- [ ] Confirm the security gate is skipped for member/collaborator PRs
- [ ] Once validated, replace the stub security check with real checks and open `pr-ci-caller.yml` to all fork PRs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46553). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
chore: update pr-ci-security-gate.yml
Author: hf-security-analysis[bot]
Pull request: yes

Update `.github/workflows/pr-ci-security-gate.yml` workflow configuration.

cc @ydshieh

Closes huggingface/tracking-issues#817
<!--slack ts:1781167229.379879 channel:C0AJSP0D53L-->

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46554)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46554). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[CI] Implement real security check in PR CI security gate
Author: ydshieh
Pull request: yes

## Summary

Replace the stub security check with a real two-layer gate:

- **Layer 1 — file path deny-list**: immediately blocks PRs that touch `.github/`, `Dockerfile`, `Makefile`, `pyproject.toml`, `setup.cfg`, `setup.py`, or `requirements*` — no tool needed, requires manual review
- **Layer 2 — Bandit**: runs static analysis on changed `.py` files only (not the full repo), at high severity + medium confidence to avoid false positives from normal ML code patterns

The PR code is checked out at its head SHA for read-only static analysis — safe in `pull_request_target` context since we never execute the fork code.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46557). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46557)
chore: update pr-ci-security-gate.yml
Author: hf-security-analysis[bot]
Pull request: yes

Update `.github/workflows/pr-ci-security-gate.yml` workflow configuration.

cc @ydshieh

Closes huggingface/tracking-issues#818
<!--slack ts:1781170569.380739 channel:C0AJSP0D53L-->

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46558)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46558). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Vision RoPE refactor
Author: IlyasMoutawwakil
Pull request: no
Labels: name: Feature request

### Feature request

Vision RoPE layer/module should return the final positional embedding, following https://github.com/huggingface/transformers/pull/41992#discussion_r3361589165 @zucchini-nlp @ArthurZucker 

### Motivation

We already create a class for it so might as well do the full work.

### Your contribution

Will open a PR

Comments:
Niiiiice, we have a few pure vision models (pixtral, siglip2) and all qwen-vl vision blocks from top of my head. Could be more vision RoPE modules 

@IlyasMoutawwakil  @zucchini-nlp  there are a few bot PRs linked to this, if this is not already handled, can i have a go at this?


Heyyy @zucchini-nlp @IlyasMoutawwakil @molbap @yonigozlan I have linked a PR with this. Any feedback would be helpful


Sorry for delay, and thanks for the PR

TBH I would prefer this to be resolved by someone internally, as we should refactor all vision-related RoPE to follow llama-format, which means adding default freq compute, moving to `config.rope_parameters` and adding RotaryMixin for validation and non-default positional embedding types

I had a quick look at the PR and it moves in the right direction! It will need a bit more to standardize tho and sync with LLM's rope modules. Not sure if @IlyasMoutawwakil has a local draft or wants to take it from here?

i will take care of it, i think this kind of library-wide refactor is very sensitive and needs a lot of testing locally and in our ci so better be done by someone internally. thanks for the pr 🤗

Hey, thanks for the update, i have been meaning to contribute to vision models or something related to them. Is there any other issue i can work on and contribute? Most of the vision related issues are very old, and i cant figure out if they are still relevant

@srishtiii28 thanks for initiative! I am not aware of any community contributions we have open atm for vision models. We are doing a lot of refactors lately, which imo is more suited for internal maintainers

If you have a painpoint when working with vision models, and can propose a solutions, pls feel free to open another issue and tag me/ other vision maintainers. Also a great option is to contrib a model if there is anything requested a lot by community, or anything trending :)
Honor a concrete dtype in AutoModel for composite checkpoints
Author: qflen
Pull request: yes

# What does this PR do?

Fixes #46459.

## Problem

`AutoModelForCausalLM.from_pretrained(repo, dtype=torch.float32)` silently ignores a concrete `dtype` for composite/multimodal checkpoints (e.g. Qwen3.5) and loads the model in the checkpoint's saved dtype instead.

When building the config, only `dtype="auto"` was popped from `kwargs`, so a concrete dtype was left in. `AutoConfig.from_pretrained` then absorbs it onto the composite config and drops it from the returned unused-kwargs. 

The factory swaps to the text sub-config (`config.get_text_config()`) for the text-only model class, and that sub-config still carries the saved dtype, so `model_class.from_pretrained` never sees the requested one and falls back to bf16.

## Fix

Pop a concrete `dtype`/`torch_dtype` before building the config and re-inject it into the unused kwargs afterwards, the same way `quantization_config` is already handled right below. The saved dtype is kept by default and only overridden when a `dtype` is explicitly passed.

New test `test_automodelforcausallm_dtype` in the Qwen3.5 and Qwen3.5-MoE test files covers it: a composite saved in bf16 and reloaded with `dtype=torch.float32` now loads in fp32 (as `...ForCausalLM`), while `dtype="auto"` / no dtype still load in bf16 and `torch_dtype=` still works; it also checks the whole VLM via `AutoModelForImageTextToText` keeps the requested dtype across sub-configs.

The related `from_config` / `config.dtype` quirk is out of scope here and tracked in #46512.

## Who can review?

@zucchini-nlp @Cyrilvallez


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46514). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46514)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, qwen3_5, qwen3_5_moe

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46514&sha=833e9b
[CI] Catch all shell/process execution issues in security gate via Bandit JSON report
Author: ydshieh
Pull request: yes

## Summary

Replace `--severity-level` filtering with targeted Bandit test ID matching:

- Run Bandit with `--format json -l` (all severities) to get the full report
- Filter the JSON report with `jq` for shell/process execution rules specifically, regardless of how Bandit classifies their severity:
  - **B601** paramiko SSH calls
  - **B602** subprocess.Popen with shell=True
  - **B603** subprocess without shell
  - **B604** any call with shell=True
  - **B605** os.system / os.popen
  - **B606** process without shell
  - **B607** partial path process start
  - **B609** wildcard injection

This fixes the previous gap where `os.system("ls")` and `subprocess.Popen("ls", shell=True)` were classified LOW severity by Bandit and silently passed the gate.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46560). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46560)
chore: update pr-ci-security-gate.yml
Author: hf-security-analysis[bot]
Pull request: yes

Update `.github/workflows/pr-ci-security-gate.yml` workflow configuration.

cc @ydshieh

Closes huggingface/tracking-issues#819
<!--slack ts:1781178486.657709 channel:C0AJSP0D53L-->

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46563)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46563). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Lower logging level CB
Author: qgallouedec
Pull request: yes

I feel like all these messages are useful for debugging purposes but for actual usage, it's probably not relevant.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40831). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Good comments, this is kind of redundant with #40828 so if that's fine with you I added a commit for the stuff here that was not already in #40828
Thanks a lot for the feedback!

Sounds good, thanks, closing this one!
Don't report `num_input_tokens_seen` when disabled
Author: qgallouedec
Pull request: yes

```python
>>> bool("no")
True
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_40899). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Will be fixed here: https://github.com/huggingface/transformers/pull/40799 ! 

Good thanks
Remove unnecessary `expand_as` in `get_placeholder_mask` across VLMs
Author: syncdoth
Pull request: no
Labels: name: Feature request

### Feature request

## Problem

The `get_placeholder_mask` function (and equivalent inline patterns) across ~70 multimodal model files expands a boolean placeholder mask from shape `(B, S, 1)` to `(B, S, H)` via `.expand_as(inputs_embeds)` before passing it to `masked_scatter`. This expansion is unnecessary because `masked_scatter` natively supports broadcasting.

For example, in `modeling_llava.py`:
```python
special_image_mask = special_image_mask.unsqueeze(-1).expand_as(inputs_embeds).to(inputs_embeds.device)
```

The validation check also uses data-dependent boolean indexing:
```python
inputs_embeds[special_image_mask].numel() == image_features.numel()
```


### Motivation

## Motivation

Mainly, Memory. While `expand_as` itself returns a stride-0 view (no copy), the subsequent `.to(device)` call materializes the full `(B, S, H)` boolean tensor when a device transfer is needed. Practically, I assume that the need for actual device transfer is quite rare; however, I believe this is still a safer implementation.


### Your contribution

## Proposed fix

1. Remove `.expand_as(inputs_embeds)`, keeping the mask as `(B, S, 1)`:
```python
special_image_mask = special_image_mask.unsqueeze(-1).to(inputs_embeds.device)
```

2. Replace the validation with equivalent arithmetic:
```python
n_image_tokens * inputs_embeds.shape[-1] == image_features.numel()
```

## Correctness

`masked_scatter`, `torch.where`, and element-wise `*` all support broadcasting a `(B, S, 1)` mask to `(B, S, H)`. Verified with:

```python
import torch
B, S, H = 2, 10, 16
inputs_embeds = torch.randn(B, S, H)
features = torch.randn(3, H)
mask_2d = torch.zeros(B, S, dtype=torch.bool)
mask_2d[0, 2], mask_2d[0, 5], mask_2d[1, 3] = True, True, True

mask_old = mask_2d.unsqueeze(-1).expand_as(inputs_embeds)
mask_new = mask_2d.unsqueeze(-1)

result_old = inputs_embeds.clone().masked_scatter(mask_old, features)
result_new = inputs_embeds.clone().masked_scatter(mask_new, features)
assert torch.equal(result_old, result_new)  # ✓ identical
```


Affects `get_placeholder_mask` and equivalent inline patterns in ~70 files across all VLM models (llava, qwen2_vl, paligemma, gemma3n, chameleon, video_llava, etc.), plus `tensor_parallel.py` and `ovis2`.

I have a PR ready: https://github.com/syncdoth/transformers/tree/remove-expand-as-placeholder-mask

Comments:
chore(sec): added a handful of security checks
Author: tarekziade
Pull request: yes

# What does this PR do?

Activated some bandit rules and fixed a few spots


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, dia, ernie4_5, ernie4_5_vl_moe, hubert, sew, sew_d, videomt

run-slow: auto, dia, ernie4_5, ernie4_5_vl_moe, hubert, sew, sew_d, videomt

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/24464020475)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/auto", "models/dia", "models/ernie4_5", "models/ernie4_5_vl_moe", "models/hubert", "models/sew", "models/sew_d", "models/videomt"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45462). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/24464020475)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [04ab8ddd](https://github.com/huggingface/transformers/commit/04ab8ddd4da435add7d8112933c7d1c18ccdd2c7) | workflow commit (merge commit) |
| PR | [d2815c19](https://github.com/huggingface/transformers/commit/d2815c1971c35231d63a2de1a265145779a64a41) | branch commit (from PR) |
| main | [18aa0866](https://github.com/huggingface/transformers/commit/18aa0866577ce3270846d5d44a61534f636f9b42) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !


In general, we don't need to touch the conversion scripts. But it's fine to keep the changes as you already did.

let's review this once the migration is done 👍 🙏
GPU go brr [DiffusionGemma]
Author: gante
Pull request: yes
Labels: name: New model

# What does this PR do?

```
------
--T---
S-T---
S-T--N
SAT--N
SAT-RN
SATURN
```

🪐

partial list of contributors: @gante @gscrivs @jg-lo @agesf @pashatyl @tamaravonglehn @iv2nl0b9v @jtarbouriech @prutianova-nastia @danielecc @liutianlin0121 @npvoid

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, diffusion_gemma

brrrrr

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46540)