noamaanMulla-03/transformers-issues huggingface dataset:
fix(masking): cast tensor q_offset/kv_offset to int in flex_attention_mask
Author: Anai-Guo
Pull request: yes
Labels: name: Code agent slop

## What does this PR do?

Fixes #46682.

When flex attention is used together with a cache, `flex_attention_mask` can crash with a `ValueError` about the mask having more than 4 dimensions:

```python
tok = ...  # Qwen3-4B, attn_implementation="flex_attention", use_cache=True
model.generate(...)  # ValueError inside create_block_mask
```

### Root cause

`_preprocess_mask_arguments` obtains the offset from the cache:

```python
q_offset = past_key_values.get_seq_length()
# To avoid graph breaks, StaticLayer returns a tensor instead of int -> this has no impact on the ops, but we
# need the correct device
q_offset = q_offset.to(inputs_embeds.device) if isinstance(q_offset, torch.Tensor) else q_offset
```

So `q_offset` (and similarly `kv_offset` from `get_mask_sizes`) can be a 0-d tensor. This is intentional and harmless on the eager/sdpa path, where the offset is only ever added to a 1-D `arange`:

```python
q_arange = torch.arange(q_length, device=device) + q_offset
```

But `flex_attention_mask` routes the offsets through `add_offsets_to_mask_function`, whose `mask_mod` is vmapped over **scalar** `q_idx`/`kv_idx`:

```python
def inner_mask(batch_idx, head_idx, q_idx, kv_idx):
    return mask_function(batch_idx, head_idx, q_idx + q_offset, kv_idx + kv_offset)
```

Adding a tensor to the scalar indices injects an extra dimension, so the resulting block mask ends up with more than 4 dimensions and `create_block_mask` raises. The original reporter confirmed that converting the offset to a Python `int` via `.item()` before building the mask resolves it (verified under `torch.compile` / inductor).

### Fix

Cast `q_offset`/`kv_offset` to Python ints **inside `flex_attention_mask` only**, right before they are baked into the `mask_mod`. The eager/sdpa path keeps the 0-d tensor (and its graph-break-avoidance benefit) untouched; flex genuinely needs scalar offsets for its vmapped `mask_mod`.

The conversion is guarded by `isinstance(..., torch.Tensor)`, so the common `int` (default `0`) case is unaffected.

## Before submitting

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request)?
- [x] Was this discussed/approved via a GitHub issue? Fixes #46682.
- [ ] Did you make sure to update the documentation with your changes? (no docs change needed)
- [ ] Did you write any new necessary tests?

## Who can review?

@ArthurZucker @CyrilVallez (flex attention masking)

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46684)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46684&sha=2aee97
feat(pipelines): accept numpy arrays and tensors in ImageClassificationPipeline (#39607)
Author: kamran-nizamani
Pull request: yes

Fixes #39607

ImageClassificationPipeline's preprocess() now accepts numpy arrays 
and torch tensors directly in addition to PIL images, removing the 
need for users to manually convert their inputs before classification.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46573)

@bot /style

@bot /style

Style fix bot fixed some files and pushed the changes.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46573). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: exaone4_5

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46573&sha=09646b
ImageClassificationPipeline preprocess should accept numpy/tensor arrays
Author: idantene
Pull request: no
Labels: name: Feature request

### Feature request

Currently, `ImageClassificationPipeline` expects a PIL image or a string pointing to a URL.
This makes using existing datasets (e.g. from `torchvision`) a bit more difficult in the generic case. A non-transformers model would work with torch tensors, but not necessarily with a PIL image, whereas `ImageClassificationPipeline` won't work with torch tensors.

Under the hood, it seems to me that the image_processor should determine what inputs it supports -- I'm not sure why the `load_image` is hard-coded and/or does not support other well-known types.

### Motivation

I have a factory method to generate `pipelines` or use `torchvision` models, and the two are not currently interchangeable without having to monkey-patch that preprocess function (or provide a wrapper class for `ImageClassificationPipeline` that does not use the `load_image`) function.

### Your contribution

I could make the changes to the code if needed. Adding support is trivial.

Comments:
Hi @idantene, this seems like a good idea, but we should be very clear about the input tensor format we expect. @zucchini-nlp, let me know if this seems like this will work, or if you think there'll be any problems with it!

Sounds good to me. Image processors and processors already can accept `torch.Tensor` as input so the issue is only within `load_image` which checks for `PIL` or `str`

@idantene in that case, I think we'd be happy to review a PR for this!

Hi, I'd like to work on this feature.

Approach: I will update ImageClassificationPipeline's preprocess() 
method to accept numpy arrays and torch tensors in addition to PIL 
images, by adding type checks and appropriate conversions before 
processing.

Ready to submit PR once assigned.
fix: honor static cache max len config
Author: he-yufeng
Pull request: yes

## Summary

- honor `generation_config.cache_config["max_cache_len"]` when auto-preparing static caches
- keep the cache length at least as large as the current generation requires
- add a tiny GPT-2 regression test that checks the prepared `StaticCache` size

Fixes #46424.

## To verify

- `PYTHONPATH=src python -m pytest tests\generation\test_utils.py::GenerationIntegrationTests::test_static_cache_uses_max_cache_len_from_cache_config -q`
- `python -m py_compile src\transformers\generation\utils.py tests\generation\test_utils.py`
- `git diff --check`


Comments:
I rechecked the red check here. The product CI is green: all CircleCI jobs passed, including tests and quality checks.

The only failing check is the GitHub Actions `comment` job from `CircleCI Failure Summary Comment`. Its log reaches `CircleCI status: completed, conclusion: success`, then fails while fetching CircleCI workflow jobs because the CircleCI API response is not valid JSON (`jq`: `parse error: Invalid numeric literal`). That looks external to this patch rather than a code failure in the PR.

Blocking because I [asked you to stop](https://github.com/huggingface/transformers/pull/46481) and your agent opened another PR on exactly the same issue
Lfm2: also thread `seq_idx` through ShortConv.slow_forward (non-fast-path)
Author: ChangyiYang
Pull request: yes

Followup to #46588.

## Why this is needed

#46588 added `seq_idx` to `Lfm2ShortConv.cuda_kernels_forward`, which calls `causal_conv1d_fn(..., seq_idx=...)` — the CUDA fast path. But when `causal_conv1d` is **not** installed (e.g. AMD ROCm containers; the stock PyPI causal_conv1d has a CUDA-only build), `is_fast_path_available=False` and `Lfm2ShortConv.forward` falls back to `slow_forward`. `slow_forward` uses plain `nn.Conv1d` with padding and slides over the full packed sequence — the boundary-smear bug is unchanged on that path.

Verified inside the ROCm container we use:
```
is_fast_path_available = False
causal_conv1d_fn = None
```

## Fix

Add `seq_idx` to `Lfm2ShortConv.slow_forward`. When supplied (and there is no kv cache), slice `Bx` by segment boundaries derived from `seq_idx` and run `self.conv(...)` on each packed sample independently. `nn.Conv1d`'s own zero padding then acts as a per-segment receptive-field reset. The else-branch behaviour (no `seq_idx` / decode mode) is unchanged.

Also flips `forward` to pass `seq_idx` down to `slow_forward` (it was only being passed to `cuda_kernels_forward` before).

## End-to-end validation

LFM2-MoE 8B GRPO training on AMD MI300X, ROCm, `use_remove_padding=True`, `micro_batch_size_per_gpu=4`, fixed config / data / seeds — only the verl branch differs:

| | step-1 k3_kl | wandb |
|---|---|---|
| `verl` main (no patch, no fix) | **0.1711** | `cdn7a5rd` |
| fast-path patch alone (#46588 equivalent in a verl runtime monkey-patch) — slow_forward unfixed | 0.1521 (~RNG noise) | `m8mbzgu2` |
| **+ this slow_forward patch** | **0.0933** | `1w1oh748` |

i.e. the fast-path patch alone was a no-op on AMD; the slow_forward patch is what actually moves the needle on this hardware.

## Note on modular conversion

Same caveat as #46588: I applied the same body change manually to `modeling_lfm2_moe.py` because the modular converter doesn't re-resolve a parent class body for a `class Lfm2MoeShortConv(Lfm2ShortConv): pass` subclass when only the parent's body changes. Happy to switch to an explicit override in `modular_lfm2_moe.py` if reviewers prefer.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: lfm2, lfm2_moe

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46633)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46633). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[CTRL] Support attn_implementation dispatch
Author: YangKai0616
Pull request: yes

# What does this PR do?

The `CTRL` model fails during the initialization phase when using `from_pretrained(..., attn_implementation="sdpa")`. This PR enables standard `attn_implementation="sdpa"` dispatch to `CTRL` .

Hi @ArthurZucker, pls help review, thx!

Comments:
Hi @vasqu , please help review this PR, thank you!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ctrl

> Have left a few smaller comments. We could further refactor to also have the same namings by using the conversion mapping but tbh, I don't think it's worth it for this model as it's very old and likely to be deprecated at some point

Agreed. Going forward, I'll prioritize model importance when deciding whether a model needs refactoring. Thanks for reviewing this PR!

run-slow: ctrl

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27618111068)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/ctrl"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46073). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27618111068)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [107fc13c](https://github.com/huggingface/transformers/commit/107fc13c44344a18ebe39bbf316bd4ca892bf7c4) | workflow commit (merge commit) |
| PR | [7dae4dc7](https://github.com/YangKai0616/transformers/commit/7dae4dc7b74ef7691a7db927d177516433e399ec) | branch commit (from PR) |
| main | [6a2eb6a0](https://github.com/huggingface/transformers/commit/6a2eb6a02e012b382da475bba2d316be057bedca) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !
[blip_2] Support attn_implementation dispatch
Author: YangKai0616
Pull request: yes

# What does this PR do?

As per the title.

Comments:
@vasqu 

For the test case `test_modeling_instructblip.py::InstructBlipModelIntegrationTest::test_inference_flant5_xl`, the output from the current default SDPA branch differs from the default eager mode in upstream/main. 
However, it matches when using eager mode on the current branch or float32 dtype. `Expectations` should be updated, but since I don't know the device used in your CI, this PR will not be updated for now.

> For the test case test_modeling_instructblip.py::InstructBlipModelIntegrationTest::test_inference_flant5_xl, the output from the current default SDPA branch differs from the default eager mode in upstream/main.
However, it matches when using eager mode on the current branch or float32 dtype. Expectations should be updated, but since I don't know the device used in your CI, this PR will not be updated for now.

Can we force eager attention at load time instead for now?

> Can we force eager attention at load time instead for now?

Done.


Friendly ping @vasqu 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: blip_2, instructblip, instructblipvideo

run-slow: blip_2, instructblip, instructblipvideo

Just checking with slow CI but merging afterwards :hugs: 

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27617899953)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/blip_2", "models/instructblip", "models/instructblipvideo"]
quantizations: []

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46401). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27617899953)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [dae955a0](https://github.com/huggingface/transformers/commit/dae955a09805b1bd9ddb5106f3c62203272f3801) | workflow commit (merge commit) |
| PR | [116fe1e7](https://github.com/YangKai0616/transformers/commit/116fe1e7c568aa85df23bb3bff67199df80ff052) | branch commit (from PR) |
| main | [7ff490ac](https://github.com/huggingface/transformers/commit/7ff490aca20f597f6c6e42a449c7c8dd28807c6b) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !
Widen tols for float16/bfloat16
Author: Rocketknight1
Pull request: yes

We have [some flaky tests](https://app.circleci.com/pipelines/github/huggingface/transformers/175084/workflows/94a9c675-ec4a-4b6c-aa39-f21b9bbb53bb/jobs/2314705) caused by bfloat16 + narrow tolerances . This PR just modifies the checks so that we use wider tolerances when the test isn't `float32`.

Comments:
cc @ydshieh @tarekziade

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46036). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Gentle ping @ydshieh @tarekziade now the trip is over!

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma3n

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46036&sha=161867
[Fix] Serve drops some attributes at serialization
Author: remi-or
Pull request: yes

This PR fixes an issue in `serve`.
In `chat_completion.py`, we create a subclass of `ChatCompletionMessage` in which `reasoning_content` is a field. When this class is serialized, the field is lost because pydantic v2 uses the serializer from the annotated class (the original `ChatCompletionMessage` that was subclassed) rather than the runtime class (the subclass that we instantiate). Since the parent class, and its serializer, don't know about this subclass's attribute, it is lost at serialisation.

This PR fixes this by removing the subclass and rather passing `reasoning_content` as an extra field, which is allowed by the base class and serialized as well.
This fixes 6 tests in `test_serve.py`

Since I am not familiar with pydantic, this may not be the best way to resolve this. On the off-chance it is, happy to help.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46680). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46680)
[docs] compile level and batch/scheduling limits
Author: stevhliu
Pull request: yes

adds docs for #46533 and #46434:

- setting levels for `default_compile_level` and the tradeoffs
- setting `max_requests_per_batch` and `safety_margin`

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46676)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46676). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`Blip2`] Fix cross attention reshape
Author: vasqu
Pull request: yes

As per title. It's a regression caused by #46401 (discovered it during #46039) where the cross attn final shape is not done properly (the root cause is that there oftentimes is some broadcasting going on between q and kv so the final shape is not "static").

Tbh, I'm not sure why this was even mergable with the merge queue as it is already indicated with fast test using granite speech plus.

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: blip_2, instructblip, instructblipvideo

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46695). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46695)

run-slow: blip_2, instructblip, instructblipvideo, granite4_vision, granite_speech, granite_speech_plus

Just sanity checking again then merging :saluting_face: 

[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27637529102)

This comment contains `run-slow`, running the specified jobs: 

models: ["models/blip_2", "models/granite4_vision", "models/granite_speech", "models/granite_speech_plus", "models/instructblip", "models/instructblipvideo"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/27637529102)

### Commit Info
| Context | Commit | Description |
|---------|--------|-------------|
| RUN | [bab2f073](https://github.com/huggingface/transformers/commit/bab2f073aecb78ed3ffa03287c31415e71d1e77f) | workflow commit (merge commit) |
| PR | [6ee8ff21](https://github.com/vasqu/transformers/commit/6ee8ff216284229307aa0f1a7e15e3805f2e3ab5) | branch commit (from PR) |
| main | [e4189101](https://github.com/huggingface/transformers/commit/e41891017cd9316662b23861338ed66430c2e3b6) | base commit (on `main`) |

✅ No failing test specific to this PR 🎉 👏 !


Merging, all green and we need it to unblock tomorrow potentially
[docs] decode fast path
Author: stevhliu
Pull request: yes

adds docs for https://github.com/huggingface/transformers/pull/45653

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45899). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix M-RoPE `inv_freq` device and `meta` → `to_empty` re-init in Qwen3-VL family
Author: jamesbraza
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/45859
Fixes https://github.com/huggingface/transformers/issues/45902

This PR:
- Adds `.to(x.device)` to `Qwen3VLTextRotaryEmbedding.forward` so the M-RoPE matmul stops raising a CUDA-vs-CPU `wrapper_CUDA_bmm` mismatch when `inv_freq` is host-resident under FSDP2 cpu-offload.
    - My https://github.com/huggingface/transformers/pull/45861 cherry picked here.
- Has Qwen3-VL family M-RoPE rotary embeddings recompute `inv_freq` on the activation device every forward (for non-dynamic rope types) via `rope_init_fn(self.config, x.device)`, making forward output independent of the buffer's prior state.

## Code Agent Policy

- [x] I confirm that this is not a pure code agent PR.

## Who can review?

@ArthurZucker @Cyrilvallez @zucchini-nlp 

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: qwen3_5, qwen3_5_moe, qwen3_omni_moe, qwen3_vl, qwen3_vl_moe

Closing this since I don't think we really want the fix, right? The comments on both of the issues suggest this isn't actually a bug, so asking Claude to open a PR was a bit premature.
[CB] Hide activation footprint by using the CUDA graph pool
Author: remi-or
Pull request: yes

After this PR, activations created when warming up for a new CUDA graph catpure will be allocated inside the existing CUDA graph pool, as long as torch version is >= 2.4 . This is because the `Mempool` object of pytorch was inroduced in 2.5 . 
This ensures there is no OOM when warming for a new CUDA graph, which could previously happen because CB did not budget for two sets of activations on the GPU at the same time. 

## Performance

No major change. Peak VRAM used goes down as expected. Investigating a weird 16K rollout number.

## Tests

TBD

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_45911). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
add conversion scripts for EUPE
Author: molbap
Pull request: yes

# What does this PR do?

As per title. Adds the 3 convnext variants and 3 ViT variants of EUPE.
Fixes #45322 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46691). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for the review! I'm a bit against adding slices to the convert script because it's not really a test contract (integration tests solely for EUPE would do better for instance?) But since I had the logits for original impl, added it

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: dinov3_convnext, dinov3_vit

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46691)