noamaanMulla-03/transformers-issues huggingface dataset:
fix: cast q_offset/kv_offset to int in flex_attention_mask to prevent tensor-type mask shape errors
Author: Vitaliy-Pikalo
Pull request: yes
Labels: name: Code agent slop

Fixes #46682

## What does this PR do?

When callers pass `q_offset` or `kv_offset` as a `torch.Tensor` (e.g. a 0-dim tensor from `cache_position`) to `flex_attention_mask`, the tensor type propagates into `add_offsets_to_mask_function` and `prepare_padding_mask`. Inside `add_offsets_to_mask_function`, the expression `q_idx + q_offset` returns a tensor instead of a plain int, which causes `create_block_mask`'s mask modulation function to return a tensor with an extra dimension, triggering a `ValueError` about the mask having more than 4 dimensions.

### Fix

Explicitly cast `q_offset` and `kv_offset` to `int` at the top of `flex_attention_mask`, before either value is used. This is safe because the type annotation already declares both as `int`, and any 0-dim tensor with an integer value converts cleanly via `int()`. The per-call-site workaround of `.item()` described in the issue is no longer needed.

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests)?
- [x] Was this discussed/approved via a Github issue? Yes: #46682

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46788)
fix(tokenization): guard against IndexError when apply_chat_template …
Author: Vitaliy-Pikalo
Pull request: yes
Labels: name: Code agent slop

Fixes #46752

## What does this PR do?

`apply_chat_template` crashes with an `IndexError` when `conversation=[]` (an empty list) is passed.

**Root cause:** In `tokenization_utils_base.py` around line 3089, the code accesses `conversation[0]` without first checking that the list is non-empty:

```python
# Before (buggy):
if isinstance(conversation, (list, tuple)) and (
    isinstance(conversation[0], (list, tuple)) or hasattr(conversation[0], "messages")
):
```

**Fix:** Add a `len(conversation) > 0` guard before accessing the first element:

```python
# After (fixed):
if isinstance(conversation, (list, tuple)) and len(conversation) > 0 and (
    isinstance(conversation[0], (list, tuple)) or hasattr(conversation[0], "messages")
):
```

This is the minimal, safe fix: an empty conversation simply falls through to the `else` branch (`conversations = [conversation]`, `is_batched = False`), which is the correct behavior.

## Reproduction

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")

# Before: raises IndexError: list index out of range
# After: returns ""
tokenizer.apply_chat_template([], tokenize=False)
```

Also reproduced on `google/gemma-2-2b-it`.

- [ ] I confirm that this is not a pure code agent PR.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46791)

> ⚠️ **Code quality check failed** — all test jobs were skipped. Fix the code quality issues and push again to run tests.

The `pr-ci / Check code quality` failure is a self-hosted runner infrastructure error ("Executing the custom container implementation failed"), not a code issue. The change is clean — single-line guard `len(conversation) > 0` at 81 chars, no ruff violations. Could a maintainer re-run the failed job when convenient? Thanks!
Add CLIP refactor migration note for compatibility guidance
Author: Charuhasini30
Pull request: yes
Labels: name: Code agent slop

This PR adds a migration note in the CLIP documentation to help users understand compatibility changes introduced by recent Transformers refactor (v5+).

It explains:
- Updated CLIP internal representation changes
- Impact on legacy LoRA and checkpoint formats
- Suggested migration approach for downstream users

This improves clarity and reduces confusion for external projects relying on CLIP models.

Comments:
Thanks! I added a small migration note to help clarify CLIP refactor compatibility for downstream users.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46803)

your agent is talking to itself
[docs] Improve formatting of DiffusionGemma common caveats
Author: Charuhasini30
Pull request: yes

This PR improves the formatting of the DiffusionGemma documentation by:

- Converting caveats into a markdown list
- Wrapping parameter names (`use_cache`, `top_k`) in code formatting for clarity

No functional changes.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46812)

Sorry, we're not accepting small doc/typo fixes like this right now! We'll probably just handle this kind of thing with a big agent pass, or in batched larger PRs
Make patched testing debug logs xdist-safe
Author: oleksii-tumanov
Pull request: yes

Fixes #45561

Patched testing debug logs. Instead of having all workers write to the same `captured_info.txt` file, each `pytest-xdist` worker now writes to its own `captured_info_<worker>.txt` file, while non-`xdist` runs keep the existing `captured_info.txt` behavior. Resetting the logs now clears only the current worker's file.

The reporting path was updated as well so worker-specific `captured_info*.txt` files are aggregated back into the existing `captured_info` reporting field.

I did not find an existing PR covering this narrower fix.

Tests:
- `python -m pytest tests/utils/test_testing_utils.py -q`
- `python -m ruff check tests/utils/test_testing_utils.py utils/notification_service.py`
- `python -m ruff format --check tests/utils/test_testing_utils.py utils/notification_service.py`


Comments:
Thank you for your contribution 🤗!

## CI Security Gate — automatic approval blocked

This PR was **not** automatically approved for CI because the security gate failed.

**Possible reasons:**
- The PR touches 50 or more files — only PRs with fewer than 50 changed files are automatically approved
- A changed file is outside the allowed directories (`src/`, `tests/`, `docs/`, `utils/`), has a disallowed extension (only `.py`, `.txt`, `.md` permitted outside `tests/` and `docs/`), or is not `.md`/`.yml` inside `docs/`
- A new high-severity security issue was detected in the changed Python files (Bandit check)

See the [workflow run](https://github.com/huggingface/transformers/actions/runs/27944110873) for the exact violations.

A maintainer can review and manually approve CI if a finding is a false positive.

Thanks for taking a look when you have a chance. I updated this to also keep the CI reporting path compatible with the new worker-specific `captured_info_<worker>.txt` files, while preserving the existing `captured_info.txt` behavior for non-xdist runs. CI is green, and I am happy to adjust the aggregation format if you prefer a different presentation.

cc @tarekziade @ydshieh
Fix broken internal links in model documentation
Author: ShamSaleem
Pull request: yes

What does this PR do?
Several model doc pages link to internal pages with malformed paths that resolve to 404s in the built docs:

bert-generation: ../quantizationbitsandbytes (missing slash)
opt: ..quantization/bitsandbytes (missing slash)
led: .longformer (stray leading dot)
instructblip (×2): blip2 → the page is blip-2
Repointed each to its existing target file, matching the ../quantization/overview convention already used on the same pages. Docs-only change, no code or behavior affected.

Fixes # (no tracking issue — self-found documentation defects)

Code Agent Policy
I confirm that this is not a pure code agent PR.
Before submitting
 This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
 Did you read the [contributor guideline](https://huggingface.co/docs/transformers/contributing) and the [Pull Request](https://huggingface.co/docs/transformers/pr_checks) checks?
 Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
 Did you make sure to update the documentation with your changes according to the [guidelines](https://github.com/huggingface/transformers/tree/main/docs)?
 Did you write any new necessary [tests](https://huggingface.co/docs/transformers/testing)?
Who can review?
Documentation: @stevhliu

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46807)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46807). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix q_offset tensor causing wrong flex attention mask shape
Author: abderahmane-ai
Pull request: yes

## What does this PR do?

Fixes a bug where `q_offset` passed as a `torch.Tensor` (from `StaticCache`) causes the `BlockMask` to have wrong shape inside `flex_attention` mask creation. When a tensor is used in `q_idx + q_offset`, the result becomes a tensor, and tensor-based indexing into `document_ids[...]` adds an extra dimension.

Two sites fixed:
1. **`masking_utils.py:_preprocess_mask_arguments`** — converts `q_offset` to Python `int` when `StaticCache.get_seq_length()` returns a tensor
2. **`flex_attention.py:make_flex_block_causal_mask`** (deprecated path) — converts `offsets[0]` and `offsets[1]` to `int` at point of use

## Details

`StaticLayer.cumulative_length` is intentionally a 1-D tensor (to avoid torch.compile recompilation on updates). When `get_seq_length()` returned it, the tensor propagated into mask creation code that expected `int`. The fix converts to `int` at the boundary, preserving `StaticLayer` internals for compilation while giving mask creation the plain int it expects.

Fixes #46682

## Tests

- `tests/utils/test_masking_utils.py` — 9 passed
- `tests/models/gpt_neox/test_modeling_gpt_neox.py::test_flex_attention_with_grads` — passed

## AI contribution

AI assistance was used to prepare this patch. The author has reviewed every changed line and run the relevant tests.

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46757)
Fix StaticLayer get_seq_length type hint
Author: he-yufeng
Pull request: yes

## Summary
- widen the cache-layer `get_seq_length()` contract to allow `torch.Tensor`
- update `StaticLayer.get_seq_length()` to match the runtime value it can return
- add a small regression for the public type-hint contract, without changing cache runtime behavior

Fixes #45987.

## To verify
- `PYTHONPATH=src python -m pytest tests\utils\test_cache_utils.py::CacheTest::test_static_layer_seq_length_type_hint_allows_tensor -q`
- `PYTHONPATH=src python -m py_compile src\transformers\cache_utils.py tests\utils\test_cache_utils.py`
- `python -m ruff check src\transformers\cache_utils.py tests\utils\test_cache_utils.py`
- `python -m ruff format --check src\transformers\cache_utils.py tests\utils\test_cache_utils.py`
- `git diff --check`
- `git fsck --no-progress`

Local note: pytest emitted an unrelated warning about an unknown local config option `env`; the targeted test passed.


Comments:
Thanks again for the review. This is approved and all visible CircleCI/status checks are green, so it should be ready to merge whenever you have time.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46474). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Approved and green except for `ci/circleci: pipelines_torch`. Could someone re-run that job, or point me at anything in the diff I should fix? It is just a `get_seq_length` type-hint change.
Fix flex_attention block mask creation when `get_seq_length` returns a tensor
Author: jiqing-feng
Pull request: yes

## Description

When generating with `attn_implementation="flex_attention"` and a `StaticCache`
(e.g. `cache_implementation="static"`), the decode step crashes with:

```
ValueError: too many values to unpack (expected 4)
  File ".../torch/nn/attention/flex_attention.py", line 1330, in _convert_mask_to_block_mask
    B, H, Q, KV = mask.shape
```

## Root cause

In `_preprocess_mask_arguments`, `q_offset` comes from `past_key_values.get_seq_length()`,
which can return a **1-element tensor** rather than a scalar. This offset is added to the
query index inside the flex `mask_mod` function, so the per-index result gets broadcast to
shape `(1,)` instead of staying a 0-dim scalar. The block mask then becomes 5D, which
`create_block_mask` cannot unpack into `(B, H, Q, KV)`.

## Fix

Reshape `q_offset` to a 0-dim scalar before use:

```python
q_offset = q_offset.reshape(()).to(inputs_embeds.device) if isinstance(q_offset, torch.Tensor) else q_offset
```

## Reproduction

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-0.6B", dtype=torch.bfloat16, attn_implementation="flex_attention",
    device_map="auto"
)

inputs = tok("Hello world, tell me a short story.", return_tensors="pt").to(model.device)
model.generate(**inputs, max_new_tokens=16, do_sample=False, cache_implementation="static")
```

Fails before the fix, generates normally after. Reproduces on both CUDA and XPU.


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46802)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46802). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix prompt lookup decoding crash when no EOS token is configured
Author: Sunt-ing
Pull request: yes


# What does this PR do?

Prompt lookup decoding (`prompt_lookup_num_tokens=N`) crashes with a `TypeError` whenever no EOS token is in effect, e.g. open-ended generation with `eos_token_id=None`, or a base checkpoint with no `eos_token_id`:

```python
import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-360M-Instruct").eval()
model.generation_config.pad_token_id = 0

# Trailing ngram repeats earlier in the input, so prompt lookup finds a match.
ids = torch.tensor([[5, 6, 7, 8, 9, 10, 11, 7, 8]])
model.generate(ids, attention_mask=torch.ones_like(ids), max_new_tokens=15,
               prompt_lookup_num_tokens=3, eos_token_id=None)
# TypeError: isin() received an invalid combination of arguments - got (Tensor, NoneType), ...
```

**Bug:** once an ngram match is found, `PromptLookupCandidateGenerator.get_candidates` runs `torch.isin(chosen_ids, self.eos_token_id)` to crop the candidate at the first EOS, but `self.eos_token_id` is `None` when no EOS is set. `eos_token_id=None` is not refilled from the config, so a normal `generate` call reaches this.

**Fix:** wrap the EOS-cropping block in `if self.eos_token_id is not None:`. This is the same guard already used for the same tensor in `SinglePositionMultiTokenCandidateGenerator`, and matches how `AssistedCandidateGenerator` normalizes a missing EOS. Behaviour is unchanged when an EOS is set; with no EOS the crop is skipped (there is nothing to stop at).

<details>
<summary>Reproduction and verification</summary>

The snippet above raises `TypeError` on `main` and returns normally with this PR. Reverting only the source change reproduces the crash; restoring it fixes it.

Regression test `tests/generation/test_utils.py::test_prompt_lookup_decoding_no_eos_token` (next to `test_prompt_lookup_decoding_stops_at_eos`) constructs `PromptLookupCandidateGenerator(eos_token_id=None, ...)`, feeds an input that forces a match, and asserts `get_candidates` returns the continuation instead of raising. It fails with `TypeError` at `candidate_generator.py:1138` without the fix and passes with it.

No regressions: `tests/generation/test_candidate_generator.py` (13 passed) and the llama `prompt_lookup`/`assisted`/`speculative` candidate-generation tests (6 passed).

Env: transformers `main` (5.13.0.dev0), torch 2.8.0+cu128, Python 3.12, Linux x86_64.
</details>

cc @zucchini-nlp


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46790). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46790)
update kernels-community/aiter-rope version
Author: Abdennacer-Badaoui
Pull request: yes

As per title. 
Related to : https://github.com/huggingface/kernels-community/pull/974

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46810). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Perfect, merging :saluting_face: thanks a lot for pushing on the AMD side, super nice

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46810)
Migrate ALBERT task heads to self.loss_function and add ForMultipleChoiceLoss
Author: Sahith59
Pull request: yes

Hey @ArthurZucker,

Continuing the `loss_function` refactor for ALBERT.

I migrated all the standard task heads (`AlbertForMaskedLM`, `AlbertForSequenceClassification`, `AlbertForTokenClassification`, `AlbertForQuestionAnswering`, and `AlbertForMultipleChoice`) to use `self.loss_function`. The hardcoded loss imports are also removed since they aren't used inline anymore.

One thing I noticed while doing this: `ForMultipleChoice` was actually missing from `LOSS_MAPPING` in `loss_utils.py`. If a model tried to use it, it would silently fall back to `ForCausalLMLoss` which expects different arguments and tensor shapes. I went ahead and added a dedicated `ForMultipleChoiceLoss` helper that cleanly computes cross-entropy over the `[batch_size, num_choices]` logits without any extra reshaping, and registered it in the mapping.

Note: I left `AlbertForPreTraining` alone since it combines MLM and SOP losses in the same forward pass and doesn't map perfectly to the single output helper pattern.

All tests are passing locally. Let me know if anything needs to be tweaked!


Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: albert

**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46820)
[T5] Support attn_implementation=sdpa dispatch
Author: YangKai0616
Pull request: yes

# What does this PR do?

The T5 model fails during the initialization phase when using `from_pretrained(..., attn_implementation="sdpa")`. This PR enables standard `attn_implementation="sdpa"` dispatch to T5 .

Hi @vasqu , pls help review, thx!

Comments:
This PR updates `T5Attention` / `T5Stack`, and it now triggers `check_copies.py` failures in downstream T5 copied models: `Pop2Piano`, `mT5`, `UDOP`, and `LongT5`.

I see two possible ways to handle this:

1. Propagate the copied-code changes to those models and keep the `# Copied from` links intact.
2. Keep the PR scoped to `T5` and mark the affected downstream blocks as `adapted/ignored` where they intentionally diverge.

Would you prefer that I sync the affected copied models as part of this PR, or keep the PR T5-only and handle the copied-code exceptions explicitly?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: musicgen, musicgen_melody, t5

Transferred to @jiqing-feng . Closing this PR.
[`CB`] Add FA2 to the fast path
Author: vasqu
Pull request: yes

Should be resolved now on upstream kernels so no longer any need to exclude FA2

Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46729)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_46729). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Not sure this is ready to merge yet, I cannot get it to work :( probably an issue with kernels, looking into it

EDIT: created this issue for tracking https://github.com/huggingface/kernels-community/issues/969

Yep it's internally discussed atm that older torch versions will be dropped. Trying to convince them to rebuild for at least FA based kernels as they are high-usage

Indeed, would be interesting to know but I'd wager a lot of usage comes from FA kernels, since they are so used and hard to install.

@remi-or I think we can still merge this for now? Getting broader support for older torch is a different matter tbh

Yes, I think we are good to go. Ran the numbers here on H100, accuracy is good:
| label | FA2 acc | FA3 acc | FA2 (tok/s) | FA3 (tok/s) | diff |
|----------------------|---------|---------|-------------|-------------|---------|
| gsm8k_default | 0.823 | 0.822 | 1405.59 | 2462.58 | +75.20% |
| gsm8k_sampling | 0.789 | 0.787 | 1241.53 | 1963.87 | +58.18% |
| gsm8k_compile | 0.824 | 0.821 | 1417.88 | 2480.69 | +74.96% |
| gsm8k_no_fast_decode | 0.823 | 0.822 | 1346.04 | 2367.29 | +75.87% |
| gsm8k_bare_bones | 0.825 | 0.821 | 1286.25 | 1888.73 | +46.84% |
| ifeval_default | 0.458 | 0.445 | 8878.32 | 9662.19 | +8.83% |
| few_blocks | - | - | 655.05 | 664.32 | +1.42% |
| multi_return_seq | - | - | 1644.36 | 1587.81 | -3.44% |
| rollouts_1024 | - | - | 3611.96 | 3552.55 | -1.64% |
| rollouts_2048 | - | - | 3411.95 | 3354.97 | -1.67% |
| rollouts_4096 | - | - | 2999.77 | 2962.53 | -1.24% |
| rollouts_8192 | - | - | 2402.45 | 2367.11 | -1.47% |
| rollouts_16384 | - | - | 1571.13 | 1568.41 | -0.17% |

throughput gap for prefill-heavy workloads is impressive though.