Fix packed-sequence mask ignored when a 2D attention_mask is passed
Author: bin123apple
Pull request: yes

# What does this PR do?

`create_causal_mask` (via `_preprocess_mask_arguments` in `masking_utils.py`) only detects
packed sequences — multiple variable-length sequences packed into one row, with boundaries
encoded in `position_ids` — when `attention_mask is None`. In packed / `use_remove_padding`
training the attention_mask is typically a 2D all-ones tensor, which silently disables packed
detection: `packed_sequence_mask` stays `None`, SDPA takes the `is_causal` fast path, and tokens
of a later packed segment attend tokens of an earlier one (cross-sequence contamination). Loss
still decreases, so it is silent and easy to miss.

This lives in the shared masking path, so it is model-agnostic (any model on the SDPA/eager mask
path). gemma-4 are the most exposed: head_dim 512 forces SDPA (FlashAttention varlen,
which handles packing via `cu_seqlens`, is unavailable), and verl-style RL packs many sequences
per row.

## Fix

Drop the `attention_mask is None` clause from the packed-detection gate, so
`find_packed_sequence_indices` runs whenever `position_ids` is present (and there is no cache):

```python
-    if position_ids is not None and attention_mask is None and past_key_values is None:
+    if position_ids is not None and past_key_values is None:
```

Safe and minimal:
- `find_packed_sequence_indices` returns `None` for non-packed (strictly +1) `position_ids`, so
  this is a no-op there; the all-ones + contiguous case still hits the `is_causal` fast path
  (no throughput regression).
- For padded inputs the spurious segment boundaries fall on padding tokens that are masked
  anyway, so real-token attention is unchanged (the existing
  `test_chunked_mask_with_left_padding_*` tests still pass).
- `create_causal_mask` already ANDs `packed_sequence_mask_function(...)` and the 2D padding mask
  into the factory, so both packing and padding are honored.
- Only the (packed `position_ids` + 2D attention_mask) sub-case changes behavior.

## Reproducer

With packed `position_ids = [[0, 1, 2, 3, 0, 1, 2]]` under SDPA:
- `attention_mask=None` → correct block-diagonal mask.
- `attention_mask=torch.ones(...)` → `create_causal_mask` returns `None` (is_causal fast path),
  so the second segment cross-attends the first.

## Tests

Added `test_packed_sequence_mask_sdpa_with_2d_attention_mask` in
`tests/utils/test_masking_utils.py`: fails on `main` (`create_causal_mask` returns `None`),
passes with the fix. The full `tests/utils/test_masking_utils.py` suite is green.

Related to #25452 (the cross-contamination reported there has this gate as its root cause).

## Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, 
**we ask that new users do not submit pure code agent PRs** at this time. 
You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents
not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this
repeatedly or maliciously. 

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result, 
this policy is likely to be updated regularly in the near future. For more information, please read [`CONTRIBUTING.md`](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md).

- [x] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
**CI Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=46634)

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=46634&sha=cc0e6b

@Cyrilvallez @vasqu — would appreciate your eyes on this small `masking_utils` fix. `find_packed_sequence_indices` is skipped whenever a 2D `attention_mask` is passed, so padding-free / `use_remove_padding` training (2D all-ones mask + packed `position_ids`) silently loses packing on SDPA-only models like gemma-3/4. CI surfaced two regressions along the way (GIT, and gpt_neox/higgs `padding_matches_padding_free`) — both fixed, `tests_torch` is green; the remaining red checks are the hub-cache / offline-mode infra flakes.

Agree here, we really don't want to add addtional complexity for basically no benefit. Is there a repro where you needed this because as Cyril noted it's only for padding-free training (where you don't need a mask)

Thanks @Cyrilvallez @vasqu. You're both right. The all-ones mask is indeed redundant. I ran into this while writing a Gemma-4 adaptation patch for `verl`'s packed/padding-free training path, where the packed position_ids come with a 2D mask and the block-diagonal structure got silently dropped.

The correct fix is to derive segments from` position_ids` and attend per-segment which belongs downstream (in my Gemma adaptation patch), not in `create_causal_mask`.

So I'll close this. Leaving the repro + the head_dim=512 finding here in case it helps others. Thanks for the review!