Validate Gemma4 audio placeholders in processor inputs
Author: Sunt-ing
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47076)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47076)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

Gemma4 processors validate image placeholders, but they do not apply the same protocol to audio. Passing audio with text that does not contain an audio placeholder returns `input_features` while `input_ids` contains no audio tokens, so the model has no positions where those audio features can be consumed. The same issue appears when image and audio are passed without text: the processor auto-builds image placeholders, but skips the audio placeholder.

<details>
<summary>End-to-end processor repro on current main and this branch</summary>

Environment:

```text
Machine: Linux-5.15.0-78-generic-x86_64 host
Hardware: RTX 4090 available, CUDA disabled for this CPU-only processor repro, no model weights loaded
Python: 3.12.3
transformers: 5.14.0.dev0
torch: 2.8.0+cu128
CUDA_VISIBLE_DEVICES: ""
torch.cuda.is_available(): False
Before code: upstream main b70d02fc724d04c916832ca4ead03ff05e8fb1ee
After code: this PR branch 7a923235be6b4648772c37f6292fbccdfb32f20d
Model revisions:
  google/gemma-4-E4B-it main fee6332c1abaafb77f6f9624236c63aa2f1d0187
  google/gemma-4-26B-A4B-it main 20da991ab4afab98e8f910c4a2e8f4fbefc404ad
```

Command:

```bash
CUDA_VISIBLE_DEVICES="" PYTHONPATH=$PWD/src python repro_gemma4_audio_placeholder.py
```

`repro_gemma4_audio_placeholder.py`:

```python
import numpy as np
from PIL import Image

from transformers import AutoProcessor


for model_id in ["google/gemma-4-E4B-it", "google/gemma-4-26B-A4B-it"]:
    print("MODEL", model_id)
    processor = AutoProcessor.from_pretrained(model_id)
    audio = [np.zeros(1600, dtype=np.float32)]
    image = Image.new("RGB", (8, 8), "white")
    cases = [
        ("text+image missing token", {"text": "Describe this.", "images": image}),
        ("text+audio missing token", {"text": "Transcribe this.", "audio": audio}),
        ("text+audio with token", {"text": f"{processor.audio_token} Transcribe this.", "audio": audio}),
        ("image+audio no text", {"images": image, "audio": audio}),
    ]
    for name, kwargs in cases:
        print("CASE", name)
        try:
            out = processor(return_tensors="pt", **kwargs)
            ids = out["input_ids"][0].tolist()
            print(
                "OK shape",
                tuple(out["input_ids"].shape),
                "image_tokens",
                ids.count(processor.image_token_id),
                "audio_tokens",
                ids.count(processor.audio_token_id),
            )
            print("keys", sorted(out.keys()))
        except Exception as e:
            print("ERR", type(e).__name__, repr(str(e)))
```

Before on upstream main `b70d02fc724d04c916832ca4ead03ff05e8fb1ee`:

```text
MODEL google/gemma-4-E4B-it
CASE text+image missing token
ERR ValueError 'The total number of 
 tokens in the prompts should be the same as the number of images passed. Found [0] 
 tokens and [1] images per sample.'
CASE text+audio missing token
OK shape (1, 4) image_tokens 0 audio_tokens 0
keys ['attention_mask', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids']
CASE text+audio with token
OK shape (1, 9) image_tokens 0 audio_tokens 3
keys ['attention_mask', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids']
CASE image+audio no text
OK shape (1, 258) image_tokens 256 audio_tokens 0
keys ['attention_mask', 'image_position_ids', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids', 'pixel_values']
MODEL google/gemma-4-26B-A4B-it
CASE text+image missing token
ERR ValueError 'The total number of 
 tokens in the prompts should be the same as the number of images passed. Found [0] 
 tokens and [1] images per sample.'
CASE text+audio missing token
OK shape (1, 4) image_tokens 0 audio_tokens 0
keys ['attention_mask', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids']
CASE text+audio with token
OK shape (1, 9) image_tokens 0 audio_tokens 3
keys ['attention_mask', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids']
CASE image+audio no text
OK shape (1, 258) image_tokens 256 audio_tokens 0
keys ['attention_mask', 'image_position_ids', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids', 'pixel_values']
```

After on this branch:

```text
MODEL google/gemma-4-E4B-it
CASE text+image missing token
ERR ValueError 'The total number of 
 tokens in the prompts should be the same as the number of images passed. Found [0] 
 tokens and [1] images per sample.'
CASE text+audio missing token
ERR ValueError 'The total number of 
 tokens in the prompts should be the same as the number of audio inputs passed. Found [0] 
 tokens in the prompts and 1 audio inputs.'
CASE text+audio with token
OK shape (1, 9) image_tokens 0 audio_tokens 3
keys ['attention_mask', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids']
CASE image+audio no text
OK shape (1, 264) image_tokens 256 audio_tokens 3
keys ['attention_mask', 'image_position_ids', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids', 'pixel_values']
MODEL google/gemma-4-26B-A4B-it
CASE text+image missing token
ERR ValueError 'The total number of 
 tokens in the prompts should be the same as the number of images passed. Found [0] 
 tokens and [1] images per sample.'
CASE text+audio missing token
ERR ValueError 'The total number of 
 tokens in the prompts should be the same as the number of audio inputs passed. Found [0] 
 tokens in the prompts and 1 audio inputs.'
CASE text+audio with token
OK shape (1, 9) image_tokens 0 audio_tokens 3
keys ['attention_mask', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids']
CASE image+audio no text
OK shape (1, 264) image_tokens 256 audio_tokens 3
keys ['attention_mask', 'image_position_ids', 'input_features', 'input_features_mask', 'input_ids', 'mm_token_type_ids', 'pixel_values']
```

</details>

This PR aligns audio with the existing image placeholder protocol in `Gemma4Processor` and `Gemma4UnifiedProcessor`: audio inputs must have matching audio placeholders, text containing audio placeholders must have audio inputs, and textless image+audio inputs now get both placeholders. The audio check uses the total number of audio tokens across the batch, so batched chat-template inputs can still contain multiple audio clips in one sample. Text-only calls still work when a tokenizer has no audio special tokens.

Tests:

```bash
pytest \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_audio_inputs_require_audio_tokens \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_text_only_input_without_audio_token \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_textless_image_audio_inputs_add_both_placeholders \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_get_num_audio_tokens \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_get_num_multimodal_tokens_matches_processor_call \
  tests/models/gemma4_unified/test_processing_gemma4_unified.py::Gemma4UnifiedProcessorTest::test_audio_inputs_require_audio_tokens \
  tests/models/gemma4_unified/test_processing_gemma4_unified.py::Gemma4UnifiedProcessorTest::test_text_only_input_without_audio_token \
  tests/models/gemma4_unified/test_processing_gemma4_unified.py::Gemma4UnifiedProcessorTest::test_textless_image_audio_inputs_add_both_placeholders \
  tests/models/gemma4_unified/test_processing_gemma4_unified.py::Gemma4UnifiedProcessorTest::test_get_num_multimodal_tokens_matches_processor_call \
  -q
```

```text
========================= 9 passed, 1 warning in 0.83s =========================
```

Common processor tests covering multimodal prepared inputs:

```bash
pytest \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_flat_kwarg_applied_when_modality_dict_lacks_it \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_model_input_names \
  tests/models/gemma4/test_processing_gemma4.py::Gemma4ProcessorTest::test_processor_with_multiple_inputs \
  tests/models/gemma4_unified/test_processing_gemma4_unified.py::Gemma4UnifiedProcessorTest::test_flat_kwarg_applied_when_modality_dict_lacks_it \
  tests/models/gemma4_unified/test_processing_gemma4_unified.py::Gemma4UnifiedProcessorTest::test_model_input_names \
  tests/models/gemma4_unified/test_processing_gemma4_unified.py::Gemma4UnifiedProcessorTest::test_processor_with_multiple_inputs \
  -q
```

```text
========================= 6 passed, 1 warning in 2.45s =========================
```

```bash
git diff --check -- src/transformers/models/gemma4/processing_gemma4.py src/transformers/models/gemma4_unified/processing_gemma4_unified.py tests/models/gemma4/test_processing_gemma4.py tests/models/gemma4_unified/test_processing_gemma4_unified.py
ruff check src/transformers/models/gemma4/processing_gemma4.py src/transformers/models/gemma4_unified/processing_gemma4_unified.py tests/models/gemma4/test_processing_gemma4.py tests/models/gemma4_unified/test_processing_gemma4_unified.py
python utils/check_modular_conversion.py --files src/transformers/models/gemma4_unified/modular_gemma4_unified.py
```

```text
git diff --check: no output
ruff check: All checks passed!
check_modular_conversion: no output
```

Additional batched multi-audio smokes:

```text
direct processor: torch.Size([2, 18]) torch.Size([3, 30, 128]) 16
apply_chat_template: torch.Size([2, 31]) torch.Size([3, 30, 128]) 16
```

## Code Agent Policy

- [x] I confirm that this is not a pure code agent PR.

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://huggingface.co/docs/transformers/contributing) and the
      [Pull Request](https://huggingface.co/docs/transformers/pr_checks) checks?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes according to the [guidelines](https://github.com/huggingface/transformers/tree/main/docs)?
- [x] Did you write any new necessary [tests](https://huggingface.co/docs/transformers/testing)?

## Who can review?

cc @zucchini-nlp

Comments:
**[For maintainers]** Suggested jobs to run (before merge)

run-slow: gemma4, gemma4_unified

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47076)
**Latest run:** [28721817635:2](https://github.com/huggingface/transformers/actions/runs/28721817635)
**Result:** `success` | **Jobs:** 8 | **Tests:** 1,659 | **Failures:** 0 | **Duration:** 9m 33s

<!-- ci-dashboard-recap:end -->