Normalize multimodal input keys in AnyToAnyPipeline
Author: Sunt-ing
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47074)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47074)
<!-- ci-dashboard-badge:end -->

# What does this PR do?

`AnyToAnyPipeline` accepts multimodal inputs through both direct keyword arguments and dict/dataset samples, but two of those paths used singular keys before calling the processor. `videos=` was repacked as `video`, and dict inputs like `{"image": ..., "video": ...}` were forwarded unchanged. Qwen Omni processors expect `images=` and `videos=`, so video inputs could be ignored and image inputs could crash when an image placeholder was expanded without an image grid.

This normalizes pipeline inputs before the processor call. Direct `videos=` stays plural, dict `image` / `video` aliases are converted to `images` / `videos`, and explicit plural dict values keep precedence.

<details>
<summary>End-to-end pipeline repro (real AnyToAnyPipeline + real Qwen3-Omni processor)</summary>

Environment:

```text
Hardware: AutoDL c4090, NVIDIA GeForce RTX 4090 present; repro run CPU only with CUDA_VISIBLE_DEVICES=""
OS: Linux AutoDL container
Python: 3.12.3
PyTorch: 2.8.0+cu128
Transformers main: b70d02fc724d04c916832ca4ead03ff05e8fb1ee
Processor metadata: Qwen/Qwen3-Omni-30B-A3B-Instruct via AutoProcessor.from_pretrained
Model: small fake Qwen3OmniMoeForConditionalGeneration; it only replaces the 30B weight download and still exercises pipeline("any-to-any") plus the real processor input path
Command: CUDA_VISIBLE_DEVICES="" python repro_any_to_any_keys.py
```

```python
import numpy as np
import torch
from PIL import Image

from transformers import AutoProcessor, GenerationConfig, pipeline
from transformers.pipelines import AnyToAnyPipeline


class Qwen3OmniMoeForConditionalGeneration(torch.nn.Module):
    input_modalities = ("image", "video", "audio", "text")
    output_modalities = ("text", "audio")

    def __init__(self):
        super().__init__()
        self.config = type("Cfg", (), {"_commit_hash": None, "model_type": "qwen3_omni_moe"})()
        self.generation_config = GenerationConfig(max_new_tokens=1)
        self.last_keys = None

    @property
    def device(self):
        return torch.device("cpu")

    def can_generate(self):
        return True

    def generate(self, input_ids=None, **kwargs):
        self.last_keys = sorted(kwargs.keys())
        batch_size = input_ids.shape[0] if input_ids is not None else 1
        token = torch.full((batch_size, 1), 42, dtype=input_ids.dtype if input_ids is not None else torch.long)
        if input_ids is None:
            return token
        return torch.cat([input_ids, token], dim=1)


processor = AutoProcessor.from_pretrained("Qwen/Qwen3-Omni-30B-A3B-Instruct", trust_remote_code=False)

direct_model = Qwen3OmniMoeForConditionalGeneration()
direct_pipe = pipeline("any-to-any", model=direct_model, processor=processor)
video = np.zeros((2, 8, 8, 3), dtype=np.uint8)
direct_output = direct_pipe(
    text="describe the video",
    videos=video,
    return_full_text=False,
    max_new_tokens=1,
    processor_kwargs={"num_frames": 2},
)
print("direct_videos", "GEN_KEYS", direct_model.last_keys, "HAS_VIDEO_GRID", "video_grid_thw" in direct_model.last_keys)
print("direct_videos", "OUT", direct_output)

dict_model = Qwen3OmniMoeForConditionalGeneration()
dict_pipe = AnyToAnyPipeline(model=dict_model, processor=processor, max_new_tokens=1)
image = Image.new("RGB", (16, 16), color="white")
image_text = f"{processor.image_token} describe"
video_text = f"{processor.video_token} describe"

for label, inputs in [
    ("dict_image", {"text": image_text, "image": image}),
    ("dict_video", {"text": video_text, "video": video}),
]:
    try:
        output = dict_pipe(inputs, return_full_text=False, processor_kwargs={"num_frames": 2})
        print(label, "OK", "GEN_KEYS", dict_model.last_keys, "OUT_LEN", len(output))
    except Exception as error:
        print(label, "ERR", type(error).__name__, repr(str(error)))
```

```text
# current main
[transformers] Keyword argument `video` is not a valid argument for this processor and will be ignored.
[transformers] Keyword argument `image` is not a valid argument for this processor and will be ignored.
direct_videos GEN_KEYS ['attention_mask', 'generation_config', 'max_new_tokens'] HAS_VIDEO_GRID False
direct_videos OUT [{'input_text': 'describe the video', 'generated_text': 'K'}]
dict_image ERR StopIteration ''
dict_video ERR StopIteration ''

# after this PR
direct_videos GEN_KEYS ['attention_mask', 'generation_config', 'max_new_tokens', 'pixel_values_videos', 'video_grid_thw', 'video_second_per_grid'] HAS_VIDEO_GRID True
direct_videos OUT [{'input_text': 'describe the video', 'generated_text': 'K'}]
dict_image OK GEN_KEYS ['attention_mask', 'generation_config', 'image_grid_thw', 'pixel_values'] OUT_LEN 1
dict_video OK GEN_KEYS ['attention_mask', 'generation_config', 'pixel_values_videos', 'video_grid_thw', 'video_second_per_grid'] OUT_LEN 1
```

</details>

I did not add a dedicated unit test for this small key-normalization change. The repro above exercises the real pipeline plus real Qwen3-Omni processor path before and after the patch.

<details>
<summary>Touched-file checks</summary>

Checks:

```text
python -m ruff check src/transformers/pipelines/any_to_any.py
python -m ruff format --check src/transformers/pipelines/any_to_any.py
git diff --check
```

</details>

## Code Agent Policy

- [x] I confirm that this is not a pure code agent PR.

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://huggingface.co/docs/transformers/contributing) and the
      [Pull Request](https://huggingface.co/docs/transformers/pr_checks) checks?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes according to the [guidelines](https://github.com/huggingface/transformers/tree/main/docs)?
- [ ] Did you write any new necessary [tests](https://huggingface.co/docs/transformers/testing)?

## Who can review?

cc @Rocketknight1 @zucchini-nlp

Comments:
Thanks @zucchini-nlp. The follow-up narrows this to the direct `videos=` path and removes the dict `image` / `video` alias normalization.

It adds a focused regression with the tiny Qwen2.5-Omni processor plus a fake generate model, and extends the shared pipeline tester path to exercise `videos=` when a video example is available. Reverting the source line makes the focused test fail because `pixel_values_videos` and `video_grid_thw` never reach `generate`; with the fix it passes.

I also tried enabling the real Qwen2.5-Omni pipeline tester directly. It enters the long Omni generation path on CPU, so I kept this at the pipeline/processor boundary.


The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_47074). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thank you for your contribution 🤗!

## CI Security Gate — automatic approval blocked

This PR was **not** automatically approved for CI because the security gate failed.

**Possible reasons:**
- The PR touches 50 or more files — only PRs with fewer than 50 changed files are automatically approved
- A changed file is outside the allowed directories (`src/`, `tests/`, `docs/`, `utils/`), has a disallowed extension (only `.py`, `.txt`, `.md` permitted outside `tests/` and `docs/`), or is not `.md`/`.yml` inside `docs/`
- A new high-severity security issue was detected in the changed Python files (Bandit check)

See the [workflow run](https://github.com/huggingface/transformers/actions/runs/29503171786) for the exact violations.

A maintainer can review and manually approve CI if a finding is a false positive.

@bot /style

Style fix is beginning .... [View the workflow run here](https://github.com/huggingface/transformers/actions/runs/29571084964).

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47074)
**Latest run:** [28930810379:1](https://github.com/huggingface/transformers/actions/runs/29503173405)
**Result:** `failure` | **Jobs:** 15 | **Tests:** 172,748 | **Failures:** 0 | **Duration:** 8h 3m

> **Code quality check failed**: test jobs were skipped. Fix the code quality issues and push again to run tests.

<!-- ci-dashboard-recap:end -->