Add support for Quatfit1 multimodal models
Author: Jatinverma0786
Pull request: yes

<!-- ci-dashboard-badge:start -->
[![CI](https://transformers-ci.lor-e.huggingface.cool/badge/pr?pr=47081)](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47081)
<!-- ci-dashboard-badge:end -->

# Add native support for the Quatfit1 multimodal architecture

## Summary

This pull request adds native support for **Quatfit1**, a multimodal architecture derived from Gemma4, to the Transformers library.

Quatfit1 introduces a dedicated `model_type` (`quatfit1`) while leveraging the existing Gemma4 implementation. Native integration allows Quatfit models to be loaded directly through the Transformers Auto Classes without requiring `trust_remote_code=True`.

---

## Motivation

Currently, Quatfit models require remote code execution to be loaded:

```python
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "Quatfit/Quatfit-Mini",
    trust_remote_code=True,
)
```

With native Transformers support, users can simply write:

```python
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "Quatfit/Quatfit-Mini"
)
```

---

## Features

Quatfit1 supports:

- Text generation
- Image understanding
- Audio understanding
- Video understanding
- Multimodal instruction following
- Long-context inference
- Hugging Face Auto Classes

---

## Architecture

Quatfit1 is implemented as a Gemma4-derived architecture with full configuration, modeling, feature extraction, image processing, video processing, and processor classes.

The following configuration classes are introduced:

- `Quatfit1Config` (`model_type = "quatfit1"`)
- `Quatfit1TextConfig` (`model_type = "quatfit1_text"`)
- `Quatfit1VisionConfig` (`model_type = "quatfit1_vision"`)
- `Quatfit1AudioConfig` (`model_type = "quatfit1_audio"`)

The following model classes are introduced:

- `Quatfit1ForConditionalGeneration`
- `Quatfit1ForCausalLM`
- `Quatfit1Model`
- `Quatfit1TextModel`
- `Quatfit1VisionModel`
- `Quatfit1AudioModel`
- `Quatfit1PreTrainedModel`

The following processing classes are introduced:

- `Quatfit1Processor`
- `Quatfit1AudioFeatureExtractor`
- `Quatfit1ImageProcessor`
- `Quatfit1ImageProcessorPil`
- `Quatfit1VideoProcessor`

A weight conversion script is also included:

- `convert_quatfit1_weights.py`

---

## Files Added

```
src/transformers/models/quatfit1/
__init__.py
configuration_quatfit1.py
convert_quatfit1_weights.py
feature_extraction_quatfit1.py
image_processing_quatfit1.py
image_processing_pil_quatfit1.py
modeling_quatfit1.py
modular_quatfit1.py
processing_quatfit1.py
video_processing_quatfit1.py
```

---

## Auto Class Support

This PR registers Quatfit1 for:

- `AutoConfig` -- via `Quatfit1Config`
- `AutoModel` -- via `Quatfit1ForConditionalGeneration`
- `AutoModelForCausalLM` -- via `Quatfit1ForConditionalGeneration`
- `AutoModelForImageTextToText` -- via `Quatfit1ForConditionalGeneration`

Auto-mappings are also registered for the audio feature extractor, image processors, video processor, and processor.

---

## Example Usage

### Text

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Quatfit/Quatfit-Mini")
model = AutoModelForCausalLM.from_pretrained("Quatfit/Quatfit-Mini")
```

### Multimodal

```python
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("Quatfit/Quatfit-Mini")
model = AutoModelForImageTextToText.from_pretrained("Quatfit/Quatfit-Mini")
```

---

## Backward Compatibility

This PR introduces a new architecture without modifying the behavior of existing Gemma4 models. Existing Gemma4 checkpoints continue to function as before.

---

## Test Files

```
tests/models/quatfit1/
__init__.py
test_feature_extraction_quatfit1.py
test_image_processing_quatfit1.py
test_modeling_quatfit1.py
test_processing_quatfit1.py
```

## Documentation

```
docs/source/en/model_doc/quatfit1.md
```

Added to `docs/source/en/_toctree.yml` and registered in auto-mappings.

---

## CI Status

All `make check-repository-consistency` checks pass:

- [x] Generate auto mappings (`check_auto.py`)
- [x] Public imports (`from transformers import *`)
- [x] Import complexity
- [x] Copied code consistency (`check_copies.py`)
- [x] Modular file conversions (`check_modular_conversion.py`)
- [x] Documentation table of contents
- [x] Modeling rules documentation
- [x] Docstring formatting (`check_docstrings.py`)
- [x] Dummy objects
- [x] Repository structure (`check_repo.py`) — model tested, documented, and public
- [x] `ruff check` — all pass
- [x] Verified model loading without `trust_remote_code=True`

---

## Checklist

- [x] Added Quatfit1 configuration classes (`Quatfit1Config`, `Quatfit1TextConfig`, `Quatfit1VisionConfig`, `Quatfit1AudioConfig`)
- [x] Added Quatfit1 model implementation (`Quatfit1ForConditionalGeneration`, `Quatfit1ForCausalLM`, `Quatfit1TextModel`, `Quatfit1VisionModel`, `Quatfit1AudioModel`)
- [x] Added Quatfit1 processing classes (`Quatfit1Processor`, `Quatfit1AudioFeatureExtractor`, `Quatfit1ImageProcessor`, `Quatfit1ImageProcessorPil`, `Quatfit1VideoProcessor`)
- [x] Added weight conversion script (`convert_quatfit1_weights.py`)
- [x] Added test files (modeling, processing, image processing, feature extraction)
- [x] Added documentation page (`quatfit1.md`) and registered in toctree
- [x] Registered `AutoConfig`
- [x] Registered `AutoModel`
- [x] Registered `AutoModelForCausalLM`
- [x] Registered `AutoModelForImageTextToText`
- [x] Registered processor and image/video processor auto-mappings
- [x] Ignored `Quatfit1AudioModel` and `Quatfit1VisionModel` from test requirement (built-in subcomponents)
- [x] Ran `check_auto.py --fix_and_overwrite` to update auto mappings
- [x] Ran `check_modular_conversion.py --fix_and_overwrite` to regenerate modeling file
- [x] Ran `check_docstrings.py --fix_and_overwrite` for docstring templates
- [x] Ran `make check-repository-consistency` — all checks pass
- [x] Verified model loading without `trust_remote_code=True`


Comments:
Thank you for your contribution 🤗!

## CI Security Gate — automatic approval blocked

This PR was **not** automatically approved for CI because the security gate failed.

**Possible reasons:**
- The PR touches 50 or more files — only PRs with fewer than 50 changed files are automatically approved
- A changed file is outside the allowed directories (`src/`, `tests/`, `docs/`, `utils/`), has a disallowed extension (only `.py`, `.txt`, `.md` permitted outside `tests/` and `docs/`), or is not `.md`/`.yml` inside `docs/`
- A new high-severity security issue was detected in the changed Python files (Bandit check)

See the [workflow run](https://github.com/huggingface/transformers/actions/runs/28770284389) for the exact violations.

A maintainer can review and manually approve CI if a finding is a false positive.

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, quatfit1

<!-- ci-dashboard-recap:start -->

---

### CI recap

**Dashboard:** [View test results in Grafana](https://transformers-ci.lor-e.huggingface.cool/d/pytest-observability-by-pr/pytest-observability-branch?var-pr=47081)
**Latest run:** [28835597075:2](https://github.com/huggingface/transformers/actions/runs/28835597075)
**Result:** `failure` | **Jobs:** 15 | **Tests:** 171,991 | **Failures:** 7 | **Duration:** 25h 24m

<!-- ci-dashboard-recap:end -->