gemma-4-12B-it-oQ4

This model was quantized from google/gemma-4-12B-it using oQ mixed-precision quantization.

Update notice

Updated on 2026-06-07 23:51 KST.

The processor_config.json file was corrected so current MLX-VLM/oMLX Gemma 4 unified loading can use the local Gemma4UnifiedProcessor instead of falling back to the Transformers processor path. The previous metadata passed unsupported video_processor keys (do_convert_rgb, do_sample_frames, resample, return_metadata) into Gemma4UnifiedVideoProcessor. In oMLX this could make VLM loading fail, after which oMLX could silently fall back to a text-only LLM engine. In that fallback state image, audio, and video inputs do not reach the model even though text generation still works.

No weight files were changed in this update.

Compatibility notes

• Image input: supported by the model and expected to work in oMLX when the model is loaded as vlm.
• Audio input: supported by Gemma 4 12B unified. oMLX input_audio with WAV base64 was verified locally after this processor metadata fix. Keep audio clips within the upstream Gemma 4 12B limit of 30 seconds.
• Video capability: Gemma 4 12B unified supports video when the serving stack actually preserves and preprocesses video input. However, current oMLX OpenAI-compatible chat input handling may drop direct video / input_video content parts before they reach the VLM engine. For oMLX today, use extracted frames/contact sheets as images, or use a runtime/API path that explicitly supports video preprocessing.
• oMLX fallback caveat: if /v1/models/status shows this model as engine_type: "batched" or model_type: "llm", multimodal inputs are not being used. Restart/reload oMLX after updating this file and confirm the model is routed as vlm.

Quantization details

• Model type: gemma4_unified
• Bits: 4
• Group size: 64
• Format: MLX safetensors

Quick oMLX sanity check

After downloading or updating the model, confirm oMLX sees it as VLM:

curl http://127.0.0.1:8000/v1/models/status

Look for:

{
  "id": "gemma-4-12B-it-oQ4",
  "engine_type": "vlm",
  "model_type": "vlm",
  "config_model_type": "gemma4_unified"
}