gemma4-12b-text-backbone

The language backbone of unsloth/gemma-4-12B-it-qat-q4_0-unquantized, extracted as a standalone text model.

The vision and audio towers (model.visual.*, embed_vision, embed_audio, vision_embedder, …) and their config are removed; the text decoder (model.language_model.*, with lm_head tied to the embeddings) is repackaged as a Gemma4UnifiedForCausalLM. Because Gemma-4 is early-fusion, a text-only forward through the full multimodal model uses only these weights — so this backbone is the substrate you get whether you strip before or after a text-only fine-tune, minus the multimodal plumbing.

Faithfulness

Verified against the original model's text-only forward: max |Δ logit| = 0.00e+00 with 100% next-token argmax agreement across prose, interior-monologue (with CIS structural tokens), code, and ASCII-grid (ARC) probes. Same decoder tensors, same text_config — no RoPE reconstruction, no weight permutation.

License

This is a Gemma derivative and is distributed under the Gemma Terms of Use, not Apache-2.0.

Produced by tools/extract_gemma4_text_backbone.py (Kenneth Grace & Kira).