gemma-4-E2B-it-text-MLX-4bit

unsloth/gemma-4-E2B-it-UD-MLX-4bit with the vision and audio towers removed. Text generation only — nothing else is changed.

The source is a unified (multimodal) MLX checkpoint. Text-only runtimes discard the vision/audio weights at load time anyway (mlx-lm's gemma4.py, mlx-swift-lm's Gemma4Model.sanitize), so shipping them costs about a gigabyte of download and storage for tensors that are never evaluated. This repo exists so a phone doesn't pay for them.

What was removed

Source size	4.52 GB
This repo	3.55 GB
Removed	0.97 GB (1551 tensors)

• vision_tower, audio_tower, embed_vision, embed_audio
• activation-range tensors (input_max/input_min/output_max/output_min)
• the redundant k_proj/v_proj/k_norm that the source ships for the trailing KV-shared layers, which own no K/V projection (both mlx-lm and mlx-swift-lm drop these; with them gone the checkpoint loads strictly)

No weights were requantized. Every remaining tensor is a byte-for-byte copy, so quality is identical to the source by construction. Verified with greedy decoding on both: all reference prompts and a 512-token sample are token-for-token identical.

Use

from mlx_lm import load, generate

model, tokenizer = load("k5p5/gemma-4-E2B-it-text-MLX-4bit")
print(generate(model, tokenizer, "Why is the sky blue?", max_tokens=200))

Produced with scripts/gemma4_text_only.py for Personai, an on-device iPhone chat app. Thanks to unsloth for the quantization and Google for the model.

License

Gemma. Use is governed by the Gemma Terms of Use, which apply to this derivative as they do to the original.