MichaelAnthony/gemma4-e2b-Snowfox-MLX-4bit

Gemma 4 E2B SnowFox MLX 4-bit (affine, group 64)

Standard MLX-VLM 4-bit affine weight quantization of the SnowFox model — the MLX equivalent of GGUF Q4_K_M. This is a genuine MLX-VLM package (quantized safetensors + config.json carrying a quantization field), not a GGUF file or a renamed HF checkpoint.

SnowFox is a language-only LoRA merge based on Google's Gemma 4 E2B instruction QAT-derived checkpoint. The image and audio towers were frozen during fine-tuning and are retained here, together with the processor and tokenizer needed by MLX-VLM.

Exact lineage

• Base: google/gemma-4-E2B-it-qat-q4_0-unquantized
• Quantized from: MichaelAnthony/gemma4-e2b-Snowfox-hf (the canonical merged BF16 source)
• FP16 reference: MichaelAnthony/gemma4-e2b-Snowfox-MLX
• Quantization: MLX affine, 4-bit, group size 64 ({"group_size": 64, "bits": 4, "mode": "affine"})

What is quantized

• 280 language-model layers (q/k/v/o projections, MLP gate/up/down, the multimodal embedding projections, and the large embeddings) are 4-bit affine quantized: packed uint32 weight (8 values per word, low nibble first) + float16 scales/biases.
• The vision tower and audio tower are left in float16 (dense) — matching MLX-VLM's convert --quantize, which skips multimodal modules. Their QAT ClippableLinear layers carry input/output clipping parameters (input_max/input_min/output_max/output_min) that must not be affine-quantized, so they stay dense and are loaded as regular nn.Linear.
• The dense per-layer input embedding (embed_tokens_per_layer) is quantized here, so the language model stays compact without exceeding the Metal buffer cap.

Package contents

• model-00001-of-00001.safetensors (3,550,670,830 bytes): the 4-bit MLX model in a single shard (~3.55 GB total).
• model.safetensors.index.json: complete shard map.
• config.json (with quantization + quantization_config), generation_config.json, processor_config.json, tokenizer files, and chat_template.jinja.

Model size vs HF parameter display

This is a ~5.1B-parameter model (2.3B effective), identical to the source SnowFox checkpoint. Hugging Face's model page reports ~1.2B because the 4-bit weights are stored as packed uint32 words (8 values each) and HF counts each packed word as one parameter. The packed word count is a storage detail, not the parameter count.

Verification performed

The conversion host has no Apple-Silicon MLX runtime, so the quantized package was structurally validated before upload:

• 1,951 source tensors mapped with no missing or extra keys; 280 language-model layers quantized; vision/audio towers left dense.
• Quantized weight format matches the MLX affine contract: 4-bit values packed 8-per-uint32 (low nibble first), dequantization scale * q + bias, group 64.
• Round-trip dequantization of sampled layers reproduces the source weights to within 4-bit precision.

Apple-Silicon MLX-VLM inference has not been run. Treat this as a structurally validated quantization pending a real Apple-Silicon text / image / audio smoke test.

Run on Apple Silicon

Use full MLX-VLM (not text-only MLX-LM) — Gemma 4 E2B includes image and audio:

python -m pip install "mlx-vlm==0.6.13"

python -m mlx_vlm.generate \
  --model MichaelAnthony/gemma4-e2b-Snowfox-MLX-4bit \
  --max-tokens 128 \
  --temperature 0.0 \
  --prompt "Explain what SnowFox is in one sentence."

Add --image /path/to/image.png for image prompting.

License

Gemma 4 is Apache-2.0. This derivative package uses the Apache-2.0 license declared by the pinned base model.