nikimori/gemma-4-e2b-8bit-slim

Gemma 4 E2B, 8-bit MLX (slim)

A slimmer re-pack of mlx-community/gemma-4-e2b-8bit, a conversion of google/gemma-4-E2B:

• The audio and vision towers are removed. The model is text-only.
• Both embedding tables are 4-bit (group size 64): the shared input/output table and the much larger per-layer table. Pretype's loader already re-quantized them to 4-bit in memory, so these are the weights it ran. A table is only a lookup, so this costs no speed.
• All other weights are unchanged.

The weights are 3.58 GB instead of 5.90 GB (−39%). In Pretype's evals the outputs match the original (details). The license is the upstream one (Apache-2.0).

Made for Pretype, on-device autocomplete for macOS. It is the completion model of Pretype's Gemma E2B 8-bit tier, the default on 24 GB Macs.

mlx-lm 0.31.3 rejects the unused key/value projections of the KV-shared layers in strict mode, so load without it:

from huggingface_hub import snapshot_download
from mlx_lm import generate
from mlx_lm.utils import load_model, load_tokenizer
from pathlib import Path
path = Path(snapshot_download("nikimori/gemma-4-e2b-8bit-slim"))
model, _ = load_model(path, strict=False)
tokenizer = load_tokenizer(path)
print(generate(model, tokenizer, prompt="The quick brown fox", max_tokens=20))

Pretype loads it with mlx-swift-lm.