jorch/gemma-4-e2b-it-lm-4bit

Gemma 4 E2B — Text-Only MLX 4-bit

Text-only extraction of Google's Gemma 4 E2B instruction-tuned model, converted to MLX format with 4-bit quantization.

This is the language model component only (no vision/audio towers), suitable for text chat on Apple Silicon devices including iPhones (8 GB RAM).

Model Details

Spec	Value
Base model	google/gemma-4-e2b-it
Architecture	Gemma 4 with Per-Layer Embeddings (PLE)
Parameters	5.1B total, 2.3B active
Hidden size	1536
Layers	35
Attention heads	8 (1 KV head)
Vocab size	262,144
Context window	128K tokens
Quantization	4-bit affine (group size 64)
Download size	2.5 GB
License	Apache 2.0

Conversion

Converted from google/gemma-4-e2b-it using mlx-lm with Gemma 4 support from ml-explore/mlx-lm PR #1095:

python -m mlx_lm convert --hf-path google/gemma-4-e2b-it -q --q-bits 4 --mlx-path ./gemma-4-e2b-it-lm-4bit

Usage with mlx-lm (Python)

from mlx_lm import load, generate

model, tokenizer = load("jorch/gemma-4-e2b-it-lm-4bit")
response = generate(model, tokenizer, prompt="What is machine learning?", verbose=True, max_tokens=512)

Usage with mlx-swift-lm (iOS/macOS)

Requires Gemma 4 text model support in mlx-swift-lm:

import MLXLLM
import MLXLMCommon

let configuration = ModelConfiguration(id: "jorch/gemma-4-e2b-it-lm-4bit")
let container = try await LLMModelFactory.shared.loadContainer(configuration: configuration)

Why text-only?

The mlx-community/gemma-4-e2b-it-4bit model was converted with mlx-vlm and includes vision/audio towers. That conversion is missing critical text model weights (norm, per_layer_projection_norm, per_layer_model_projection), making it incompatible with text-only LLM loaders.

This conversion uses mlx-lm which properly extracts and preserves all 1,338 text model weight keys.

Built for SoloAI

This model powers SoloAI — a privacy-first iOS chat app that runs Gemma 4 entirely on-device.