gemma-4-E2B-5bit

MLX-format, 5-bit quantization of Google's google/gemma-4-E2B base (non-instruction-tuned) checkpoint, converted with mlx_lm.convert for use with MLX and MLX-Swift on Apple Silicon.

This is not an official Google release. "Gemma" is a trademark of Google LLC; this repository is not affiliated with, endorsed by, or sponsored by Google.

What was changed from upstream

Quantized from google/gemma-4-E2B to 5-bit precision (group size 64) using mlx_lm.convert -q --q-bits 5, mlx-lm version 0.31.3. No other modification was made — architecture, tokenizer, and generation config are unchanged from upstream.

License

Distributed under the Apache License 2.0 (see LICENSE in this repository), matching the license Google publishes google/gemma-4-E2B under.

Google additionally applies use restrictions to the Gemma family that are not waived by the Apache License tag and continue to apply to these weights regardless of who redistributes them:

• Gemma Terms of Use
• Gemma Prohibited Use Policy
• Gemma Intended Use Statement

By downloading or using this checkpoint you agree to comply with the Prohibited Use Policy.

See NOTICE for the upstream attribution this repository carries forward.

Usage

pip install -U mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("Animateus/gemma-4-E2B-5bit")

prompt = "The quick brown fox"
response = generate(model, tokenizer, prompt=prompt, verbose=True)

This is a base (non-instruction-tuned) checkpoint — prompt it with raw text to continue, not with chat-formatted instructions.