Gemma 4 E2B (it) for Apple Core AI — 8K context, vision
A conversion of google/gemma-4-E2B-it to Apple Core AI, made with
apple/coreai-models (commit d3382a5, iOS static-shape flow, models/gemma4/export.py).
The original model is by Google and is released under the Apache License 2.0; this conversion keeps that license.
Changes from the original
- Weights converted to Core AI (
.aimodel) and compressed with 4-bit k-means palettization (embeddings 8-bit) - Per-layer embeddings stored as a separate
safetensorssidecar (memory-mapped at runtime) - Static-shape context ladder: 1024 and 8192 tokens (decode tile 8, prefill tile 64)
- Vision tower exported at 384 px (64 soft tokens) and 768 px (256 soft tokens)
The bundle is not pre-specialized: specialization for the Neural Engine happens on the device at first load.
Bundle contents
| Path | Purpose |
|---|---|
gemma4-e2b-8k.aimodel/ |
Language model |
gemma4-e2b-8k_ple.safetensors |
Per-layer embeddings (2.3 GB) |
vision.aimodel/ |
Vision encoder |
tokenizer/ |
Tokenizer and chat template |
metadata.json |
Bundle description (kind: vlm) |
Total size: about 3.8 GB.
Usage
Requires macOS / iOS 27 or later with Core AI. With the Swift tools from apple/coreai-models:
swift run -c release llm-runner --model /path/to/gemma4-e2b-8k --prompt "Hello" --sampling-strategy greedy
swift run -c release llm-runner --model /path/to/gemma4-e2b-8k --image photo.jpg --prompt "Describe the image."
What was checked
Checked on a MacBook with Apple M1 and 16 GB RAM only:
- Text generation works (answers in Polish and English were coherent) and image input is processed
- First load (specialization) took about 3 minutes; later loads from the cache took about 1 second
- 7000-token prompt: roughly 1100–1400 tokens/s prefill; decoding roughly 26 tokens/s
- Peak memory footprint of the process about 745 MB
Not checked: iPhone / iPad, background execution, answer quality compared to the original full-precision model (one test image was described correctly in general but with a wrong car make, so image quality is not benchmarked). Quantization may reduce quality relative to the original.