someone15/gemma4-E4B-CoreML

Gemma 4 E4B — Core ML (INT4 Apple Silicon / ANE)

This is a Core ML format model of google/gemma-4-E4B-it, optimized for the Neural Engine environment.

It splits Gemma 4's Per-Layer Embedding (PLE) and 42-layer decoder into chunks. This reduces memory usage and maximizes on-device inference performance.

---

Quick Start

With the CoreML-LLM library, this model can be downloaded and run with streaming inference in a single line.

1. Add the package

dependencies: [
    .package(url: "https://github.com/john-rocky/CoreML-LLM", branch: "main")
],
targets: [
    .target(
        name: "YourApp",
        dependencies: [
            .product(name: "CoreMLLLM", package: "CoreML-LLM")
        ]
    )
]

2. Load the model and run inference

import CoreMLLLM

// Download and load the model
let llm = try await CoreMLLLM.load(repo: "someone15/gemma-4-E4B-coreml")

// Generate text
let messages = [
    CoreMLLLM.Message(role: .user, content: "Hello! Tell me about the Gemma 4 model.")
]

let stream = try await llm.generate(messages, maxTokens: 512)
for await chunk in stream {
    print(chunk, terminator: "")
}

---

Python / Standalone Usage

You can use coremltools and huggingface_hub to download the chunk models locally and verify them.

from huggingface_hub import snapshot_download
import coremltools as ct
import json

local_dir = snapshot_download(
    repo_id="someone15/gemma-4-E4B-coreml",
    allow_patterns=[
        "chunk*.mlmodelc/*",
        "prefill_chunk*.mlmodelc/*",
        "embed_tokens*.bin",
        "per_layer_*.bin",
        "cos_*.npy",
        "sin_*.npy",
        "model_config.json",
        "hf_model/*"
    ]
)

config = json.load(open(f"{local_dir}/model_config.json"))

# Load the 4 decode chunks
chunks = [
    ct.models.MLModel(f"{local_dir}/chunk1.mlmodelc"),
    ct.models.MLModel(f"{local_dir}/chunk2.mlmodelc"),
    ct.models.MLModel(f"{local_dir}/chunk3.mlmodelc"),
    ct.models.MLModel(f"{local_dir}/chunk4.mlmodelc"),
]

---

Repository Structure

File / Folder	Size	Role
chunk{1..4}.mlmodelc/	INT4 Palettized	Token decoder split into 4 stages (Generation / Decode)
prefill_chunk{1..4}.mlmodelc/	INT4 Palettized	Prefill chunks to speed up encoding of the initial prompt
embed_tokens_q8.bin	INT8 (~400MB)	Main input token embeddings (Vocab 262,144)
embed_tokens_scales.bin	Float Scales	Quantization scales for the embeddings
embed_tokens_per_layer_q8.bin	INT8 PLE (~2-4GB)	Per-Layer Embeddings (PLE) binary
embed_tokens_per_layer_scales.bin	Float Scales	Quantization scales for the PLE
per_layer_projection.bin	Projection Weight	Per-layer projection weights
per_layer_norm_weight.bin	Normalization Weight	Per-layer normalization weights
cos_{full,sliding}.npy / sin_{...}	Precomputed RoPE	Precomputed RoPE tables for sliding-window and full attention
model_config.json	JSON	Runtime inference settings (number of layers, attention heads, scale parameters, etc.)
hf_model/	Config & Tokenizer	Standard Hugging Face tokenizer and architecture config

---

Model Specs

Parameter	Value
Base Model	google/gemma-4-E4B-it
Layers (num_hidden_layers)	42
Hidden Size	2560
Intermediate Size	10240
Attention Heads (Q / KV)	8 heads / 2 heads (GQA)
Head Dim / Global Head Dim	256 / 512
Context Length	2048
Sliding Window Size	512
Vocab Size	262,144
Quantization	INT4 (Weights) / INT8 (Embeddings)
Target Hardware	Apple Silicon (M-series / A-series), Compute Units: CPU_AND_NE

---

Recommended Specs

• RAM: An Apple Silicon Mac with 16GB or more of unified memory is recommended
• OS: macOS 15.0+ Sequoia

---

License

This model follows the license of Google's original model, and the Gemma Terms of Use apply.