ariacompute/gemma-4-e4b-it_q8

Model Card for Gemma-4-E4B-IT (Aria Quant Bundle, q8)

Model Details

Model Description

Gemma-4-E4B-IT is a ~4-billion-parameter instruction-tuned multimodal language model developed by Google, part of the Gemma 4 family. Its text backbone features hybrid attention (36 of 42 layers sliding-window linear attention + 6 standard full-attention layers), GeGLU activation, Grouped Query Attention (GQA, 2 KV heads for 8 query heads), per-layer input projections, and 128K native context length. Pre-trained on diverse web-scale corpora and aligned via instruction tuning + RLHF. This distribution is provided by Aria Compute as an aria-quant-bundle — a quantized package using Hadamard pre-processing + uniform per-channel 8-bit quantization. Optimized for CPU-only, on-device inference on mobile phones, edge devices, and single-board computers via the Aria Engine runtime. No GPU or cloud connection is required.

• Developed by: Google
• Quantized and distributed by: Aria Compute
• Model type: Dense Transformer decoder-only (multimodal base: image/audio + text inputs, text outputs; this bundle ships the text backbone)
• Language(s): English (primary), Chinese, and 30+ additional languages
• License: Apache 2.0
• Finetuned from model: google/gemma-4-e4b-it

Model Sources

• Original Repository: google-gemma/gemma-4
• Original Paper: Gemma 4 Technical Report
• Finetuned from: google/gemma-4-e4b-it
• Aria Compute Dashboard: ariacompute.com/dashboard/models
• Aria Engine: ariacompute.com

Uses

Direct Use

This quantized bundle is intended for on-device, offline text-generation tasks on resource-constrained hardware, including:

• On-device chat and conversational assistants
• Real-time text completion and basic code snippet generation
• Instruction-following tasks for mobile and IoT applications
• Lightweight text embeddings for on-device retrieval and classification
• Short-form summarization of notifications, messages, and local content
• Local document analysis up to 32K context (chunked)

All inference runs locally on CPU. No data is sent to external servers.

Target Devices

Platform	Runtime Memory	Feasibility
High-end smartphone (8 GB)	~4.0 GB	✅ Recommended
Mid-range smartphone (4–6 GB)	~4.0 GB	⚠️ Tight fit
Budget phone (2–3 GB)	~4.0 GB	❌
Raspberry Pi 5 / SBC (4–8 GB)	~4.0 GB	✅
IoT gateway (1–2 GB)	~4.0 GB	❌
Wearable (1 GB)	~4.0 GB	❌

Memory breakdown (q8, at 4K context): ~3.8 GB quantized model weights (mmap) + ~80 MB KV cache + ~40 MB runtime overhead + ~80 MB per-channel metadata overhead ≈ ~4.0 GB.

Note: KV cache is compact thanks to GQA (2 KV heads for 8 query heads) and hybrid attention — 36 of 42 layers use sliding-window attention (window 512, KV bounded by the window), so only the 6 full-attention layers scale KV with context. Combined with tie-word-embeddings, this keeps 128K context practical on ~4.0 GB-class devices.

Out-of-Scope Use

• Long-form creative writing (>2K tokens per generation)
• Mathematical theorem proving or formal verification
• Full program/application synthesis
• Multimodal input (image/audio encoding pipeline is pending audit for this quantized bundle — text-only in this release)
• Real-time audio/speech processing (use Aria speech models)
• Safety-critical decision systems without human oversight
• Deployment in production when batch inference or GPU acceleration is required (this bundle targets CPU-only, single-prompt inference)
• Tasks requiring factual precision beyond the model's ~4B parameter capacity

How to Get Started with the Model

Download from Aria Compute

Authenticated dashboard users can download the bundle via: https://ariacompute.com/dashboard/models

Quantization Recipe

This bundle uses a per-channel quantization recipe, one of several precision options in the Aria Compute lineup:

Component	Quantization Strategy	Details
Attention Q/K/V/O weights	8-bit	Uniform per-channel codebooks, Hadamard pre-processing
FFN gate/up/down weights	8-bit	Uniform per-channel codebooks, Hadamard pre-processing
RMSNorm weights	FP16	Preserved at full precision
Embedding table	FP16	Preserved at full precision (tie_word_embeddings: true)

• Bundle size: ~3.8 GB (BF16 text backbone: ~8.0 GB)
• Generation quality: Awaiting gen_quant_eval audit. The uniform 8-bit per-channel recipe is the highest-fidelity bundle in the Aria lineup — near-lossless versus FP16. Per-channel codebooks preserve per-output-channel distribution characteristics. Formal quality benchmarks against FP16 and other Aria quant recipes are pending
• Calibration-free: Hadamard pre-processing + per-channel quantization, no task-specific calibration data required
• Other precision options: Also available for Gemma-4-E4B-IT: gemma-4-e4b-it_q326_channel (mixed precision, attn 4-bit + FFN ~3-bit, recommended quality-size trade-off) and gemma-4-e4b-it_q4 (uniform per-channel 4-bit, smallest bundle)

Model Architecture

Gemma-4-E4B-IT's text backbone employs a dense Transformer decoder with GeGLU activation, hybrid attention (sliding-window linear attention + standard full attention), GQA, and per-layer input projections (256-dim layer input → 2,560 hidden):

Parameter	Gemma 4 E4B
Layers	42
Hidden size	2,560
Layer input size	256
FFN intermediate size	10,240
Attention heads (Query)	8
Attention heads (KV)	2 (GQA)
Head dimension	256 (global 512)
Full-attention layers	6
Sliding-window layers	36
Sliding window	512
Activation	GeGLU (gelu_pytorch_tanh)
Position encoding	RoPE (full-attn θ = 1,000,000, partial rotary 0.25; sliding θ = 10,000)
Normalization	RMSNorm (pre-norm)
Vocabulary size	262,144
Max context length	131,072 (128K)

Design highlights (Gemma 4 family):

• Hybrid attention: 36/42 layers use sliding-window attention (window 512); 6/42 layers use standard full softmax attention — dense-attention KV cost is confined to a few layers
• GQA (Grouped Query Attention): 2 KV heads serving 8 query heads — KV Cache memory is 4× smaller than full attention, enabling 128K context on edge devices
• Per-layer input projections: a 256-dim layer-input embedding is projected to the 2,560-dim hidden state at each layer — compact embedding table with shared layer-input processing
• GeGLU activation: GELU with tanh approximation gating, efficient for on-device inference
• RoPE position encoding: 1M base frequency for full-attention layers (partial rotary factor 0.25) with 10K base for sliding-window layers, supporting 128K context
• RMSNorm pre-normalization: Lightweight normalization before each sub-layer
• Final logit softcapping: Output logits capped at ±30.0 for training stability
• Tied embeddings: tie_word_embeddings: true — input and output embeddings share weights, reducing footprint

Bias, Risks, and Limitations

Limitations

• Reasoning depth: Multi-step logical reasoning is limited for a 4B-class model. Verify outputs in high-stakes scenarios; consider larger Gemma 4 variants for reasoning tasks.
• Mathematics: Simple arithmetic may be attempted but is unreliable. Advanced quantitative reasoning is out of scope. Use larger models for mathematical tasks.
• Code generation: Capable of single-line completions and basic snippets; unreliable for multi-line code or structured programs.
• Factual knowledge: Moderate world knowledge due to ~4B parameter scale. Always verify factual claims against authoritative sources. This model is better suited for instruction-following and general text tasks than encyclopedic knowledge retrieval.
• Instruction following: Handles simple single-constraint instructions. Complex multi-constraint prompts may cause degradation, especially at longer contexts.
• Quantization drift: Uniform 8-bit per-channel quantization exhibits minimal generation drift versus FP16 thanks to near-lossless precision. For a smaller bundle, use gemma-4-e4b-it_q326_channel (mixed precision) or gemma-4-e4b-it_q4 (uniform 4-bit).

Bias and Risks

• Bias: As with all large language models trained on web-scale data, Gemma may reflect societal biases present in its training corpus. Evaluate outputs before deployment in sensitive domains (hiring, healthcare, law).
• Toxicity: The instruction-tuned model has been safety-aligned with RLHF. However, no safety filter is exhaustive. Consider an additional output classifier in high-risk environments.
• Hallucination: May generate plausible-sounding but factually incorrect information. Implement output verification for critical applications. Hallucination risk is elevated for smaller models due to limited memorization capacity.
• Dual-use risk: Text-generation capabilities could be misused for spam, disinformation, or impersonation. Deploy responsibly and in accordance with the Apache 2.0 license terms.

Recommendations

Users (both direct and downstream) should be made aware of the above risks, biases, limitations, and constraints of the model. We recommend:

• Adding a lightweight output safety classifier for user-facing deployments
• Verifying factual claims with external knowledge bases
• Not using the model for high-stakes decisions without human review
• Considering larger Gemma 4 variants for tasks requiring stronger reasoning or factual recall (note: E4B is already a mid-size variant)