jfan/gemma-4-e4b-web-mtp-vision-litert-lm

Gemma 4 E4B — WebGPU Optimized with Neural Multi-Token Prediction (MTP) & Vision

Web-optimized gemma-4-E4B-it LiteRT-LM bundle for browser (WebGPU) execution, fused with authentic trained neural Multi-Token Prediction (MTP) drafter (Sec 11 + Sec 21, 45.13 MB) and multimodal vision encoder, powered by jfan/litert-lm-web-runtime (@litert-lm/core@0.17.8-dev).

This release delivers 42-layer Gemma 4 E4B with Grouped Query Attention (GQA), zero-copy shared KV cache in VRAM, and 100% client-side WebGPU acceleration.

---

⚡ Verified Hardware Performance (WebGPU)

Benchmarked on physical NVIDIA GPU WebGPU hardware runner (chrome-dev-linux-hardware-gpu-webgpu):

Metric	Standard Decode (1-tok)	Neural MTP Decode (γ=2)	Improvement / Value
Decode Throughput	28.5 – 33.2 tok/s	36.8 – 44.5 tok/s	1.25x – 1.35x speedup
Summarize / Coding Speedup	31.0 tok/s	41.2 – 43.5 tok/s	1.33x – 1.40x speedup
Draft Steps	—	γ = 2	Optimal latency/throughput balance
Multimodal Vision Prefill	2,400 – 3,100 tok/s	2,400 – 3,100 tok/s	Direct WebGPU texture upload (~560 vision tokens)
Time-to-First-Token (TTFT)	0.16 – 0.19 s	0.16 – 0.19 s	Responsive multimodal interaction
Output Equivalence	Baseline	Exact match	100% token-for-token greedy equivalence

---

📦 Model Architecture & Fused Sections

Container format: gemma-4-E4B-it-web-mtp-vision.litertlm (3.25 GB, exactly 3,246,253,760 bytes)

1. LlmMetadata (12.2 KB): Embedded Gemma 4 tokenizer configs, multimodal vision markers (<|image>, <image|>, patch 16x16, max 2520 patches), thought channel, and jinja chat templates.
2. SP_Tokenizer (4.69 MB): SentencePiece tokenizer for fast client-side tokenization.
3. tf_lite_artisan_text_decoder (2.96 GB, WebGPU Artisan):
   • 42 transformer layers, hidden dimension $d = 2560$, intermediate dimension 10240.
   • Grouped Query Attention (GQA) with number_of_kv_heads = 2 and 16 query heads.
   • Dynamic zero-copy local sliding-window (layer 22, 1024 context) and global full-context (layer 23) ringbuffers.
   • Zero host memory PLE duplication.
4. tf_lite_vision_encoder (224.1 MB, fp16):
   • High-throughput SigLIP image encoder delegated to WebGPU with on-the-fly dequantization.
5. tf_lite_vision_adapter (7.89 MB, CPU):
   • Projection adapter mapping image tokens to model hidden space ($d = 2560$).
6. tf_lite_end_of_vision (10.9 KB):
   • Multimodal boundary delimiter token embeddings.
7. tf_lite_mtp_drafter (45.13 MB, LiteRT CPU / WebGPU Native eevee):
   • Authentic trained Gemma 4 E4B MTP drafter combining Section 11 (3.84 MB graph) and Section 21 (41.29 MB externalized weights).
   • 4 transformer layers cross-attending over shared main model KV cache layers 22 & 23 directly in VRAM (0 bytes extra allocation).
   • Full 262,144-vocabulary INT4 LM projection head (embedder.decode).

---

🚀 Quickstart Guide

1. Install Runtime Package

Download the @litert-lm/core@0.17.8-dev tarball from jfan/litert-lm-web-runtime:

npm install https://huggingface.co/jfan/litert-lm-web-runtime/resolve/main/litert-lm-core-0.17.8-dev.tgz

Or add to your package.json:

{
  "dependencies": {
    "@litert-lm/core": "https://huggingface.co/jfan/litert-lm-web-runtime/resolve/main/litert-lm-core-0.17.8-dev.tgz"
  }
}

2. Browser Usage: Multimodal Vision + MTP Speculative Decoding

import {
  loadEngine,
  Backend,
  EngineSettings,
  SessionConfig,
  Message,
} from '@litert-lm/core';

// 1. Initialize Engine with WebGPU Artisan backend and Hosted WASM
const engine = await loadEngine({
  wasmPath: 'https://huggingface.co/jfan/litert-lm-web-runtime/resolve/main/wasm/',
  backend: Backend.GPU_ARTISAN,
  modelUrl: 'https://huggingface.co/jfan/gemma-4-e4b-web-mtp-vision-litert-lm/resolve/main/gemma-4-E4B-it-web-mtp-vision.litertlm',
});

// 2. Configure conversation with MTP speculative decoding enabled
const conversation = await engine.createConversation({
  sessionConfig: {
    maxOutputTokens: 512,
    temperature: 0.0, // Greedy sampling for maximum MTP acceptance
    enableSpeculativeDecoding: true,
  },
});

// 3. Send prompt with an image (Blob, Uint8Array, or Base64)
const imageBlob = await fetch('https://images.unsplash.com/photo-1546069901-ba9599a7e63c').then(r => r.blob());

const response: Message = await conversation.sendMessage({
  content: [
    { type: 'image', data: imageBlob },
    { type: 'text', text: 'Examine this image and write a detailed analysis of what is shown.' },
  ],
});

console.log(response.content);

// 4. Telemetry
const bench = await conversation.getBenchmarkInfo();
console.log(`Prefill: ${bench.lastPrefillTokensPerSecond.toFixed(1)} tok/s`);
console.log(`Decode:  ${bench.lastDecodeTokensPerSecond.toFixed(1)} tok/s`);

---

⚖️ Model Family Comparison

Spec	Gemma 4 E2B	Gemma 4 E4B
Model Size	2.28 GB	3.25 GB
Layers	35	42
Model Dim ($d$)	1536	2560
Attention Type	Multi-Query (1 KV head)	Grouped-Query (2 KV heads)
Shared KV Cache	Layers 13 & 14	Layers 22 & 23
MTP Drafter Size	44.33 MB	45.13 MB
HuggingFace Repo	jfan/gemma-4-e2b-web-mtp-vision-litert-lm	This repository
Web Runtime	jfan/litert-lm-web-runtime	jfan/litert-lm-web-runtime