gemma-4-e2b-tq3-litert
Gemma 4 E2B (it) packaged for VoxSumDroid's TurboQuant TQ3 summarizer
path: LiteRT (TFLite) graphs with a 3-bit packed KV cache consumed by the
fused voxsum.tq3_attention custom ops, for low-RAM (< 4.5 GB) Android
devices. 4096-token context export.
Files
| file | what |
|---|---|
model_tq3_4k.tflite |
int8 transformer, fused TQ3 attention (prefill_128 + decode signatures), cache_len 4096 |
ple_table_int8.bin |
per-layer-embedding table, int8 + per-column scales (PLETBL01 header) |
embedder_quantized.tflite |
token embedder (int8) |
auxiliary.tflite |
rope + mask graphs matched to the 4k export (16k masks produce NaN — never mix) |
tokenizer.bin |
compact Gemma BPE vocab+merges (from tokenizer.json; TQ3TOK01) |
wcache.bin |
pre-packed XNNPACK weight cache. Bound to the exact libLiteRt.so build shipped in VoxSumDroid (com.google.ai.edge.litert 2.1.6 AAR arm64, md5 0563b6bc…). Repack after any lib upgrade. |
assets/ |
TQ3 rotation matrices + Lloyd-Max codebooks (d=256 / d=512, 3-bit) |
Provenance & acceptance
Export + quantization pipeline: vieenrose/LiteRT branch turboquant-tq3
(litert/samples/llm/turboquant/). Phase 2a/3/4 acceptance (teacher-forced
64-step top-1 vs fp32-KV baselines): zh 0.9688 exact match with x86 on
both Pi 4B (glibc) and Boox Tab Mini C (bionic); en 0.9273 on arm vs 0.9455
x86 (near-tie flips, documented dynamic-quant noise). Free-run zh meeting
summaries are coherent and faithful.
Measured on-device (warm, pre-packed cache): load ~1 s, peak anonymous RSS ~120-300 MB end-to-end (model / PLE / weight cache stay evictable file-backed pages: ~7 GB flash, not RAM); prefill ~12-20 tok/s, decode ~1 tok/s on Cortex-A72/A73 at 2 threads.
Consumed by VoxSumDroid ModelManager (dir tq3-litert, revision-pinned,
per-file sha256). Not a general-purpose chat runtime: greedy decode,
Gemma turn format, summarization-tuned integration.
- Downloads last month
- 22