runanywhere

/

gemma4_e2b_HNPU

like 1

Follow

RunAnywhere, Inc. 7

Adds the Hexagon v75 Gemma-4-E2B (text) bundle under v75/ (per-arch sibling of the existing v79/v81 dirs).

Files (~7.7G):

• gemma4-e2b.json — manifest (family=llm, host-op gemma_generate, registered in the QHexRT runtime)
• gemma4_dec_i8.bin — 35-layer decoder, int8 weights
• gemma4_lmhead_f16.bin, gemma4_embed_f16.bin
• gemma4_ple_norm_f16.bin / gemma4_ple_projw_f16.bin / gemma4_ple_table_f16.bin — per-layer embeddings (PLE) host tables (the Gemma-3n/4 memory-offload feature)
• gemma4_rope_inv_f.bin / gemma4_rope_inv_s.bin — rope inverse-freq tables; tokenizer.json

Params: hidden 1536, 35 layers, vocab 262144, head_dim 256, max_ctx 512, eos 1.
Compile: QAIRT 2.47.0, dsp_arch v75, soc_model 57 (O3/VTCM). int8 decode (weight-only) + fp16 lm-head/embed; PLE + rope tables are host-side.

Status: export + v75 compile complete; host-op gemma_generate is present in the runtime so the bundle is runnable. The v81 sibling was device-validated 5/6 on the standardized answer suite (~12 tok/s, ~3.0G RSS); v75 on-device parity on the 8GB Galaxy S24 is pending (bundle fits: ~3G RSS < 8G).