Gemma4-E2B-Dflash2-Android

English | 한국어

Experimental image bridge tested on S22. Two synthetic color images returned Red/Blue through CPU mmproj encoding and NPU verification. Target-only and DFlash2 output matched; three text regressions had identical tokens and zero native U16 logit error. This is limited validation, not broad vision certification. See vision-device-check.json for the new measurements; historical text speeds below are separate records.

This package prepares Google Gemma 4 E2B instruction-tuned assets and a locally trained Z Lab DFlash2-derived drafter for Android CPU drafting with Qualcomm NPU block verification. It is designed for the app-independent RaInfer engine, not a particular application. No app, APK, proprietary SDK or QNN runtime libraries are included.

Start here: engine and required runtime files

Engine Git repository: LEPTONNW/RaInfer. Build its Android runner using the build guide. The complete download/startup checklist is also available in the engine repository. This package provides data for that engine.

Download all eight files below into npu/, keeping filenames unchanged:

Component	File	Role
Gemma target	gemma4-e2b-dflash2-vision-verify-2k.pte	NPU verifier, about 3.36 GB; also accepts ordinary text requests
Target contract	io-contract.vision.json	Matching target IO, quantization and image contract
Draft body	cpu-body-fp32.bin	FP32 CPU draft computation
Draft embedding	embedding-cpu-fp32.pte	CPU token embeddings
Draft output head	head-cpu-int8.bin	CPU INT8 vocabulary projection
Draft selector	selector-cpu-fp32.bin	Candidate selection
Draft vocabulary	active-vocabulary.u32	Active draft vocabulary
Draft manifest	draft-manifest.json	Draft filenames, architecture and hashes

The six draft files form one runtime bundle; weights are not embedded inside RaInfer or the Gemma target. The embedding .pte executes on CPU, whereas the Gemma target .pte executes on NPU. The target uses mixed W4/W8 weights.

Also obtain tokenizer/tokenizer.json, tokenizer/tokenizer_config.json, tokenizer/chat_template.jinja and tokenizer/config.json for your host's prompt formatting and token encoding/decoding. With the Hugging Face CLI:

hf download RoundRobin237/Gemma4-E2B-Dflash2-Android --include "npu/*" --include "tokenizer/*" --local-dir ./model-assets

This fetches the approximately 4 GB runtime bundle, including additional weightless tokenizer GGUFs. It excludes BF16 and the full Q4 GGUF, which are optional training/conversion/reference formats. DFlash2 is a separate six-file CPU bundle, not embedded in the Gemma PTE or engine. N-gram needs no extra model.

After building the runner and preparing the matching QNN/ExecuTorch runtime on Android, run from model-assets/ with the executable on PATH:

mobile_runner --serve npu/gemma4-e2b-dflash2-vision-verify-2k.pte npu/cpu-body-fp32.bin npu/draft-manifest.json npu/io-contract.vision.json

Your app waits for ready and sends templated token IDs through stdin JSONL. See the protocol and app integration. For images only, add tokenizer/gemma4-e2b-vision-vocab.gguf, the separately built CPU image encoder and the external mmproj described in external-assets.json / the image guide. The CLI command above already includes the weightless vision vocabulary. Text-only generation uses the same target PTE without mmproj. Other Qualcomm architectures require their own validation; the reference target is SM8450/v69, QAIRT 2.37.0.250724 and fixed 2K context. No application/APK or SDK/runtime is bundled here.

Choose the files for your backend

Directory	Contents / purpose
original-bf16/	Unmodified Google BF16 safetensors and configuration, including original multimodal weights
q4-gguf/	Unmodified Unsloth Q4_K_M GGUF, for a compatible llama.cpp backend/reference
npu/	Experimental vision-enabled mixed W4/W8 QNN/ExecuTorch target PTE; CPU DFlash2 body, embedding, output head, selector, vocabulary and manifests
external-assets.json	Upstream F16 mmproj URL/hash; mmproj is not redistributed here
tokenizer/	Matching tokenizer, conversation template and weightless GGUF metadata
Root JSON	File hashes, source lineage, quantization, hardware and measured historical benchmarks

These are different formats, not three interchangeable runtime files. The Q4 GGUF is not the NPU verifier. The CPU drafter is tied to the compiled target's feature taps and export contract; copying it beside an arbitrary GGUF does not enable DFlash2. Download the directories needed for your backend rather than assuming all formats must remain resident simultaneously.

Google names this variant E2B. The original multimodal safetensors contain 5,123,178,979 stored parameters across 2,011 BF16 tensors, not simply two billion stored values. The source includes audio and vision; this bridge supports one static image, not audio/video. The NPU target remains a fixed 2,048-position decoder. Images consume part of that budget.

Actual text measurements on Galaxy S22 Ultra

These records are from 2026-10-15 and precede public source packaging and the vision bridge. They use the existing text target. They are not new measurements of the published source revision or promises of sustained performance.

Historical prompt	N-gram	Output tokens	Accepted / proposed	Generation t/s
Small Java example	off	72	42 / 90	19.0963
Korean meeting-note request	off	52	14 / 114	10.7347
English meeting-note request	off	49	3 / 138	8.6995
Same Java prompt, another run	on	72	42 / 90	19.9379

The 19.9379 t/s sample is the maximum among the 14 retained records; it is not the cross-prompt average. Identical accepted/proposed counts do not establish that n-grams caused that timing difference. A later session's corresponding Java sample was 15.6789 t/s. Temperature, foreground scheduling, heat and run conditions matter. The 25–30 t/s goal has not been achieved. See benchmark.json for all 14 measured records, exact timing and acceptance definitions.

The app-level historical metric was completion tokens / generation duration, excluding prefill and model loading. In the native runner, decode_tokens_per_second excludes the first prefill-produced token from its numerator. Do not compare those counters without aligning their definitions. The first Java sample had 70 prompt tokens, 2.244620259 s prefill and 3.770364686 s generation. These are small single-request examples, not a large-context throughput benchmark.

Tested device and selected text settings

• Galaxy S22 Ultra SM-S908N, Snapdragon 8 Gen 1 SM8450, marketed 12 GB RAM.
• CPU: 4 Cortex-A510 efficiency cores, 3 Cortex-A710 performance cores and 1 Cortex-X2 prime core. The engine requests performance-cluster execution. Android cpuset rules may limit which cores are actually available.
• GPU: Adreno 730. It is present but not used in the selected text baseline; experimental GPU draft-head/offload paths did not improve the selected result.
• NPU: Hexagon HTP v69, executing QNN graphs through ExecuTorch Qualcomm backend.
• Android 16 / API 36; actual tested page size 4 KiB.
• QAIRT/QNN 2.37.0.250724, Android NDK 29.0.14206865, ExecuTorch ca56e1ffdba1c902d9202021d7297982d3501439.

Selected text settings: greedy / temperature 0, target context 2K, verification width 4 (one pending token plus three draft candidates), CPU output-head workers 4, CPU body workers 1, fast argmax and compact logits enabled, n-gram enabled, GPU head off, burst vote off, fixed per-worker affinity off, profiling off. The 19.0963 t/s reference sample used n-gram off. All device clocks and thermal conditions were not logged for every historical sample; no advertised TOPS value is used to infer a measured throughput.

Quantization and drafter provenance

The NPU target is not pure 4-bit: mixed block-32 W4 FFN weights, selected W8 projections/tail paths, 16-bit activation boundaries and KV state. See quantization.json for the retained conversion recipe. Twelve QNN delegates are used. Sensitive paths were retained at higher precision during earlier quality work. Original BF16 files and the separately sourced Q4_K_M GGUF remain unmodified.

The locally trained drafter has hidden size 384, 3 layers, intermediate size 1,536, 512-token context, four feature taps [3,13,23,33], vocabulary 262,144, and a three-candidate/AR4 runtime. Selected checkpoint step: 1,100. Its CPU body uses FP32, the output head uses per-row INT8 weights with dynamic per-query INT8 activations, and the selector uses a native FP32 path. It is not an official Google MTP or an official Z Lab checkpoint. No claim of compatibility with arbitrary public DFlash/DFlash2 weights is made.

Z Lab attribution: algorithmic components and grouped dynamic-convolution / selection implementation are derived from z-lab/dflash, revision 07ebd93db9f472af339b644bb70221ad8428328a (MIT, Copyright 2026 Z Lab). RaInfer's target verification, transaction/cache management and mobile wiring are local implementations. Training data and intermediate checkpoints are not included. dflash2-lineage.json and provenance.json retain the source record.

Using the files with another app

Follow RaInfer's build guide, process protocol, and integration guide. An app owns tokenization, local file installation, foreground execution, deadlines, process lifetime and UI. No specific app is required.

The image path requires the vision-enabled runtime, matching experimental PTE and IO contract, CPU mmproj encoder and weightless tokenizer metadata. Read the image guide before using it. Encode the formatted image/prompt into a local bundle, then send the bundle to the persistent worker via JSONL. Ordinary text requests remain token-ID based. The mmproj is F16, not a QNN graph. Image encoding, target prefill and generation must be timed separately.

The SDK/runtime are not bundled. Obtain compatible dependencies under their applicable licenses. A different Qualcomm SoC, HTP architecture, SDK, OS or page size may need a separate conversion/build. This is not an all-Android binary.

After building RaInfer and arranging the matching licensed runtime, start the worker with package-relative asset paths:

mobile_runner --serve npu/gemma4-e2b-dflash2-vision-verify-2k.pte \
  npu/cpu-body-fp32.bin npu/draft-manifest.json npu/io-contract.vision.json

The mmproj is an external dependency, not our own conversion. Google provides the original vision weights, and the exact tested GGUF is Unsloth's mmproj-F16.gguf. Download it there and save it locally as external/mmproj-F16.gguf; verify SHA256 140be8d7849741f88c50757d529b84373ee8e27052cc2236855b537f4a8215fa. The URL, pinned revision and size are also in external-assets.json.

For images, use RaInfer's local-template prompt tool, then encode on the same device that can access the files:

python tools/make_vision_prompt.py --tokenizer-dir tokenizer \
  --prompt "What is the dominant color in the image? Answer with one color word." \
  --output image-prompt.txt
rainfer_vision_encoder --text-model tokenizer/gemma4-e2b-vision-vocab.gguf \
  --mmproj external/mmproj-F16.gguf --image input.png --prompt-file image-prompt.txt \
  --output session/input.bundle.json --threads 4 --image-max-tokens 256

Wait for the worker's ready event, then send one JSON line to its stdin:

{"id":1,"vision_bundle":"/absolute/path/session/input.bundle.json","max_new_tokens":64,"cpu_workers":4,"body_workers":1,"fast_argmax":true,"compact_logits":true,"ngram":true,"gpu_head":false}

Output events start with @DFLASH2 . Read report.tokens and detokenize using the matching tokenizer. For text-only requests, pass a correctly templated tokens array instead of vision_bundle. Do not send both. The Python prompt tool is a PC-side preparation option; a phone app can apply the same template locally without Python. None of the commands downloads SDKs automatically.

Licensing, changes and limits

Distributed model weights are under Apache-2.0; the Google Gemma 4 source is Apache-2.0, rather than the earlier Gemma-specific license. Google retains its copyright. The unmodified Q4 GGUF comes from Unsloth's Apache-2.0 repository; mmproj is linked upstream rather than redistributed here. Conversion, mixed quantization, compiled graph packaging, feature outputs, vision operator alias and local drafter training are modifications by the package author, not an official Google/Qualcomm release.

RaInfer and llama.cpp/mtmd code are MIT; Z Lab DFlash code is MIT; ExecuTorch is BSD-3-Clause. Their notices are separate and preserved in NOTICE / licenses/. Apache-2.0 for weights does not relicense SDKs or linked engine dependencies. No endorsement is implied. Current checks cover a few examples and greedy regressions; they do not certify general answer quality, code correctness, safety, multilingual competence or universal device compatibility.

New image-bridge device check

On the same S22, four workers were requested but observed CPU masks varied: 4–5,7 around target-only runs, 4,6–7 before the persistent run and 4–7 after it. Thermal status before the persistent run was 2; these are state snapshots, not continuously recorded clocks or core availability. With 49 prompt tokens and a 145-token Java answer, original target-only / vision-target-only / vision-target+DFlash2 measured 6.9460 / 6.9749 / 15.5555 native decode t/s, respectively. DFlash2 accepted 83/183 candidates (45.36%). Output tokens matched; the code compiled and printed 12. Native decode counts 144 post-prefill tokens here. These settings/prompts and metric differ from the historical app-level records.

For 224x224 red/blue images, CPU encoding took 9.4474 / 9.4604 s, encoder loading 3.2702 / 3.1627 s, and target-only NPU prefill 3.4431 / 3.4776 s. Both images used 64 projected rows in a 95-token prompt. Image answers were only one color word plus EOS; 0/3 draft candidates were accepted in each single post-prefill round. This tiny output is not a long-answer speed/acceptance test. Image→Java→other-image→first-image repetition passed state isolation checks. No OCR, photographs or complex visual reasoning have been certified.