ginigen-ai/Edge-4B-TELL

GINIGEN Edge 4B · TELL

Edge series — domain-specific models that run entirely on the device, with a calibration signal that tells you when the model is on shaky ground.

This repository contains two things:

1. Inference weights — a verbatim mirror of google/gemma-4-E4B-it-qat-q4_0-gguf. Unmodified. All model credit belongs to Google DeepMind.
2. tell_probe.f32 — a linear readout we fitted on these exact weights, which estimates whether an answer is likely to be correct from the model's own last-layer hidden state. This is the GINIGEN contribution here, and it is running on a phone today.

These are the exact weights that ship with HeliGO, an Android app that keeps working when the network does not.

File	Size	Purpose
gemma-4-E4B_q4_0-it.gguf	4.8 GB	Text generation
gemma-4-E4B-it-mmproj.gguf	0.92 GB	Vision + audio projector
tell_probe.f32	10 KB	GINIGEN TELL readout — float32 little-endian, first value is the intercept, the rest are the 2560 weights

We host our own copy of the weights so that a disaster-response app does not depend on a repository we do not control: if an upstream path moves, every installed copy of the app stops being able to fetch its model.

---

GINIGEN TELL — a confidence signal that is not the model's opinion

Asking the model how sure it is does not work — it is worse than a coin flip. Prompted to state its confidence alongside each answer, this checkpoint reported a mean of 0.863 across 665 Korean disaster-procedure questions. Ranking answers by that number gives AUROC 0.441: below 0.500, meaning the model tends to sound more certain on the answers it gets wrong. Self-reported confidence is not a weak signal here. It points the wrong way.

TELL reads the model's last-layer hidden state instead and scores it with a linear readout fitted on these weights. Measured on 665 Korean disaster-procedure questions (answer accuracy 27.5%), stratified 5-fold cross-validation repeated 10 times — held-out scores only:

Signal	AUROC
Self-reported confidence (mean 0.863)	0.441 — worse than chance
Surface features (answer length, digit count, formatting)	0.736 ± 0.007
GINIGEN TELL (hidden-state readout)	0.759 ± 0.006

The honest margin: +0.023 ± 0.009 over the surface baseline (2.6σ). Real, but small. On this question set, answer length and formatting alone already carry most of the separable signal, and the hidden state adds a modest amount on top. We publish the surface baseline precisely because without it "the hidden state carries the signal" would be an unfalsifiable claim — and here it shows the readout clearing a strong baseline by a narrow margin rather than a wide one.

What TELL is compared against in the shipped app is not that baseline, though: it is nothing. The app has no length heuristic. The choice there is 0.759 or no calibration signal at all.

A readout is specific to the model it was fitted on. Hidden-state geometry differs between checkpoints — Gemma-4-12B exposes 3840 dimensions where this E4B build exposes 2560 — so a readout does not transfer. Loading a readout of the wrong width must disable the signal, not reshape it; a score computed from a mismatched vector looks like a number and means nothing.

What it costs on a phone. TELL generates zero tokens — it re-reads a forward pass rather than writing new text. That is not the same as free: on a Galaxy S25 the read took 3.8 s measured end-to-end, because the prompt and answer must be prefilled again. Cheap against a second generation pass, not instant.

Pick the threshold from the data, not from 0.5. A readout fitted where the model answers 27.5% of questions correctly puts most of its mass low; 0.5 is not "an even split," it is a cutoff that sends 83.5% of answers to the fallback. An alarm that is always on is not an alarm. Chosen from the held-out score distribution:

Cutoff	Routed to source	Wrong answers caught	Correct answers routed away
0.15	48.4%	282 / 482	40 / 183
0.25	60.9%	348 / 482	57 / 183
0.50	83.5%	444 / 482	111 / 183

The app ships 0.25: it catches 72% of wrong answers at the cost of routing 31% of correct ones to the official text instead. That trade is acceptable only because of where the answer lands — the Ministry's own guidance, not silence. A different readout needs this re-chosen; score distributions are not comparable across probes.

What TELL is not. It reads "did this model wobble", not "what is the right answer." It routes answers to a more conservative path; it never authorizes one. At 0.759 it is a ranking signal, not a verdict, and the hard rules above it do not depend on it.

Readouts are published as plain weight vectors. Training and serving code is not part of this repository.

---

Why this model, on a phone

Edge AI is not "a small model." It is a set of constraints that must hold at once: the weights fit in phone RAM, inference is fast enough to matter during an emergency, and nothing depends on a network that is already gone.

We selected this checkpoint after publishing an evaluation of 330 of the 426 models available on OpenRouter across seven Korean-language criteria — only 7.6% reached grade A (leaderboard). Quantization-aware training is what makes the Q4_0 build usable here: it keeps quality close to bfloat16 while cutting the memory needed to load the model.

Measured on a real device (Galaxy S25)

Network required for inference	0
Resident memory with model loaded	~3.6 GB
Runtime binary shipped in the APK	12.6 MB (llama.cpp server, symbols stripped)
ARM acceleration	dot-product and matrix instructions enabled (sdot 898, smmla 244 in the shipped binary)
First download	4.8 GB + 0.92 GB, once, Wi-Fi recommended

Numbers we have not measured are not published. Tokens per second depends on thermal state and concurrent load, and we do not have enough repeated runs to quote a figure with an error bar.

---

How it runs inside the app

HeliGO starts a llama.cpp server inside the phone, bound to loopback with a random port and a random API key, and talks to it over HTTP. There is no remote endpoint. The shared object is packaged with useLegacyPackaging so Android extracts it to disk, where it can be executed.

--host 127.0.0.1 --port <free> --api-key <random>
-c 4096 -t <cores/2> -ngl 0 --jinja --no-warmup
--embeddings --pooling last --embd-normalize -1
--mmproj gemma-4-E4B-it-mmproj.gguf

The embedding flags are what expose the hidden state that TELL reads. They must match the flags the readout was fitted under exactly — a different pooling or normalization yields a vector the readout was never trained on, and the resulting score would be meaningless rather than merely noisy.

A warm-up generation runs once after load: the first answer took 127 s cold versus 11 s warm, because the weights have to be paged in.

One thing we tried and removed. Pre-reading the whole model file into page cache to speed up the first answer caused Android's low-memory killer to terminate about twenty other apps and froze the phone. Page-cache preheating is only safe when free RAM clearly exceeds the file size — on a phone it does not.

---

Safety layering

A model that is confidently wrong about a mushroom can kill someone. The app does not let the model speak freely where a wrong answer causes harm. Three layers, in order:

1. Hard rule — the app never outputs "safe to eat," regardless of any confidence score.
2. Fall back to the source — high-stakes questions are answered from the original text of Korea's Ministry of the Interior and Safety (75 official guidance documents shipped in the app).
3. TELL — when a hidden-state readout is present and its score is low, the answer is routed down to layer 2 rather than shown; if neither layer can answer, the app says nothing. Layers 1 and 2 stand on their own and do not depend on a readout being present.

The asymmetry is deliberate: the app will never say something is safe to eat, but it will always warn that something is toxic. The cost of silence is not the same on both sides.

Distances, elevations, bearings, sunset times and coordinates are not produced by the model. They are computed. A model states wrong numbers with confidence; arithmetic does not.

---

Intended use

Offline assistance during disaster and distress: identifying plants, wounds and hazards from a photo, explaining emergency procedures, and answering questions with no connectivity.

Not intended as a substitute for emergency services. HeliGO tells the user, in the app and in its privacy policy, to contact emergency services first whenever a connection is available.

Limitations

• Quantized to Q4_0; quality is below the bfloat16 checkpoint.
• Korean-first. Other languages inherit the base model's behaviour and are not evaluated by us.
• TELL is fitted on 665 Korean disaster-procedure questions and is not validated outside that domain. The readout has 2560 dimensions against 665 samples, so the held-out AUROC is the number to trust, never an in-sample one — an in-sample fit at this ratio reaches 1.000 and means nothing.
• TELL's margin over a plain answer-length baseline is 0.023. Treat it as a useful ordering of answers, not as a detector.
• The 665 questions skew hard: the model answered 27.5% of them correctly. A readout fitted where most answers are wrong may behave differently on an easier distribution.
• Vision and audio run through the projector file; both must be present for multimodal input.
• The model knows nothing about the map, terrain or shelter data — those are separate datasets bundled with the app, not knowledge inside the weights.

License

Apache 2.0, inherited from the upstream checkpoint. See the Gemma 4 license. We redistribute the weight files unmodified; all model credit belongs to Google DeepMind. tell_probe.f32 is GINIGEN AI's own work and is released under the same terms.

Citation

Weights: Google DeepMind, Gemma 4 — technical report. TELL calibration readout, edge deployment and safety layering: GINIGEN AI.

---

GINIGEN AI — domain-specific edge AI. ginigen-ai-edge.static.hf.space