bpmatt/EmbeddingGemma-300M-Phone-Command-Router-ONNX

EmbeddingGemma-300M Phone Command Router (ONNX, 4-bit)

What it does: turns something a person says to their phone ("wake me at half six", "text mom I'm on my way", "how long is the drive to Boston") into a ranking of 205 phone actions: set an alarm, send a text, start navigation, add to a list, and so on. It is the first stage of a fully on-device voice assistant for Android: this model picks which action, and a small companion model, LFM2.5-350M Phone Command Extractor, fills in the details (times, names, messages).

It is google/embeddinggemma-300m, fine-tuned for this task and exported to ONNX with 4-bit weights (~195 MB). It runs on a phone CPU in about 30-40 ms per request.

How it works

"remind me in 20 minutes to check the oven"
        │  query text:  "task: search result | query: remind me in 20 minutes to check the oven"
        ▼
  EmbeddingGemma (this model)  →  768-d vector  →  cosine vs. 205 tool embeddings (tool_embeddings.f32)
        ▼
  set_timer 0.64 · add_reminder 0.47 · set_screen_timeout 0.43 …
        ▼  the app's keyword rule ("remind me … to …" is a reminder) moves add_reminder to the top

Files

File	What it is
model_q4.onnx + model_q4.onnx_data	the model (inputs input_ids, attention_mask; output sentence_embedding [batch, 768])
tokenizer.json	the Gemma tokenizer (for Python / Hugging Face tokenizers)
gemma_tok.bin	the same tokenizer in a compact format read by the Android app's Java tokenizer
tools.json	the 205 actions (name + description), in the row order of the embeddings
tool_embeddings.f32	205 × 768 little-endian float32, unit-length tool embeddings

Quick start (Python)

import json, numpy as np, onnxruntime as ort
from huggingface_hub import snapshot_download
from tokenizers import Tokenizer

d = snapshot_download("bpmatt/EmbeddingGemma-300M-Phone-Command-Router-ONNX")
sess = ort.InferenceSession(f"{d}/model_q4.onnx")
tok = Tokenizer.from_file(f"{d}/tokenizer.json")
tools = json.load(open(f"{d}/tools.json"))
emb = np.fromfile(f"{d}/tool_embeddings.f32", dtype="<f4").reshape(tools["count"], tools["dim"])

def route(text, k=5):
    ids = tok.encode(tools["query_prefix"] + text).ids
    out = sess.run(["sentence_embedding"], {"input_ids": np.array([ids], dtype=np.int64),
                                            "attention_mask": np.ones((1, len(ids)), dtype=np.int64)})[0][0]
    q = out / (np.linalg.norm(out) + 1e-9)
    s = emb @ q
    return [(tools["tools"][i]["name"], round(float(s[i]), 3)) for i in np.argsort(-s)[:k]]

print(route("wake me up at half six"))

Encode one query per run: the 4-bit model is sensitive to padding, so batching changes the scores slightly.

Results

Top-1 accuracy (the right action ranked first among 205), on held-out request sets never used for training or tuning:

Set	Requests	Top-1
Assistant dev set (everyday phone requests)	294	96.7 %
Public-dataset dev set (rephrased intents from public benchmarks)	607	95.1 %

Inside the full assistant (this router + the extractor + safety rules + a confirmation step), 66 % of held-out requests are completed correctly with no questions asked, 83 % are completed or settled with a single yes/no, and 4.6 % lead to a wrong action.

Training

Fine-tuned (contrastive, query → tool description, with hard negatives between easily confused actions) on about 12,000 examples: synthetic phone commands written for this project, covering every one of the 205 actions, plus samples from public intent datasets (MASSIVE, TOPv2, MTOP, CLINC150, SNIPS, Home Assistant intents, and mobile-action datasets), mapped onto the 205 actions. Held-out sets were checked for near-duplicates of the training data.

Use in the app

The Android app runs this model with ONNX Runtime, then adds a few deterministic rules (keyword anchors for families such as alarms vs. timers, a chat filter, and a "Did you mean…?" question when the top two are close). Recognising the action is all this model does: dates, numbers and names are filled in by the extractor and checked by app code.

Limitations

• English only; 205 fixed actions (a new action needs a new tool embedding, and ideally fine-tuning).
• It ranks actions; it does not decide whether a request is a command at all (the app uses a score threshold and rules).
• Requests that ask for two things at once ("set a timer and text Sam") are ranked as one; the app asks to split them.

Licence

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. This model is a Model Derivative of Gemma; see NOTICE, GEMMA_TERMS_OF_USE.txt and GEMMA_PROHIBITED_USE_POLICY.txt.