Smaug-Decisions-LiteRT: Smaug-Decisions on phones

Frontier-level audio decisions, offline on a phone.
3.45 GB on the phone GPU, 59.4 Earshot audio skill: level with Gemini 4 Argon (59.4), within 0.7 of full-precision Smaug-Decisions.

1. Model Introduction

Smaug-Decisions, the 4B decision model that hears, sees and reads, in one 3.45 GB file for Google's on-device runtime LiteRT-LM, with the audio and vision encoders inside, running on the phone GPU. It answers a typed question about audio, an image or text with a probability per option, in one prefill and with no generated text.

It scores 59.4 on Earshot audio decisions, 53.9 on Decision Index text and 55.1 on images (full-precision Smaug-Decisions: 60.1 / 56.6 / 57.2 on the same sets), and runs fully offline in the Cascade app on iPhone and Android next to Smaug-Audio-Decisions-0.7B, which answers the easy audio questions first.

import json
from huggingface_hub import hf_hub_download
manifest = json.load(open(hf_hub_download("abacusai/Smaug-Decisions-LiteRT", "manifest.json")))
path = hf_hub_download("abacusai/Smaug-Decisions-LiteRT", "smaug-decisions-full-vis280.litertlm")      # 6 GB phones; vis560 for 8 GB+

2. Files: which device gets which

All files share the same weights; the smaller ones leave out the encoders a device does not need. In the names, vis560 means images at 560 image tokens (more detail, for phones with 8 GB of RAM or more) and vis280 images at 280 tokens (for 6 GB phones; an image needs about 1 GB of working memory instead of 2 GB). manifest.json lists each file with its size, sha256, inputs and download URL.

file	size	answers	for
`smaug-decisions-full-vis560.litertlm`	3.45 GB	text, audio, images	phones with 8 GB of RAM or more
`smaug-decisions-full-vis280.litertlm`	3.45 GB	text, audio, images	6 GB phones
`smaug-decisions-noaudio-vis560.litertlm`	3.15 GB	text, images	8 GB+ phones where Smaug-Audio-Decisions-0.7B handles all audio
`smaug-decisions-noaudio-vis280.litertlm`	3.15 GB	text, images	the same on 6 GB phones
`smaug-decisions-novision.litertlm`	3.28 GB	text, audio	audio and text, no photos
`smaug-decisions-textonly.litertlm`	2.97 GB	text	text only

Phones under 6 GB run Smaug-Audio-Decisions-0.7B alone.

In the Cascade app

The iOS and Android apps ship the same two models:

	iOS	Android
Smaug-Audio-Decisions-0.7B (audio, inside the app, works with no download)	Core ML, int8 weights, int8 embedding table	ONNX Runtime, int8 weights, the same int8 embedding table
Smaug-Decisions (downloaded in the app from this repo)	LiteRT-LM on the phone GPU (Metal)	LiteRT-LM on the phone GPU
app size	602 MB	about 582 MB (Play download, arm64)

Which Smaug-Decisions file a phone gets:

phone RAM	default download	file
8 GB or more	Full cascade, 3.45 GB	`smaug-decisions-full-vis560.litertlm`
6 GB	Full cascade, 3.45 GB	`smaug-decisions-full-vis280.litertlm`
under 6 GB	nothing: Smaug-Audio-Decisions-0.7B only	-

Nothing downloads until the user taps Download, and before that they can switch from the full cascade to text + images (smaug-decisions-noaudio-*, 3.15 GB; Smaug-Audio-Decisions-0.7B answers all audio), audio + text (smaug-decisions-novision, 3.28 GB; no photos) or Smaug-Audio-Decisions-0.7B only (no download). smaug-decisions-textonly is for other apps; the Cascade app does not offer it. Audio goes to Smaug-Audio-Decisions-0.7B first and escalates to Smaug-Decisions below 0.9 confidence; text and images go straight to Smaug-Decisions.

3. Model Summary

Architecture	Gemma 4 E4B (language model + audio and vision encoders), fine-tuned as Smaug-Decisions; letter-logit readout
Weights	int4 decoder, int2 embedding tables, int8 audio encoder, an output head that holds only the 20 option letters
Input	Text; audio (16 kHz mono); one image at 280 or 560 image tokens; a question with 2 to 20 options
Output	The logits of the option letters `A`..`T` at the last position, read as a probability per option
Context	2,048 tokens (prompt included)
Runtime	LiteRT-LM, GPU backend (Metal on iPhone) or CPU
Base Model	`abacusai/Smaug-Decisions`

4. Evaluation

The 3.45 GB file keeps almost all of Smaug-Decisions' quality: within a point on audio and its calibration, and two to three points on text and images, at about a fifth of the size.

	Smaug-Decisions (full precision, 16 GB)	Smaug-Decisions-LiteRT (3.45 GB, on the phone)	change
Earshot audio skill ↑	60.1	59.4	−0.7
Earshot calibration error (ECE) ↓	0.069	0.074	+0.005
Decision Index text ↑	56.6	53.9	−2.7
Decision Index images, 560 image tokens^* ↑	57.2	55.1	−2.1
Decision Index images, 280 image tokens^* ↑	54.8	51.8	−3.0

Both columns on the same items with the same harness. The phone-build scores come from a PyTorch replica of the file's quantized weights (int4 decoder, int2 embedding tables, int8 vision encoder). ^*10 of the 11 public vision sets (Winoground left out); Smaug-Decisions' index on all 11 is 57.9. 6 GB phones use the 280-token build. See the Smaug-Decisions card for the benchmarks.

5. Memory and speed

iPhone 14 Pro Max (6 GB RAM), smaug-decisions-full-vis280, phone GPU, in the Cascade app:

	measured
first load (one time, writes a cache)	13-21 s
later loads	0.48 s
text decision	~0.6 s
audio decision	~1.6 s
image decision (280 image tokens)	~7.4 s
peak app memory, with Smaug-Audio-Decisions-0.7B loaded	1.35 GB

Storage: the file plus a cache of about 2.4 GB written on first load, so plan for about 6 GB.

6. Limitations

Per request: 2,048 tokens including the prompt, one image, up to 20 options.
Decisions only: the files answer typed questions; for text generation use full Smaug-Decisions.
Thresholds: set decision thresholds on your own data for high-stakes uses.

7. Usage

Reading the decision. Prefill the prompt rendered by smaug_decide.render in abacusai/Smaug-Decisions (no chat template re-applied) and take the logits of the option letters at the last position. Apply the softcap 30 (30 * tanh(z / 30)), then softmax over the options (one answer) or sigmoid(z - 8.0106) per option (select-all). Stock LiteRT-LM's RunTextScoring, renormalized over the options, gives the one-answer probabilities; select-all needs the absolute logits, which the runtime in the app repository exposes.

Cache. Put the cache written on first load in a directory the OS does not purge, and attach the vision encoder only when an image arrives.

Apps. The iOS (Swift) and Android (Kotlin) Cascade apps, including the device policy that picks the file, are at github.com/abacusai/Smaug-Decisions-Mobile.

8. License

Apache 2.0, the same license as the base model, Gemma 4 E4B-it.

9. Citation

@misc{abacusai2026smaugdecisions,
  title  = {Smaug-Decisions: calibrated typed decisions over audio, images and text},
  author = {Abacus.AI},
  year   = {2026},
  url    = {https://huggingface.co/abacusai/Smaug-Decisions}
}