Reef

A general typed-decision model that runs entirely on a Google Coral Edge TPU. You send a state (text or JSON) and typed questions: yes/no (noul), one-of-N (choice) or an ordinal rubric (score). You get a calibrated probability for every option. About 11M parameters in total, int8, one Edge TPU subgraph that fits in the Coral's on-chip memory.

Part of Reef: give your retired Coral a new job.

Decision Index 0.2.1

Self-measured with the public harness (apolinario/decision-index, MIT) on the full local suite (150,246 requests). This is not an official leaderboard entry. The comparison rows are the published board scores.

Model	Params (total)	Index
Reef (this model, int8)	11.2M	4.73
GLiNER 2.5 small (fastino/gliner2.5-small-v1)	73.9M	3.82
Reef v1 (previous version, a homelab specialist)	10.3M	2.38

How the Reef score was measured:

• Serving mode. Measured with the server's default (lenient) mode: over-long inputs are truncated to the trained limits instead of refused. With --strict, the score is about 0.15–0.2 lower (measured on a 19,640-row subset with an earlier checkpoint).
• HLE. HLE has no rows in the local suite and is scored 0, so the number is a lower bound for the board.
• Run-to-run variance. Retraining moved single benchmarks by up to ±0.1 skill (PhishNChips most), so treat differences of about ±0.3 Index points between checkpoints as noise.
• Params. 7.38M are in the int8 encoder on the Edge TPU and 3.78M in the host's token-embedding table (frozen potion-base-8M vectors projected to 128 dimensions). All neural-network compute runs on the Coral. The host tokenises, looks up token vectors and applies a softmax.

Area	Skill (×100)
Retrieval	10.47
Tools	5.56
Language	4.79
Arts & human taste	1.62
Knowledge	0.84

Where it does well: intent detection (BANKING77 0.26 skill, CLINC150+OOS 0.14 with 77/151 options), When2Call (0.25), FinEntity (0.37), PhishNChips (0.18), ContractNLI (0.10).

Where it doesn't: it sits at chance, with no detectable signal, on tool retrieval and selection (BFCL, ToolRet), BRIGHT, HoVer, ANLI, VAST, Amazon ESCI and the knowledge and reasoning benchmarks (MMLU-Pro, BBH, GSM8K, CLadder, CRUXEval). An 11M model trained from scratch has little world knowledge.

Shortcuts deliberately not used. We did not rank documents by the harness's BM25 order (ToolRet/BRIGHT), and we did not decode BFCL as "exactly one tool". Both would have added Index points without any model skill.

On the Edge TPU

Same int8 model on a Coral USB Accelerator:

Test set (held out)	Accuracy	KL ↓
Homelab packs (5 packs, Gemma 4 soft labels)	0.807	0.212
Answer-audit (judge for agent transcripts)	0.705	0.634
typed-decisions test	0.535	0.294
Open-Jev OOD	0.695	0.610

For the narrow homelab packs alone, the previous homelab-specialist version is still slightly better (0.84). This model is the generalist.

Quickstart

pip install "reef-decide @ git+https://github.com/SeraphimSerapis/reef"
reef serve --host 0.0.0.0 --port 8080      # fetches this model; Edge TPU if attached, else CPU

curl -s localhost:8080/v1/systemone -d '{
  "state": {"source": "uptime-kuma", "monitor": "nextcloud", "status": "down", "down_for": "14m"},
  "questions": {"page": {"type": "noul", "instructions": "Should someone be woken up for this?"}}
}'

Files

File	What
reef_int8_edgetpu.tflite	encoder compiled for the Edge TPU (1 subgraph, 7.36 MiB on-chip, 0 B off-chip)
reef_int8.tflite	the same int8 encoder for CPU (LiteRT, XNNPACK)
embeddings.npz	token / role / level embedding tables (looked up on the host)
tokenizer.json, config.json	tokenizer, shapes and per-type calibration temperatures
reef_float.weights.h5	float training weights (Keras), for reef train --init seraphimserapis/reef

Model

• Convolutional encoder, no attention (the Edge TPU compiler cannot map it): 11 blocks, width 256, MLP expansion 3, a 3×5 depthwise conv and mean+max state summaries, plus two question bands with their own summaries and a multiplicative gate.
• Input: a 384-token state region plus a 384-token question region, folded into a 24×32 grid. Long states are read in overlapping windows, and questions with many options as a knockout.
• Training stages:
  1. pre-training on about 300k public decision cases;
  2. fine-tuning on homelab packs, answer-audit, typed-decisions, Open-Jev and the public train sets below;
  3. quantisation-aware fine-tuning, so the int8 model stays within about 1–2 accuracy points of the float model.

Training data

All public data comes from train splits only, deduplicated against the Decision Index test inputs (normalised exact-match on distinguishing inputs, lines and options), with permissive licences. It uses human labels with label smoothing. One exception was kept on purpose: tool descriptions shared by public tool catalogs (Glaive, ToolACE, When2Call) also appear in the ToolRet/BRIGHT candidate corpus; the model scores at chance on ToolRet.

• Intents: MASSIVE en/de, SNIPS, BANKING77, CLINC150, MINDS-14
• NLI and contracts: WANLI, MultiNLI (OANC genres only), ContractNLI
• Tools: Glaive function calling v2, ToolACE, When2Call, API-Bank
• Retrieval: Amazon ESCI
• Sentiment, finance and toxicity: GoEmotions, Civil Comments, SMS Spam, SEntFiN, Twitter financial news topic/sentiment, iSarcasmEval
• Commonsense: HellaSwag, WinoGrande, GSM8K
• Benchmark data: ZefanCai/Open-Jev (CC0) and LocalLLaMA/typed-decisions (Apache-2.0)
• Generated home-appliance households: the harness's generator, MIT, with seeds disjoint from the test households

Licences: CC BY 4.0, CC BY 3.0, CC0, Apache-2.0 and MIT.

The homelab packs and the answer-audit pack are synthetic (no real home data). Their soft labels are answer distributions from Gemma 4:

• homelab packs: E4B and 26B-A4B, an ensemble of option logprobs;
• answer-audit: E4B.

Thank you to the Gemma team. Question wording for the public sets mirrors the Decision Index adapters (MIT).

Limitations

• Decisions, not reasoning. It doesn't know facts or do arithmetic. Use it to filter, route and rank cheap, frequent decisions, and send low-confidence ones to a bigger model.
• Inherited judgement. Distilled labels inherit the teacher's judgement and blind spots. Keep a human in the loop for anything safety-critical.
• Speed. On a modern laptop CPU it runs at a similar speed. The Coral makes it cheap to run next to Home Assistant on a Pi, an N100 or a NAS.

License

Apache-2.0. See NOTICE for third-party credits.