⚡ TAMEV-Nano-TinyBERT — Nano System One Decision Model & LLM Router (ONNX INT8, CPU & Edge Inference)

A self-hosted, open-source alternative to TypeSafe Jev: a nano-tier System One decision model and LLM router. Give it a state plus typed questions and it returns calibrated probabilities over your candidate options (choice / noul / score) in one forward pass — no text generation, no JSON repair. Use it for LLM routing, tool-call gating, intent classification and multiple-choice scoring on CPU and on-device. Ships ONNX FP32/INT8, Metal MPS, MLX and TorchScript exports for edge-CPU and on-device inference.

Task framing: this is an option-selection / decision model, not a fixed-label text classifier — the label set is supplied per call as the candidate list. It loads through transformers (AutoModel / AutoTokenizer, pipeline_tag: text-classification) and runs on CPU or on-device.

Open-source, Apache-2.0, self-hostable, and a drop-in protocol alternative to hosted decision APIs such as TypeSafe Jev. Related projects: Jared Palmer's Kev, Laya, and SemIf.

---

💡 What is TAMEV-Nano-TinyBERT? A System One decision model and LLM router

A general-purpose LLM answers a routing or triage question by generating text that your application then parses. TAMEV-Nano-TinyBERT answers it by scoring a candidate list:

• ⚡ 4.67 ms p50 / 159.7 req/s measured on Commodity CPU (single thread).
• 🎯 Typed answers: choice (pick one of K options), noul (probability of a yes/no question), and score (expected value over an ordinal rubric) in one call.
• Order-independent scoring: context and options are encoded separately and compared with a symmetric bilinear head, $\text{Score}(c, o_i) = (W_q c)^T (W_k o_i) / \sqrt{d}$, so reordering options permutes the score vector rather than changing it; the model is exactly permutation equivariant by construction. Measured drift: ≤ 2.2e-8 (float32 noise floor, dev+gate probes) with a 0.00% decision-flip rate.
• 🎯 Calibration you can inspect: calibrated probabilities, ECE 0.0564 and Brier 0.5130 on the 896-item development benchmark.
• 💰 Zero marginal cost: run it locally or in-process; no cloud round-trip and no per-token fee.

This tier is self-contained: the backbone and pointer head are already fused in model.safetensors.

---

📊 Model Overview

Dimension	Specification
Tier	Nano
Model type	encoder
Base model	huawei-noah/TinyBERT_General_4L_312D
Released artifact	Fused model (backbone + bilinear pointer head) — 14,390,185 parameters
Hidden / projection dim	312 / 64
Pooling	cls
Max state / option length	128 / 64 tokens
Calibrated temperature	1.05
Export formats shipped in this repo	PyTorch FP32 (model.safetensors), Metal MPS FP16, MLX (weights.npz), ONNX FP32, ONNX INT8, TorchScript
Export paths available from source (not shipped, not validated)	Core ML (.mlpackage, requires coremltools), GGUF (decision_engine/export/gguf_exporter.py), dynamic-INT8 PyTorch
Suggested deployment	Raspberry Pi-class ARM CPU, edge x86 CPU, mobile via Core ML/ONNX export

ℹ️ Calibrated temperature vs. preset. This card and config.json ship the post-hoc calibrated temperature (1.05). decision_engine/config/presets.py declares a training-time default (2.2) for new training runs, which is not the value these artifacts were calibrated to; do not quote the preset as this model's served temperature.

---

🏆 Evaluation

Source: the published weights in this repository, evaluated with the documented predict_decision helper over a 896-item development benchmark (data/processed/test.jsonl) built from the public datasets listed in the front matter, plus the leakage-free acceptance gate below. The dev suite is not a leakage-controlled held-out set: 390/896 items (43.5%) reuse a passage that also appears in training, 5/896 share an exact id with train.jsonl, and the same items have been reused across development rounds. The frozen 491-item acceptance set (runs/research/holdout_v1.manifest.json) is saturated (nano/micro/small top-1 = 1.000) and has no discriminative power. Suite composition (measured): agnews 270, yelp 158, boolq 82, banking77 80, mnli 59, imdb 55, sst5 54, trec 47, amazon 41, dbpedia14 39, plus 11 synthetic/game fixtures (compositional 7, legacy_policy 4) not listed in the front-matter datasets. The audit script is scripts/audit_hf_artifacts.py, shipped in the TAMEV source distribution.

Metric	Measured	Notes
Top-1 accuracy	56.25%	Development benchmark, 896 samples
Top-3 accuracy	86.50%	Development benchmark, 896 samples
ECE (temperature-scaled)	0.0564 (95% CI 0.040–0.088)	is above the repository's ≤ 0.05 target; on the 896-item development benchmark the 95% bootstrap interval is wider than the distance to the gate, so read the interval, not just the point estimate
Brier score	0.5130	Proper scoring rule, lower is better
NLL (temperature-scaled)	1.1481	Log-loss of the chosen option, lower is better
Permutation drift	≤ 2.2e-8 (float32 noise floor, dev+gate probes)	0.00% decision-flip rate under option permutation; exact by construction
Latency p50 / p95	4.67 ms / 22.13 ms	Documented predict_decision helper, Commodity CPU (single thread)
Throughput	159.7 req/s	Same run as the latency row

🚦 Leakage-free acceptance gate (pubtest_v1, n=838)

The table above is the 896-item development benchmark. The acceptance authority for cross-tier claims is the leakage-free gate runs/research/pubtest_v1.jsonl (n=838): 0 exact-id overlap and 0 passage overlap with train.jsonl/the dev suite, drawn from the same 14 (source, qid) cells and templates. It is harder and differently calibrated than the dev suite, and it changes the ranking: the dev suite is a smoke-scale estimate, the gate is the acceptance gate.

This tier on the gate: top-1 0.5263 (95% CI 0.4916-0.5573), top-3 0.8699, ECE 0.1002 (95% CI 0.0768-0.1324), Brier 0.5967, on cpu.

Tier	Device	Top-1 [CI95]	Top-3	ECE [CI95]	Brier
Nano	cpu	0.5263 [0.4916, 0.5573]	0.8699	0.1002 [0.0768, 0.1324]	0.5967
Micro	cpu	0.5823 [0.5489, 0.6134]	0.9010	0.1201 [0.1016, 0.1582]	0.5609
Small	cpu	0.6313 [0.5979, 0.6635]	0.9547	0.1270 [0.1066, 0.1608]	0.5055
Medium	mps	0.7136 [0.6826, 0.7458]	0.9690	0.0866 [0.0717, 0.1222]	0.4097
Large	mps	0.7064 [0.6754, 0.7387]	0.9749	0.1190 [0.0964, 0.1480]	0.4093

Pairwise McNemar on top-1: nano→micro p=0.0017, micro→small p=0.0028, small→medium p<0.001 (all CI-separated). Medium vs Large is not separated (top-1 .7136 vs .7064; b=55, c=49, p=0.6241), and Large's paired ECE is worse with a CI excluding zero (ΔECE = +0.0324 point difference; paired-bootstrap median +0.0243, 95% CI [0.0022, 0.0466]).

Medium is the best accuracy/calibration tier tested on the gate (highest top-1 .7136, lowest ECE .0866); Large is Pareto-dominated by Medium (strictly_dominated_by: ["tamev-medium-qwen3.5-0.8b"]) — 5.3× the total inference parameters (4,661,176,321 vs 873,963,585) for no top-1 gain and worse calibration. The large tier is the most expensive artifact, not the most accurate one. Source: runs/research/tier_compare_pubtest_v1.json (gate suite sha256 f16761d6…).

Device caveat (from the report itself): the pareto rows mix cpu and mps; compare parameters/top-1/ECE across devices and latency only within a device. Gate-run latency is load-contaminated and is not quoted on this card until the idle re-measure lands.

External public benchmarks (upstream test splits, never in TAMEV training data)

Dataset	n	Top-1	Top-1 95% CI	ECE
ARC-Easy	2376	0.2614	[0.2445, 0.2786]	0.0619
ARC-Challenge	1172	0.2363	[0.2125, 0.2602]	0.0845
OpenBookQA	500	0.2780	[0.2400, 0.3180]	0.0516
CommonsenseQA (validation)	1197	0.2139	[0.1896, 0.2364]	0.0595
Pooled	5245	0.2465	[0.2345, 0.2576]	0.0652

Accuracy on these sets is near chance (they are K=3-5 multiple-choice sets), while calibration stays in the same band as the development benchmark (pooled ECE 0.0652). Full run and pinned upstream revisions: runs/research/pub_bench_tamev-nano-tinybert.md + runs/research/pub_bench_provenance.json.

How these numbers were produced. The published weights in this repository were run through the documented predict_decision helper over the 896-item development benchmark (data/processed/test.jsonl), one call per item, single-threaded on CPU, with the state/question truncated at max_state_len = 128 and options truncated at max_opt_len = 64; each call pads only to the longest sequence it actually received, so short inputs are not billed for the full configured length. Top-1, Top-3 and the permutation benchmark do not depend on the path; the probabilities do: forward(...)["probs"] and predict_decision apply the calibrated temperature (1.05), which is what the ECE and Brier rows use, while forward(...)["logits"] are pre-temperature. Every latency figure on this card is the same documented helper path - there is no separate raw-kernel number to compare against. Reproduce the rows with .venv/bin/python scripts/audit_hf_artifacts.py <model_dir> cpu 896 from a TAMEV source checkout.

These are the numbers from one measured run, not guarantees. All five tiers were scored on the same 896 items with their full option sets (K ∈ (2, 3, 4, 6, 14, 77)), so cross-tier accuracy is like-for-like on this suite — but an 896-item mixed development benchmark is a smoke-scale estimate, not a deployment estimate, and the per-tier 95% bootstrap interval on Top-1 is roughly ±0.03.

---

🔒 How the permutation guarantee works

• Separate encoding: the state and each candidate option are encoded independently; options never attend to each other.
• Symmetric head: the score uses the same bilinear form regardless of option position, so a permutation of inputs is a permutation of outputs.
• Dedicated option budget: each option gets its own token window (up to 64 tokens), and option encoding is chunked, so a 77-option question is scored as 77 options instead of being truncated.

---

📦 Artifacts in This Repository

Runtime	Target hardware	File	Precision	Size
Hugging Face Transformers	Universal PyTorch / Python	model.safetensors	Fused model (backbone + pointer head)	54.90 MiB
Apple Metal MPS	macOS PyTorch (Metal)	model_mps_fp16.pt	FP16	27.47 MiB
Apple MLX	Mac M-series unified memory	weights.npz + tamev_mlx_config.json	Float32 arrays	54.92 MiB
ONNX Runtime INT8	Edge CPU / Raspberry Pi-class ARM / x86	model_int8.onnx	Dynamic INT8 linears, FP32 embeddings	31.35 MiB
ONNX Runtime FP32	Server CPU / containers	model.onnx	Float32	54.71 MiB
C++ LibTorch	Embedded / robotics	model.torchscript	TorchScript (no Python runtime)	55.05 MiB

ℹ️ Sizes are measured from the files in this repository (MiB). The ONNX INT8 artifact keeps embeddings and LayerNorms in FP32, so it is larger than the 13.7 MiB of quantized linear weights alone.

Download

hf download Tamkimd/tamev-nano-tinybert --local-dir ./tamev-nano-tinybert

---

🚀 Quick Start

Method 1: transformers with trust_remote_code=True

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "Tamkimd/tamev-nano-tinybert"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True)

result = model.predict_decision(          # `model.decide(...)` is an alias
    state="My transaction on a Visa card was rejected while I was travelling in Tokyo.",
    question="Which service queue should handle this incident?",
    options=[
        "verify_travel_unblock",
        "file_fraud_dispute",
        "replace_damaged_card",
        "branch_appointment",
    ],
    tokenizer=tokenizer,
)

print(f"Selected: {result['best_option']} (confidence {result['confidence']:.2%})")
print("Probabilities:", result["probabilities"])
print("Permutation drift guarantee:", result["drift_guarantee"])  # static architectural guarantee, not a per-call measurement

Requirements: torch>=2.4, transformers>=4.40, Python ≥ 3.10.

Method 2: in-process serving with the tamev package

The tamev package is not on PyPI yet; install it from the repository:

# From a TAMEV source checkout
uv venv && uv pip install -e ".[serve]"

from tamev import Choice, Noul, Score, TypeSafeDirectClient

# The default engine auto-loads the Nano checkpoint on CPU.
# `model_name` is only a display label; pass `checkpoint_path=`/`backbone=` to change the weights.
with TypeSafeDirectClient(model_name="TAMEV-Nano-TinyBERT") as client:
    res = client.system_one(
        state="Customer reports a debit card block during an overseas ATM withdrawal.",
        questions={
            "action": Choice(
                instructions="Select the incident resolution playbook",
                criteria={
                    "travel_unblock": "Verify identity and lift the travel restriction",
                    "dispute_charge": "Open an unauthorized-transaction fraud case",
                    "branch_visit": "Direct the customer to the nearest branch",
                },
            ),
            "is_emergency": Noul(instructions="Is the customer stranded and in urgent need of cash?"),
            "urgency_score": Score(
                instructions="Rate the incident urgency",
                criteria=["Routine", "Elevated", "Critical"],
            ),
        },
    )

    # Server-side `confidence` is the winning probability normalized against uniform
    # chance for choice answers, (max(p) - 1/K) / (1 - 1/K); it is not a probability.
    print("Decision:", res.answers["action"].choice)
    print("Confidence:", res.answers["action"].confidence)
    print("P(emergency):", res.answers["is_emergency"].noul)
    print("Urgency score:", res.answers["urgency_score"].score)

Method 3: HTTP server

uv run tamev serve --checkpoint Tamkimd/tamev-nano-tinybert --port 8008 --device cpu

curl -X POST http://127.0.0.1:8008/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{"state": "Payment processor latency spiked to 4s", "questions": {"playbook": {"type": "choice", "instructions": "Pick a remediation", "criteria": {"throttle": "Throttle traffic", "scale": "Scale replicas", "restart": "Restart pods"}}}}'

---

🎯 Intended Use

• Agent workflow routing and tool-call gating: pick the next tool, subagent, or queue.
• Safety and policy triage: classify an input into your own labelled categories and thresholds.
• High-cardinality classification: intent catalogues such as Banking77-style 77-option menus.
• Batch and offline scoring: label or route records with no network dependency.

Not intended for: open-ended text generation, factual question answering, multilingual deployment without validation, or acting as an unaudited safety control. The model scores the options you give it; it does not know your policy.

---

⚠️ Limitations

• English-only training data. The suite is drawn from the datasets in the front matter, which are English (Banking77 excepted for its intent taxonomy). Other languages are unvalidated.
• Calibration is per tier and not uniformly ≤ 0.05. This tier measures ECE 0.0564 on the 896-item development benchmark, which is above the repository's ≤ 0.05 target; on the 896-item development benchmark the 95% bootstrap interval is wider than the distance to the gate, so read the interval, not just the point estimate. On the leakage-free gate this tier measures ECE 0.1002 (95% CI 0.0768-0.1324), and no tier is below 0.05 on the gate — the best gate ECE is Medium .0866 and the worst is Small .1270. The "≤ 0.05" framing is a development-benchmark result only; on the gate the ECE order is Medium (.0866) < Nano (.1002) < Large (.1190) < Micro (.1201) < Small (.1270).
• Accuracy is suite-specific and this suite is a development benchmark. 390/896 items (43.5%) reuse training passages, 5/896 share an exact id with train.jsonl, and the same 896 items were reused across development rounds, so this is a development benchmark, not leakage-controlled held-out data. The frozen 491-item acceptance set is saturated (nano/micro/small top-1 1.000) and cannot discriminate between tiers. Measured on the Nano tier: three retrains on the same corpus gained +4.5 to +6.4 points here while the external ARC/OpenBookQA/CommonsenseQA splits moved by at most +0.8 points with overlapping 95% CIs, and external ECE degraded (0.0652 to 0.0792-0.0893). This benchmark therefore cannot be used as a ship gate on its own. Validate on your own distribution.
• Formats other than the ones listed above (Core ML, GGUF, dynamic-INT8 PyTorch) are exportable from a source checkout but are not shipped in this repository and are not validated here.
• Tokenizer assets are the backbone tokenizer. Tokenization is identical to huawei-noah/TinyBERT_General_4L_312D; no TAMEV-specific vocabulary or special tokens were added.
• Backbone licence caveat (Nano only). The backbone huawei-noah/TinyBERT_General_4L_312D publishes no declared licence on the Hub (verified live, 2026-09-27). This repository's apache-2.0 licence covers TAMEV's own code and pointer head; it is not a grant for the upstream backbone weights. Confirm the upstream terms before commercial redistribution, or switch the Nano backbone to a licence-declared checkpoint.
• Teacher/KD coverage. Teacher probabilities exist only for the 3,500 synthetic game rows (snake_navigation 2,000, tetris_placement 1,500) of the 10,658-row merged training set; the 7,158 public-dataset rows carry none. Do not describe the public-dataset training as knowledge distillation.
• Latency is hardware-specific. 4.67 ms p50 was measured on Commodity CPU (single thread); your numbers will differ. No latency is claimed for the MLX or TorchScript artifacts, nor for the source-only Core ML / GGUF / dynamic-INT8 paths.
• Permutation equivariance is an architectural property, while the drift value above is measured by the benchmark suite (≤ 2.2e-8 (float32 noise floor, dev+gate probes)); treat it as a measurement, not a contract.
• Protocol compatibility is not an endorsement. TypeSafe-compatible means the /v1/systemone request/response shape, not TypeSafe's model or quality guarantees.

---

🥊 TAMEV Model Zoo

Tier	Model	Backbone	Released params	Latency p50	Top-1	Top-3	Calibration	Permutation drift
Nano	tamev-nano-tinybert	TinyBERT_General_4L_312D	14,390,185	4.67 ms	56.25%	86.50%	ECE 0.0564 / Brier 0.5130	≤ 2.2e-8 (float32 noise floor, dev+gate probes)
Micro	tamev-micro-minilm	all-MiniLM-L6-v2	22,762,369	7.98 ms	66.52%	90.85%	ECE 0.0502 / Brier 0.4248	≤ 2.2e-8 (float32 noise floor, dev+gate probes)
Small	tamev-small-modernbert	ModernBERT-base	149,407,489	57.64 ms	72.32%	95.98%	ECE 0.0508 / Brier 0.3545	≤ 2.2e-8 (float32 noise floor, dev+gate probes)
Medium	tamev-medium-qwen3.5-0.8b	Qwen3.5-0.8B-Base	524,801	145.77 ms	82.81%	97.43%	ECE 0.0402 / Brier 0.2235	0.032
Large	tamev-large-qwen3.5-4b	Qwen3.5-4B	1,311,233	547.54 ms	80.80%	97.66%	ECE 0.0346 / Brier 0.2413	0.035

Medium/Large ship a pointer head and lazy-load the Qwen3.5 backbone; encoder tiers ship the full fused model. The dev-suite ranking is not the gate ranking: on the leakage-free acceptance gate Medium is the best accuracy/calibration tier tested (top-1 .7136, ECE .0866) and Large is dominated by Medium — 5.3× the total inference parameters for no top-1 gain and worse calibration. Large is the most expensive artifact, not the most accurate one. Accuracy across tiers also comes from different option budgets (4 vs 8), so compare with care.

---

📄 License & Citation

Apache License 2.0.

@misc{tamev2026,
  title={TAMEV: System One Decision Models for Edge AI, LLM Routing and Tool-Call Gating},
  author={TAMEV Contributors},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/Tamkimd/tamev-nano-tinybert}
}