emrevrg/AUBIN-Mini

AUBIN Mini — AUBIN by Norovox

🔬 Preview — This model is under active development; stronger versions will be released in this repository.

Typed decisions on the phone: question + options + state in, calibrated probabilities out. CPU only, 267 MB.

AUBIN Mini is the on-device member of the AUBIN family. It answers structured questions (choice, yes / no, any number of options) about a piece of state (a ticket, a message, a form, a review) and returns a probability for every option, so an app can act on the answer or ask for help when the confidence is low. It needs ONNX Runtime and a tokenizer, nothing else: no torch, no GPU, no network after the first download.

parameters	395M (ModernBERT-style encoder, inputs up to 512 tokens)
phone file	aubin_mini_nb4_g8.onnx, 267 MB (4-bit block weights, 8-bit embeddings), 5.9x smaller than fp32
also included	aubin_mini_nb4.onnx (422 MB), model.safetensors (full-precision PyTorch weights for fine-tuning)
runtime	ONNX Runtime CPU (desktop, server, Android / iOS builds of ONNX Runtime)
output	one calibrated probability per option (temperature fitted on Kev's development split)
languages	trained and measured in English

Results on Kev's public locked suites

Same items and metrics as jaredpalmer/kev (decision-v7 test = 1,440 questions from sources Kev trained on; transfer-v4 test = 764 records from sources it never trained on). Kev numbers are the published ones from Kev's README. Lower Brier / ECE is better.

model	file	trained sources acc (test)	new sources acc (test)	trained sources Brier / ECE (test)	new sources Brier (test)
AUBIN Mini, phone build (aubin_mini_nb4_g8.onnx)	267 MB	0.847	0.622	0.222 / 0.039	0.503
AUBIN Mini, 4-bit (aubin_mini_nb4.onnx)	422 MB	0.846	0.617	0.223 / 0.032	0.502
AUBIN Mini, full precision (PyTorch)	1.58 GB	0.844	0.618	0.226 / 0.036	0.503
EmbeddingGemma-300M, same training recipe (ours)	—	0.763	0.487	0.330 / 0.053	0.613
Kev-0.8B (Qwen3.5-0.8B + LoRA)	—	0.838	0.697	—	0.416
Kev-4B (Qwen3.5-4B + LoRA)	—	0.865	0.838	—	0.242

The phone build was measured with this repository's own code (code/eval_kev.py, ONNX Runtime CPU) on all 1,440 + 764 test questions; its tokenizer reproduces the training tokenizer exactly (960 / 960 checked pairs). On sources it was trained on, the 267 MB phone build is ahead of Kev-0.8B; on sources never seen in training, Kev-0.8B and larger generative models are ahead. EmbeddingGemma-300M row: Google's embedding model trained by us with exactly the same data, teacher and number of steps, as a like-for-like on-device baseline.

Speed (one question = all of its options): on an ordinary Windows PC CPU (all cores) a 2–3 option decision took 0.6–1.0 s and a two-question request 2.6 s, after a one-time load. Single-thread phone stand-in: median 5.9 s per question on one CPU thread of a 4-core Kaggle CPU machine with four evaluation processes sharing it; 2.9 s with a single process (300 development questions).

Per source, phone build, decision-v7 test (sources seen in training):

source	n	accuracy
agnews	116	0.905
agnews_yn	184	0.908
amazon	80	0.588
banking77 (77 options)	116	0.724
boolq	80	0.750
contrastive_age_eligibility	40	1.000
contrastive_quantity_limit	40	1.000
contrastive_return_window	40	0.775
contrastive_spend_threshold	76	1.000
dbpedia14	116	0.974
imdb	80	0.950
mnli	116	0.819
sst5	80	0.588
trec	116	0.966
yelp	80	0.637
yelp_yn	80	0.938

Per source, phone build, transfer-v4 test (sources never seen in training):

source	n	accuracy
composition_final_combination	32	0.500
composition_final_exception	32	0.812
composition_final_negation	32	0.750
contrastive_authorization	40	0.850
contrastive_deadline	40	0.450
emotion	116	0.491
mmlu	116	0.345
paws	80	0.550
qnli	80	0.787
sciq	116	0.784
tweet_offensive	80	0.775

Use

pip install "git+https://huggingface.co/emrevrg/AUBIN-Mini#subdirectory=code"

from aubin_mini import AubinMini

m = AubinMini.from_pretrained("emrevrg/AUBIN-Mini")      # phone build (267 MB); variant="nb4" for the 422 MB file
m.decide({"ticket": "Package arrived broken, customer wants money back"},
         {"route":  {"type": "choice", "instructions": "Which team should handle this?",
                     "criteria": {"billing": "refunds and charges", "shipping": "delivery and damage", "tech": "software"}},
          "urgent": {"type": "noul", "instructions": "Is this urgent?"}})
# → {"route":  {"answer": "...", "confidence": ..., "probabilities": {"billing": ..., "shipping": ..., "tech": ...}},
#    "urgent": {"answer": True/False, "confidence": ..., "probabilities": {"false": ..., "true": ...}}}

m.choose("Which team should handle this?", ["billing", "shipping", "technical support"],
         state="The app crashes every time I open the camera screen.")   # → {"answer": "technical support", ...}

Command line and a local HTTP endpoint:

aubin-mini decide request.json                 # {"state": ..., "questions": {...}}
aubin-mini serve --port 8009                   # POST /decide (same JSON), GET /health

Reproduce the table: python code/eval_kev.py --model emrevrg/AUBIN-Mini --variant phone (add --shard k/n to split).

How it works

• Cross-encoder, one score per option. Each option is read together with the question and the state (QUESTION: … OPTION: … ‖ STATE: …), the encoder's first token is mapped to a single score, and the scores of a question are turned into probabilities with a softmax at the fitted temperature. Because options are scored one by one, there is no limit on the number of options (banking77 has 77) and option order does not matter.
• Long state is cut, the question never is: the state side is truncated at 512 tokens.
• Training: Ettin-encoder-400M, fine-tuned on Kev's decision-v7 train split with knowledge distillation from the AUBIN ensemble's probabilities (α 0.7) plus the gold label; checkpoint chosen on the development split only. No transfer source was used for training, selection or calibration.
• Phone build: ONNX export, 4-bit block-wise (32) symmetric weight quantization of every matrix multiply and 8-bit embedding table. On the same 300 development questions it scores 0.833 vs 0.837 for fp32 (one question).

Notes

• Every number comes from the scripts in code/ on Kev's public suites; nothing is hand-edited.
• Latency is measured on one CPU thread on a Kaggle CPU machine, as a stand-in for a phone core, not on a phone. ONNX Runtime uses all cores by default, which is faster.
• The phone file uses ONNX Runtime's MatMulNBits operator. The standard onnxruntime pip package includes it; on mobile use the full ONNX Runtime package, not a reduced-operator build.
• Base model: jhu-clsp/ettin-encoder-400m (MIT). This release: Apache-2.0.

AUBIN by Norovox · version 0.1.0

---

Support Norovox

Norovox is built by one 17-year-old high school student in Türkiye: no company, no sponsor, no budget. Every model on this page was trained on free GPU quotas that run out within days. When they run out, training stops halfway and the next, stronger version has to wait, sometimes for a week.

Your support goes directly into GPU time. Even a small contribution lets a training run finish instead of stopping, and helps the Nythos, N1, AUBIN and Norovox Embed models get better faster. Supporters are credited in the model cards and get early access to new releases.

Contact: zgremre@gmail.com · emrevrgdev@gmail.com

_{🇹🇷 Norovox'u Türkiye'de 17 yaşında bir lise öğrencisi tek başına geliştiriyor: şirket yok, sponsor yok, bütçe yok. Buradaki modellerin hepsi birkaç günde biten ücretsiz GPU kotalarıyla eğitildi; kota bitince eğitim yarıda kalıyor ve daha güçlü sürüm bekliyor. Desteğin doğrudan GPU süresine gider; küçük bir katkı bile bir eğitimin yarıda kalmadan tamamlanmasını sağlar. Destekçiler model kartlarında anılır ve yeni sürümlere erken erişir. İletişim: zgremre@gmail.com · emrevrgdev@gmail.com}