ollaya-dev/coba-150m-ternary

Coba 150m ternary

Coba 150m ternary is a small English decision model by Ollaya. Give it a state (a message, a support ticket, a JSON object, an agent's next tool call) and questions, and it answers every question in one forward pass with calibrated probabilities:

• noul: yes or no, as the probability of yes;
• choice: one of up to 255 options, each described in plain words;
• score: a level on a scale of 2 to 10 described levels.

Ternary: every weight of the encoder layers and the head is −1, 0 or +1 times a scale per output row, and the token embeddings have 4 bits, so in a browser tab the whole model is a 66 MB download (onnx/model.onnx and the tokenizer); model.safetensors holds the same weights as float32, for laya and Ollaya. It trained in-domain: its data includes the train splits of ten of the public benchmark's datasets (see Results).

Run it

In a web page (npm install @ollaya/web): the model downloads once (66 MB) and runs on the device.

import { fromHub, load } from '@ollaya/web';

const model = await load(fromHub('ollaya-dev/coba-150m-ternary'));
const { answers } = await model.decide({
  state: "I was charged twice for order A-104. Refund the duplicate today or I'm cancelling.",
  questions: {
    refund: { type: 'noul', instructions: 'Is the customer asking for a refund?' },
    team: { type: 'choice', instructions: 'Which team handles this?',
            criteria: { billing: 'Payments, charges, refunds', technical: 'Bugs, outages, errors' } },
    urgency: { type: 'score', instructions: 'How urgent is this?', criteria: ['Can wait', 'This week', 'Today'] },
  },
});

With Ollaya (Linux, macOS, Windows; CPU, CUDA, Metal):

curl -fsSL https://ollaya.dev/install.sh | sh
ollaya run coba:150m-ternary --ask "Is the customer asking for a refund?" < ticket.txt

or POST http://localhost:11435/api/decide with {"model": "coba:150m-ternary", "state": …, "questions": …} (API). Ollaya also speaks TypeSafe's /v1/systemone, so Jev clients work unchanged.

In Python with laya (pip install laya):

import laya

agent = laya.load("ollaya-dev/coba-150m-ternary")
agent.system_one("I was charged twice for order A-104.", {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
})

Ollaya and laya give the same answers, and so does the browser (checked against Ollaya's runtime on the same questions): Ollaya checks its runtime against this checkpoint's PyTorch outputs on 2,383 questions (every decision the same, probabilities within 3.6e-6).

Results

Model	Weights	typed-decisions	Public benchmark
coba:1b	fp32	0.608	0.561 zero-shot
coba:400m	fp32	0.591	0.561 zero-shot
coba:400m-ternary	ternary	0.559	0.654 in-domain
coba:150m-ternary (this model)	ternary	0.535	0.622 in-domain
coba:68m-ternary	ternary	0.483	0.603 in-domain
pplx-decider:flash (the teacher, 4B decoder)	bf16	0.726	0.824 zero-shot
decider:0.8b (0.8B decoder)	bf16	0.506	0.669 zero-shot
laya:en (Laya's English model)	fp32	0.358	0.532

• typed-decisions: accuracy on the 400 rows of LocalLLaMA/typed-decisions' test split, which no model here trained on.
• Public benchmark: macro accuracy over the 3,880 records of the eleven datasets Ollaya's bench uses (MNLI, BoolQ, PAWS, VitaminC, civil_comments, Aegis 2.0, HelpSteer2, PubMedQA, SQuAD 2.0, MASSIVE, SummEval). In-domain: the model also trained on the train splits of ten of them (never their test records; rows that share any text with a test record were dropped). Zero-shot: trained without them.
• For scale: pplx-decider:flash, the 4B decoder this model learned from, scores 0.726 and 0.824 zero-shot; 0.8B decoders score 0.46–0.51 and 0.61–0.67.

Training

• Network: Laya's (laya 0.3.7): the question, a [MASK] before every option and the state in one sequence of up to 512 tokens; a two-layer transformer head scores each option at its marker. Encoder: Ettin 150m (ModernBERT architecture, MIT).
• Data: about 400,000 English questions in 344,000 states from permissively licensed decision datasets (Apache-2.0, MIT, CC BY, CC BY-SA, CC0, BSD; every source and its license is in DATA_SOURCES.tsv), plus 180,000 from the benchmark datasets' train splits. Rows from datasets with a non-commercial or unknown license were dropped.
• Labels: each question's gold answer where the data has one, mixed with the full probability distribution of pplx-decider:flash (Apache-2.0), computed through Ollaya.
• Recipe: soft cross-entropy plus the ranked probability score on score questions; options shuffled half the time; one temperature per question type fitted on held-out data. Ternary weights: quantization-aware training from the fp model of the same size (absmean per row, as BitNet b1.58, with a straight-through estimator), distilled from it (logits, attention relations and every layer's hidden states), with the fp weights blended into the quantized ones over the first 30% of the steps.

Limits

• English only. Other languages are not refused but are not measured.
• 512 tokens for the state and the question together: messages, tickets and small JSON states, not long documents or agent traces.
• score questions are the weakest kind for every model of this size; read the probabilities, not only the expected level.
• Probabilities are calibrated on average, not per domain: check them on your own data before acting on them automatically, and keep a person in the loop for decisions that matter.

License

Apache-2.0. The encoder started from Ettin (MIT; see NOTICE). The training data's sources and their licenses are in DATA_SOURCES.tsv; CC BY and CC BY-SA sources are credited there.