cox-1-3b

What it is

Cox is a decision model. Given a text and a set of typed questions — yes/no, pick one of these options, or rate on this scale — it returns a probability for every option of every question, in one pass, without generating text. An answer can only be one of the options the caller offered, and the probabilities are meant to be used directly: automate above a threshold, escalate below it.

• Name: cox-1-3b (open weights).
• The name: Cox is named for Richard T. Cox (1898–1991), whose theorem (1946) showed that any consistent way of reasoning about uncertain statements must follow the rules of probability, which is why every answer Cox gives is a probability over the options you offered.
• Base model: fine-tuned by Vitund from Qwen3.5-4B-Base (Apache-2.0), with a decision head added. Text only.
• Not a text generator: cox-1-3b is a decision model and does not produce text; load it with coxlm.
• Size: about 2.9B parameters (20 transformer blocks); 5.8 GB in bf16.
• Answer types: yes/no; choice (options supplied by the caller, each with an optional description); score (an ordered scale supplied by the caller).
• Formats: bf16 safetensors. FP8 and 4-bit builds will follow.
• Developer: Vitund. Licence: Apache-2.0 (weights and code), with the data attributions below.

How to use it

Write the question the way you would ask a careful colleague, and describe each option: Cox reads the descriptions. One request can carry many questions about the same text; the text is read once and each question is answered independently of the others. Use the returned probability, not just the top answer: pick a threshold for automatic handling from your own labelled sample, and send the rest to a person. Load it with the coxlm package (pip install coxlm).

Good fits: routing and triage, policy and rule compliance with the rule stated in the question, moderation, change detection between two versions of a text, "is there enough information here to decide?", choosing among supplied options, grading against a rubric.

Putting things in order. coxlm's order_items orders a set of events, steps or tasks by asking cox-1-3b, for every pair, which comes first, and combines the answers into one order. Besides the order, it reports how sure the model is, which items could happen at the same time, and how consistent the pairwise answers are. In a small check of our own (8 hand-written cases, several of them telling the events out of order in the text, such as a delivery described before the order that caused it), it found the correct order every time, in about 140 ms per case for four or five items on an RTX 4090. See the incident timeline example.

from coxlm import order_items

result = order_items(model, ["checkout latency alarms fire", "the configuration change is deployed",
                             "the change is rolled back", "checkout recovers"],
                     mode="pairwise", context=incident_review_text,
                     instructions="In what order did these events happen?")
print(result["ordered_items"])

Use code instead (or as well): rules that reduce to a pattern — "contains a URL", "longer than 100 words", "contains the digit 7". Cox reads surface features reasonably well but it is not a character counter, especially near a threshold. For compound rules ("contains a phone number and is abusive"), ask the parts as separate questions and combine them in code.

Evaluation

We scored every model with the same code on 15 public datasets, about 18,500 decisions, choosing datasets that none of the compared models is known to have trained on (Jev's training data is not published). Before scoring we reviewed labels the models disagreed with confidently and corrected or removed the clearly wrong ones, the same way for every model.

model	size	skill (95% interval)	accuracy	calibration error
cox-1-3b	2.9B	+0.648 (+0.635 to +0.660)	80.3%	0.082
Kev 4B	4B	+0.615 (+0.601 to +0.628)	78.7%	0.082
Kev 9B	9B	+0.652 (+0.639 to +0.664)	80.7%	0.082
Jev 1.14.0	hosted	+0.669 (+0.655 to +0.682)	82.0%	0.064

Skill is 0 for always giving the most common answer and 1 for perfect, so it can be compared across tasks with different numbers of options; calibration error is the average gap between stated confidence and accuracy. cox-1-3b's confidences lean slightly conservative: averaged over these tasks, its stated confidence is about one point below its actual accuracy (Kev 4B two points below, Kev 9B one, Jev 1.14 about five points above), which is the safer side when you automate everything above a confidence threshold.

Accuracy on each dataset:

dataset	what is asked	decisions	cox-1-3b	Kev 4B	Kev 9B	Jev 1.14
ATIS	which details a flight request gives (origin, destination, date, time, airline): five yes/no questions	4,465	94.6%	95.6%	95.2%	96.3%
Civil Comments: toxicity	is a comment toxic, is it insulting	1,999	73.3%	75.9%	68.2%	66.8%
Civil Comments: threat	does a comment threaten someone	1,000	85.7%	82.5%	86.9%	84.6%
Civil Comments: identity attack	does a comment attack a group	1,000	91.7%	89.4%	93.0%	82.3%
ContractNLI	does an NDA clause support, contradict or not mention a statement	1,000	80.2%	77.3%	81.8%	82.7%
SemEval-2018 irony	is a tweet ironic	783	82.8%	73.8%	80.8%	83.4%
LEDGAR	which of 100 clause types a contract provision is	1,000	57.7%	59.1%	60.4%	65.4%
MASSIVE	which of 60 intents a voice-assistant request has	992	77.4%	75.6%	77.6%	85.7%
PAWS	are two sentences paraphrases	998	79.8%	79.2%	79.0%	84.8%
PubMedQA	does a medical abstract answer the question yes, no or maybe	999	76.1%	74.1%	76.0%	79.6%
SciFact	does an abstract support or refute a scientific claim, or not cover it	320	80.9%	77.8%	84.1%	85.0%
SMS Spam	is a text message spam	998	96.9%	92.7%	95.2%	98.6%
STS-B	how similar two sentences are, on a 0 to 5 scale (exact level)	999	48.5%	47.1%	49.3%	54.3%
SUBJ	is a sentence subjective or objective	998	90.1%	93.4%	94.0%	91.9%
WikiQA	does a Wikipedia sentence answer a question	999	88.7%	87.0%	88.9%	88.6%

JevBench v1, an independent public benchmark for decision models maintained by Benchmark Heaven (github.com/fstandhartinger/jevbench, 231 scenarios), run with its unmodified harness:

model	accuracy	Brier score
cox-1-3b	77.1%	0.307
Kev 4B	75.8%	0.323
Kev 9B	79.7%	0.290
Jev 1.14.0	87.9%	0.142

Other decision models. Most other decision models trained on some of the public datasets above, so we score each one only on the 21 suite tasks its makers don't list as training data, with cox-1-3b on the same tasks beside it. Where a model's training data isn't published, its row uses every task and may flatter it.

model	open weights	size	tasks	its skill	cox-1-3b, same tasks	its training data
Liquid d1-ultra	no	not disclosed	21	+0.750	+0.657	not published
Perplexity Decider V1.2	yes	27B	14	+0.747	+0.628	partly published; also official training splits of benchmarks
Perplexity Decider Flash V1.2	yes	4B	14	+0.705	+0.628	as above
Jev 1.14.0	no	not disclosed	21	+0.701	+0.657	not published
Strands Decider 2B	yes	2B	9	+0.536	+0.659	published
Jeff 2B	yes	2B	12	+0.432	+0.683	published

Perplexity Decider Flash, an open 4B model on the same base model as cox-1-3b, scores higher on every task set we can compare, including the five tasks that no model with a published data list trained on (+0.657 against +0.596). The full comparison, with intervals and per-task scores, is at vitund.ai/articles/decision-models-compared.

Speed (one request at a time with a yes/no and a four-option question, bf16, RTX 4090, PyTorch, median): 36 ms for a short text of about 150 tokens, 63 ms at 1k tokens, 200 ms at 4k and 405 ms at 8k. Peak GPU memory is 6.2 GB at 1k tokens and 7.1 GB at 8k in bf16.

Limitations

• World knowledge: Cox judges what a text says, not whether it is true. Questions about the world ("will this forecast come true?") need the evidence supplied in the text.
• Multi-step questions (several facts or rules chained together) are a weak spot; ask the steps as separate questions where you can.
• Exact counting and length thresholds near the boundary: use code.
• Long texts: finding a single fact held up to 64k tokens in our tests; questions about every item in a long text ("has every order shipped?") and combining facts from far-apart places are weak at any length.
• Wording: Cox's answers move with the wording of the question; clear questions with option descriptions work best.
• Languages: trained and evaluated in English.
• Calibration is measured, not guaranteed: fit your automation threshold on your own data.

Training data

Public datasets whose licences permit commercial use, converted into typed questions, and generated data whose labels come from code or from independent teacher models released under Apache-2.0; no labels from the model being trained. Every source was reviewed for licence terms; sources without clear commercial terms were excluded. Training texts that match evaluation texts were removed. Contact details in web-crawled texts were replaced with placeholders.

About 5.5% of the training rows come from datasets under share-alike licences (CC BY-SA); they are marked below.

Public datasets used

Each entry gives the source, how many training rows came from it, its licence, and the citation.

• bard, via Natural Instructions: 3,096 rows, Apache-2.0
  Fulda, Tibbetts, Brown and Wingate (2017), Harvesting Common-sense Navigational Knowledge for Robotics from Uncurated Text Corpora, CoRL (PMLR 78:525-534)
• human_ratings_of_natural_language_generation_outputs, via Natural Instructions: 3,096 rows, CC BY 4.0
  Novikova, Dušek, Cercas Curry and Rieser (2017), Why We Need New Evaluation Metrics for NLG, EMNLP, pp. 2241-2252
• blimp, via Natural Instructions: 3,096 rows, CC BY 4.0
  Warstadt et al. (2020), BLiMP: The Benchmark of Linguistic Minimal Pairs for English, TACL
• casino, via Natural Instructions: 3,096 rows, CC BY 4.0
  Chawla et al. (2021), CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems, NAACL
• atomic, via Natural Instructions: 2,646 rows, CC BY (version not stated; HF allenai/atomic: CC BY 4.0)
  Hwang et al. (2021), COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs, AAAI
• winomt, via Natural Instructions: 2,476 rows, MIT
  Stanovsky, Smith and Zettlemoyer (2019), Evaluating Gender Bias in Machine Translation, ACL
• stereoset, via Natural Instructions: 2,257 rows, CC BY-SA 4.0 (share-alike)
  Nadeem, Bethke and Reddy (2021), StereoSet: Measuring stereotypical bias in pretrained language models, ACL
• cod3s, via Natural Instructions: 2,064 rows, MIT (repo)
  Weir, Sedoc and Van Durme (2020), COD3S: Diverse Generation with Discrete Semantic Signatures, EMNLP
• schema_guided_dstc8, via Natural Instructions: 2,064 rows, CC BY-SA 4.0 (share-alike)
  Rastogi et al. (2020), Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset, AAAI; Lee et al. (2022), SGD-X, AAAI
• ruletaker, via Natural Instructions: 1,548 rows, Apache-2.0
  Clark, Tafjord and Richardson (2020), Transformers as Soft Reasoners over Language, IJCAI
• babi, via Natural Instructions: 1,548 rows, BSD (repo LICENSE) / CC BY 3.0 (HF card)
  Weston et al. (2016), Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks, ICLR
• mathmatics_dataset, via Natural Instructions: 1,548 rows, Apache-2.0
  Saxton et al. (2019), Analysing Mathematical Reasoning Abilities of Neural Models, ICLR
• creak_2018, via Natural Instructions: 1,548 rows, MIT
  Onoe, Zhang, Choi and Durrett (2021), CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge, NeurIPS Datasets and Benchmarks
• wiqa, via Natural Instructions: 1,548 rows, Apache-2.0
  Tandon et al. (2019), WIQA: A dataset for "What if..." reasoning over procedural text, EMNLP
• clinc150: 1,548 rows, CC BY 3.0
  Larson et al. (2019), An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction, EMNLP
• dynahate: 1,499 rows, CC BY 4.0
  Vidgen et al. (2021), Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection, ACL
• wiki_detox: 1,492 rows, labels CC0; text CC BY-SA 3.0 (share-alike)
  Wulczyn, Thain and Dixon (2017), Ex Machina: Personal Attacks Seen at Scale, WWW; text from Wikipedia talk pages
• copa, via Natural Instructions: 1,032 rows, BSD 2-Clause
  Roemmele, Bejan and Gordon (2011), Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning, AAAI Spring Symposium
• defeasible_nli_atomic, via Natural Instructions: 843 rows, MIT (annotations) + upstream CC BY-SA 4.0 text (share-alike)
  Rudinger et al. (2020), Thinking Like a Skeptic: Defeasible Inference in Natural Language, Findings of EMNLP
• coached_conv_pref, via Natural Instructions: 774 rows, CC BY 4.0
  Radlinski et al. (2019), Coached Conversational Preference Elicitation: A Case Study in Understanding Movie Preferences, SIGDIAL
• smcalflow, via Natural Instructions: 774 rows, CC BY-SA 4.0 (data); MIT (code) (share-alike)
  Semantic Machines et al. (2020), Task-Oriented Dialogue as Dataflow Synthesis, TACL
• aegis1: 499 rows, CC BY 4.0
  Ghosh et al. (2024), AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Experts
• oasst_human: 499 rows, Apache 2.0
  Köpf et al. (2023), OpenAssistant Conversations: Democratizing Large Language Model Alignment, NeurIPS Datasets and Benchmarks
• convabuse: 451 rows, CC BY 4.0
  Cercas Curry, Abercrombie and Rieser (2021), ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI, EMNLP
• perspectrum, via Natural Instructions: 281 rows, CC BY-SA (version not stated) (share-alike)
  Chen et al. (2019), Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims, NAACL
• overruling, via Natural Instructions: 193 rows, Apache-2.0 (repo)
  Zheng et al. (2021), When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset, ICAIL
• head_qa, via Natural Instructions: 193 rows, MIT + Spanish Ministry reuse conditions
  Vilares and Gómez-Rodríguez (2019), HEAD-QA: A Healthcare Dataset for Complex Reasoning, ACL
• generated_reviews_enth, via Natural Instructions: 140 rows, CC BY-SA 4.0 (share-alike)
  Lowphansirikul, Polpanumas, Rutherford and Nutanong (2021), A large English-Thai parallel corpus from the web and machine-generated text, Language Resources and Evaluation 56(2):477-499
• deepset_prompt_injections: 97 rows, Apache-2.0
  deepset, prompt-injections dataset (Hugging Face: deepset/prompt-injections)

Natural Instructions (Wang et al., 2022, "Super-NaturalInstructions"; Apache-2.0) supplied the task framing for the sources marked "via Natural Instructions"; the licence shown is the underlying dataset's. Generated families (about 101,000 rows) are Vitund's own, written by Apache-2.0 teacher models or constructed in code.

Citation and contact

@misc{vitund2026cox1,
  title        = {cox-1-3b: a decision model for typed questions about text},
  author       = {{Vitund}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Vitund/cox-1-3b}}
}

Questions, problems and licensing: opensource@vitund.ai. More about Vitund: vitund.ai.