local-decision-model

A small, fast decision model: unstructured state in, typed probabilistic decisions out. One forward pass of a 150M parameter encoder answers a whole schema of typed questions about a piece of text, typically in 10 to 25 ms on a consumer GPU. It never generates text, so every output is guaranteed to match its declared type.

Code, training recipe, experiments and full results: https://github.com/Pdbz199/local-decision-model

Usage

These weights need the small decisionmodel package from the GitHub repo (they are not a standard transformers head).

pip install git+https://github.com/Pdbz199/local-decision-model

from decisionmodel import DecisionModel, Bool, Enum, MultiLabel, Int, Extract

dm = DecisionModel.load("Pdbz199/local-decision-model")

d = dm.decide("My order #48213 arrived with a cracked screen. Refund me today or I dispute the charge.", {
    "urgent":   Bool("Does this need a response within the hour?"),
    "team":     Enum("Which team should handle this ticket?", ["billing and refunds", "technical support", "sales"]),
    "anger":    Int("How angry is the customer, from 1 (calm) to 5 (furious)?", 1, 5),
    "order_id": Extract("What is the order number?"),
})
print(d.team.value, d.team.confidence, d.urgent.p(), d.order_id.value)

Types: Bool, Enum (2 to 255 options), MultiLabel, Int, Float, Extract (a verbatim substring of the input, or None).

How it works

ModernBERT-base with two small heads. The schema is written into the input as [CLS] [MODE] question [OPT] option 1 [OPT] option 2 ... [SEP] state [SEP]. The hidden state at each [OPT] token is scored, so all options are judged in the same pass; a span head handles extraction. Trained with log loss on about one million examples recast from roughly 50 public datasets, then temperature-calibrated on tasks that were never trained on.

Results on tasks never seen in training

Held-out task	Type	Result
IMDB sentiment	Bool	94.4% accuracy, calibration error 0.016
SQuAD v2 dev	Extract	82.8% exact match, F1 85.8
AG News topics	Enum (4)	74.0%
Twitter financial news sentiment	Enum (3)	74.1%
Banking77 intents	Enum (77)	63.0% top 1, 80.2% top 3; 83.6% on the 48% of calls with confidence of at least 0.9
deepset prompt injections	Bool	69.6% accuracy, AUROC 0.87
SMS spam	Bool	64.9% with a vague question, 89.4% with a precise one (AUROC 0.94 to 0.96)

Latency (RTX 5070 Ti, bfloat16, median): single yes/no 10 ms, 8 typed fields 16 ms, 77 options 11 ms, 255 options 21 ms, 8,000 token input 351 ms.

Limitations

• A small model: good at judgments a person could make at a glance, weak at implied or conditional meaning.
• Sensitive to question wording. Test a schema on a few real examples before trusting it.
• MultiLabel under-predicts when several options are true at once. Ask important tags as their own Bool.
• Confidence is well calibrated on familiar kinds of task and can be overconfident on unfamiliar ones.
• English only. Guardrail decisions can be wrong: do not use it as a sole security boundary.

Licensing

The code is MIT licensed. The base model is Apache 2.0. These weights were trained on public datasets with mixed licenses, some of which restrict commercial use (for example lmsys/toxic-chat, CC BY-NC 4.0, and the Yelp reviews dataset). The weights are therefore released under cc-by-nc-4.0. If you need different terms, retrain with scripts/build_data.py after removing the sources that do not fit your use; every source is listed in that file.

Relation to Jev

This project was inspired by TypeSafe AI's blog post introducing "System One Models" and their model Jev. It is an independent, generic implementation built only from that public post, and is not affiliated with TypeSafe AI.