Kapteeni v1 (kapteeni-v1-meticulous) — a Jev-compatible System One decision model

Send a state plus typed questions; get back calibrated probability distributions your code can branch on. No text generation. Kapteeni implements the TypeSafe System One decision-model interface (the wire format of Jev, POST /v1/systemone).

This variant: kapteeni-v1-meticulous — conservative confidence. The safe default for unknown or messy traffic.

Kapteeni v1 ships as two variants of the same architecture. The other, kapteeni-v1-intuit, decides more accurately on well-formed numeric, temporal, and multi-step policy traffic but is less careful about its confidence there.

Quickstart

pip install torch transformers safetensors huggingface_hub   # plus the kapteeni package below
python -m kapteeni.serve --dist . --port 8000        # run from this snapshot
curl localhost:8000/v1/systemone -d '{
  "state": "Help! My payouts have been failing for 3 days.",
  "model": "jev-latest",
  "questions": {
    "is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"},
    "department": {"type": "choice", "instructions": "Which team should handle this?",
      "criteria": {"billing": "Payments, invoicing, refunds",
                    "technical": "Bugs, outages, integrations",
                    "sales": "Pricing, upgrades, new accounts"}},
    "frustration": {"type": "score", "instructions": "How frustrated is the customer?",
      "criteria": ["Calm, just stating facts", "Frustrated but civil", "Very angry"]}
  }}'

Or in-process:

from kapteeni.model import SystemOneModel
m = SystemOneModel.from_dist("<snapshot dir>")
answers, usage = m.evaluate(state, questions)

The kapteeni/ Python package is included in this snapshot; the full repository (data pipeline, training, tests, benchmark history and pre-registrations) lives at https://github.com/TriusAI/Kapteeni (Apache-2.0).

Question types (the Jev contract)

type	answer	readout
noul	{type, noul} — P(yes) in [0,1]	absolute: sigmoid head on the final hidden state; not complement-consistent (P(A) + P(not-A) may differ from 1, matching the reference)
choice	{type, choice, probabilities, confidence}	relative: per-option passes through the backbone, shared head, group softmax; probabilities sum to exactly 1
score	{type, score, legend, probabilities, confidence}	independent levels; score is the probability-weighted expectation and can land between levels

Serving adds a verbalizer blend: the backbone's own next-token yes/no logits are geometrically blended with the trained head (weights in kapteeni-config.json), which measurably improves out-of-domain robustness and calibration. --readout head or --readout verb select pure variants.

Benchmark (JevBench v1.4 public half, self-reported)

231 public decisions, scored with the benchmark's own code, end-to-end through the server. Numbers only — no placement claims; see caveats.

JevBench-style score	Intelligence	top-label ECE	public accuracy
65.71	60.3	0.0496 (Calibration 90.1)	0.710 (easy 1.000 / standard 0.889 / hard 0.469)

Axes common to both variants: Speed 81.0 (p50 0.17 s, p95 1.2 s on an AMD Strix Halo iGPU, x2 self-hosted adjustment) · Cost: 597 input tokens/decision at the hosted list price — verified 2026-09-26: $0.03/M (Novita, qwen3-4b-fp8) = $0.018/1k decisions; nearest official Alibaba tier (qwen-turbo, $0.05/M) = $0.030/1k; an earlier $0.14/M assumption ($0.083/1k) is kept as the pessimistic bound. Under the benchmark's published Jev-class definition (cost <= $0.080/1k decisions, adjusted median latency <= 1.30 s), both variants qualify at verified prices (adjusted p50 0.34 s). Composite = harmonic mean; the Intelligence<50 gate does not apply.

Caveats, stated plainly: self-reported public half (judge and sealed items are private; not an official rank); all serving constants pre-registered on held-out validation (no benchmark selection); Intelligence is renormalized without the sealed judge tier; Calibration shown is the ECE half only. Differences of a few points are within single-seed pipeline noise (~±2-3 composite points, measured).

How it was trained

Qwen3-4B-Instruct-2507 (frozen for the data pipeline, then) LoRA r=32 on all attention and MLP projections for one epoch over an 8.7M-token decision mix: BoolQ + FEVER (teacher-soft-labeled with k=5 sampled agreement), Banking77, CLINC150, GoEmotions, HelpSteer2, and 3.6k synthetic temporal/numeric/policy items with ground truth by construction. Readout heads (2-layer MLPs on the final hidden state) trained with proper scoring rules only. Per-head temperature scaling; blend constants fit on held-out mixed-domain validation. Out-of-domain gate: MNLI excluded from training, held 0.88 -> 0.893 through all 1,205 steps.

Training data provenance

data	license
Qwen3-4B-Instruct-2507 (base)	Apache-2.0
GoEmotions	Apache-2.0
Banking77, HelpSteer2	CC BY 4.0
BoolQ	CC BY 3.0
MultiNLI	CC BY-SA 4.0
FEVER (underlying)	CC BY-SA 3.0
CLINC150	research use (no explicit license)
synthetic items	Apache-2.0 (this project)

Weights: CC BY-SA 4.0 (the ShareAlike terms of MultiNLI/FEVER carry the recommendation; see WEIGHTS-LICENSE.md for the full attribution obligations). Code: Apache-2.0 (LICENSE).

Limitations

• English-primary; long-policy and multi-hop reasoning remain weak (hard-tier accuracy 0.469); temporal/numeric judgment is unreliable on this variant — prefer -intuit for such traffic.
• Probabilities are calibrated in aggregate; individual answers are not guaranteed correct — branch on confidence where it matters.
• Not affiliated with or endorsed by TypeSafe AI; "Jev" is their model and trademark; this is an independent implementation of the documented interface, evaluated on the public JevBench items.