altslate/certo-decision-v2.1

Certo v2.1 (ModernBERT-base) — calibrated decision model

A generic, non-generative decision model: it scores supplied runtime options against a state and returns a calibrated probability distribution (one forward pass, no text generation). ModernBERT-base (~150M), trained 3 epochs on the representative real+synthetic corpus altslate/certo-decisions-v2 (v2.1, 971k), temperature-calibrated (T=1.38).

from certo import DecisionModel
m = DecisionModel.load("altslate/certo-decision-v2.1")
r = m.decide("charged twice, want a refund",
             [{"id": "billing", "description": "charges and payments"},
              {"id": "tech", "description": "app problems"}])

Measured — be honest about where it stands

JevBench public tasks (official certo_local adapter, calibrated):

• accuracy: easy 0.583 / original 0.389 / hard 0.297; by type choice 0.31 / yes-no 0.50 / score 0.50
• best of our small encoders on raw accuracy; cheap (~$0.001/1k decisions) and fast (one forward pass)

Honest caveats:

• Under JevBench v1.3.0 scoring (chance-corrected Intelligence + a near-chance penalty for Intelligence < 50), the composite is low (~3–4) — like all small encoders, because hard-tier accuracy sits near random chance. v1.3.0 structurally favors above-chance generative reasoners.
• Calibration transfers in-distribution but not out-of-distribution: dev ECE ≈ 0.003, yet JevBench (OOD) ECE ≈ 0.14–0.21. Temperature fit on our dev does not fix OOD miscalibration.

Best use

As a calibrated, cheap, open decision component (routing/triage/gating over supplied options), not as a JevBench-v1.3.0 leaderboard entry. Also available as tag v2.1 on altslate/certo-decision-model.

Code/recipe: github.com/AltSlate-Labs/certo.