cortex-agent-llc/kodiak-small-r1-preview

Kodiak small — research preview (R1)

Superseded: see kodiak-small-v2-preview (fewer wrong refusals, better calibration).

Research preview. An early model, shared while we build in public. It's useful and fast, and it also makes mistakes (see Known weaknesses). A better-trained version (new synthetic data, then a larger backbone) is in progress; the full v0.1 release will have a complete model card. Code, docs, evaluation and the whole build story: https://github.com/grizzlypeaksoftware/kodiak Feedback and failure cases are very welcome: open an issue on GitHub.

Kodiak is an encoder-only "System One" decision model by Cortex Agent LLC: a state (text, list of texts, or JSON) plus typed questions in, calibrated answers out, in one forward pass. Choice answers are always one of your labels, scores stay inside your range, and every question can come back as "not answerable from this state" with its own probability.

# pip install "kodiak-s1[infer] @ git+https://github.com/grizzlypeaksoftware/kodiak"
from kodiak_s1.hub import Kodiak

kodiak = Kodiak.from_pretrained("cortex-agent-llc/kodiak-small-r1-preview")   # GPU if available, else CPU
kodiak.decide(
    "I was charged twice for my order and nobody answers my emails!",
    [{"type": "choice", "id": "intent", "text": "What does the customer want?", "labels": ["refund", "track order", "cancel account"]},
     {"type": "score", "id": "urgency", "text": "How urgent is this?", "min": 0, "max": 10}],
)

• Backbone: ModernBERT-base (Apache-2.0); 152M parameters with Kodiak's heads. Run b-small-s1-R1-cap3, step 6000.
• Calibration: temperatures fit on validation data are baked into the weights and listed in calibration.json; the abstain threshold defaults to 0.75 (override per request with null_threshold).
• Speed: ~8 ms per request on a GPU; ~80 ms on 8 ARM CPU cores (DGX Spark).
• Eval (frozen eval set v0.1): overall 0.780, in-domain 0.808, held-out 0.664, ECE 0.049. Against open zero-shot classifiers it leads overall by a wide margin but is slightly behind the best one on never-seen classification tasks (see the repo's docs/STRATEGY.md).
• Known weaknesses: never-seen category-inference tasks (e.g. occupation from a biography); some judgment scores (it rates the urgency of the example above far too low); agent tool routing when the right tool is only implied; states longer than 512 tokens are truncated.
• Hosting: includes handler.py for Hugging Face Inference Endpoints (Deploy → Inference Endpoints; CPU is enough).
• Not for high-stakes decisions about people without human review. Bias in Bios (occupation) is a held-out task with known gender bias.