Chon-1.0-12B

Chon-1.0-12B is a typed-decision model. Given a state (text, JSON or a conversation) and one question with a closed set of options, it returns a calibrated probability for every option:

• choice: pick one of 2-256 labelled options (more than 16 are read in groups, then a final round)
• yes/no: P(yes)
• score: a distribution over ordered levels

It is a LoRA adapter on google/gemma-4-12B-it (12B) and serves the TypeSafe /v1/systemone wire format used by JevBench.

Serving code, full guide and Docker image: github.com/xoai/chon (MIT). The commands below are enough to run it.

Base model	google/gemma-4-12B-it, unmodified, loaded alongside this adapter
Adapter	LoRA (~1.0 GB)
Question types	choice (2-256 options), yes/no, score
Input	state up to 6,000 tokens (longer states are truncated, keeping the start); prompt up to 8,192 tokens
Languages	English; Vietnamese also supported
Hardware	one GPU with 40 GB or more for bf16 (measured on one H100 80 GB)
Licence	Apache-2.0, as the base model

Use

pip install "chon[serve] @ git+https://github.com/xoai/chon@v1.1.1"
python -m chon.serve --model ninhnv/chon-1.0-12b --port 8000      # downloads the base model and this adapter

The first start downloads the base model, google/gemma-4-12B-it (public, no token needed), into the Hugging Face cache. --device defaults to auto: an NVIDIA GPU if there is one, else Apple's GPU, else the CPU.

curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.",
  "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.",
    "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}
}'

The response holds answers.decision with choice and probabilities for choice, noul (P(yes); also as probabilities keyed true / false) for yes/no, or probabilities over the levels for score. usage reports input_tokens and output_tokens: most decisions are a single forward pass with no output tokens, while some take a short extra pass whose generated tokens are counted in output_tokens and included in the cost figures below.

Python

from chon.infer import DecisionModel

dm = DecisionModel.load("ninhnv/chon-1.0-12b", device="auto")   # NVIDIA GPU, else Apple's GPU, else CPU
answers, usage = dm.predict(
    "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.",
    {"damaged": {"type": "noul", "instructions": "Did the order arrive damaged?"}})
print(answers["damaged"]["noul"])   # P(yes)

When option labels are only ids ("0", "1", ...), add "option_keys": "hide" to the question so the model sees only the option descriptions.

Faster reasoning (vLLM)

On questions it is unsure about, the model first writes up to 512 tokens of reasoning with the plain base model, then reads its answer. chon.serve hands that writing step to a vLLM server of google/gemma-4-12B-it (FP8 weights) when one can run: it starts vllm serve itself (from $CHON_VLLM_BIN, vllm on the PATH or /opt/vllm-env/bin/vllm) on a free local port, with the GPU memory left after loading this model, or it uses $CHON_VLLM_URL if you run the server yourself. Without vLLM, or with too little free memory, it uses a static KV cache instead (slower than vLLM, no second copy of the weights). The answer is always read by this model. On a machine that has not run it before, vLLM compiles its kernels at start-up, so the server takes about 5 minutes before /health answers (2 minutes once the compiled kernels are cached; up to 20 on a slow disk).

vLLM goes in its own environment (it brings its own torch):

python -m venv /opt/vllm-env && /opt/vllm-env/bin/pip install uv
/opt/vllm-env/bin/uv pip install --python /opt/vllm-env/bin/python "vllm==0.32.0" --torch-backend=auto

or use the Docker image in the code repository (docker/Dockerfile.serve), which has both environments.

Changes

22 Oct 2026. Same adapter and calibration as the first release (revision cf3c0d3), two serving changes: the reasoning on unsure questions is written by vLLM with FP8 weights instead of plain transformers generation, and the "unsure" thresholds are lower, so about 11% of questions take that path instead of 20%. Measured on one H100 80 GB: p95 response 24.3 s -> 4.8 s, mean 5.0 s -> 0.6 s, cost $0.047 -> $0.042 per 1,000 decisions; held-out capability 86.7 -> 84.9, public 85.8 -> 84.7. Needs chon 1.1.0 or later for the vLLM path (older versions serve this package with plain generation).

What is in this repository

File	What it is
adapter_model.safetensors, adapter_config.json	the LoRA adapter (PEFT)
chon_config.json	serving settings (prompt and calibration) read by the chon server
tokenizer files	the base model's tokenizer and chat template, unchanged
README.md	this card

Evaluation (our measurements, not official board numbers)

set	Intelligence	Calibration (proxy)	Capability (proxy)
held-out replica (720)	79.2	90.6	84.9
JevBench public (231)	81.0	88.5	84.7

Serving, measured on one H100 80 GB, one request at a time: median response 0.11 s, p95 4.8 s; 744 input and 51 output tokens per decision on average, about $0.042 per 1,000 decisions at $0.05 / $0.1 per million input / output tokens.

Official JevBench harness (typesafe adapter, public files) against the served package, accuracy: easy 100.0%, original 98.6%, hard 84.7%.

• Held-out set: 720 items in the JevBench v1.5 sealed-set mix (Choice 360 / yes/no 180 / score 180; tiers 14/28/28/30%), written for this project and never trained on.
• Scoring: the board's published formulas where they are public (equal type weights, tier weights 10/20/30/40, yes/no answers inside 0.2-0.8 count as wrong). The board's calibration mapping is not fully published, so the calibration column is our proxy.

Licence

Apache-2.0, as google/gemma-4-12B-it.