Argentis-Decision-4B

A 4B decision model that answers when the evidence supports an answer and abstains — citing the missing input — when it does not. Fine-tuned from Qwen3-4B-Instruct-2507 on programmatically labeled decision cases with counterfactual augmentation; evaluated on the frozen argentis-decision-bench protocol.

Version: v1.0-sft (revision-tagged; the main card always describes the tagged release it ships with).

What it does

Given a rule-governed decision case (rules text + case record + evidence inventory), the model emits one JSON object:

{"decision": {"kind": "scalar", "value": 1245.0, "unit": "USD"} ,
 "status": "resolved",
 "evidence": ["bag:E02", "field:customer_class"],
 "rules_applied": ["overweight fee schedule", "overall fee aggregation"]}

On a record that cannot support a decision, it abstains and names what is missing:

{"decision": null,
 "status": "abstain",
 "evidence": ["field:bag_e03_weight"],
 "rules_applied": []}

There is deliberately no confidence field: in a deterministic solver world a self-reported confidence has no supervised target, and this model's abstention is triggered by structural insufficiency of the record, not by low confidence. Calibrated uncertainty is future work tied to domains with proper scoring.

Training recipe

• Base: Qwen/Qwen3-4B-Instruct-2507 @ cdbee75f… (Apache-2.0), bf16.
• Method: QLoRA SFT — nf4-quantized base, LoRA r=16/alpha=32 on all linear layers, one epoch, completion-only loss. This is supervised fine-tuning on solver-generated labels; it is not distillation (no teacher signal of any kind).
• Data: 3,000 prompt/completion examples deterministically derived from the first 1,000 rows of the benchmark's train split — each case plus one seeded remove-required-input abstention variant and one seeded nuisance variant (training-set sha 55ca6d99…). The frozen test split (and every variant of it) was excluded by info-hash deduplication: zero collisions.
• Released weights: the LoRA delta merged into full bf16 weights (adapter sha eceb31ac…; per-shard hashes in the repo).

Results — frozen protocol 4a8363c6…, re-run on the released checkpoint

Pre-registered threshold (Gate C): fine-tuned minus base ≥ ~20 points on the composite. All rows: 1,599 frozen test cases, greedy decoding, 512 max new tokens, scorer s1.1.0, A100-80GB class.

Metric	Base 4B	30B-A3B reference	This model
Valid grounded decisions (composite)	0.0%	0.0%	21.0%
Correct abstention (n/300 unresolvable)	0/300	0/296	298/300
False abstention (n/1,299 resolvable)	0/1,299	0/1,238	0/1,299
Exact decision accuracy	0.0%	0.0%	2.9%
Median decision error	−$1,815	−$1,758	+$50
Spurious dependence	45.5%	54.1%	54.3%

Verification run ID: 20261017T162706Z-s7-cuda-sft_realcf — a clean-instance re-run on 2026-10-17 that reproduced the original evaluation (20260914T202702Z-s7-cuda-sft_realcf) metric-for-metric: identical composite, abstention, and error figures under greedy decoding on the frozen protocol, 33 days apart. Released shard sha256 6e1ca2a638ba44e1….

Read the composite honestly: it is dominated by abstention competence (89% of passing cases); on resolvable cases alone the composite equals exact decision accuracy (2.9%). The controlled claim this model earns is selective abstention: near-perfect refusal on unresolvable records with zero false refusals on resolvable ones — a behavior both the stock 4B and a stock 30B score exactly 0 on.

Known failure modes — do not use it for these

• Fee arithmetic is weak (~3% exact): this checkpoint is a research artifact about abstention behavior, not a production fee calculator.
• Presentation sensitivity is unsolved: its dollar answers move under irrelevant perturbation (reordering, relabeling) at rates comparable to stock models. Do not assume answer stability across paraphrase.
• Abstention is trained on one transform family (remove-required-input in one domain). Generalization of the abstention behavior beyond the generating transform's surface distribution is not yet demonstrated.
• Single domain (airline fees), English, greedy decoding per the protocol.
• Not an intent-detection model; not a general assistant.

Reproduce the evaluation

# from the dataset repo
pip install scoring/argentis_decision-0.6.0-py3-none-any.whl
python -I -m argentis_decision.smoke_s1   # scorer self-check (golden digest e3320025…)
python eval.py prompts --cases data/test.jsonl --rules rules/airline_rules.txt --out prompts.jsonl
# run THIS model over prompts.jsonl (greedy, max_new_tokens=512)
python eval.py score --cases data/test.jsonl --completions completions.jsonl \
  --model-name argentislabs/Argentis-Decision-4B --revision v1.0-sft

Linked artifacts

• Dataset / benchmark: argentislabs/argentis-decision-bench
• Eval pipeline: ships inside the dataset repo (eval.py + self-verifying scorer wheel)
• White paper: WP04, "Scored Abstention: Evidence-Grounded Decisioning in a 4B Model" (forthcoming)

License & provenance

Apache-2.0 (inherited from the Qwen3 base; fine-tune and merge by Argentis Labs). Benchmark cases derive from RuleArena (MIT, commit 59f60f6…).