5ak3t/scm-mid-4b-v0

SCM-mid 4B v0 (experimental)

SCM ("Sovereign Classification Model", repository rev) makes typed, non-generative decisions in one forward pass: choice (pick one of K), noul (yes/no) and score (ordinal level) questions about a state. This repo holds a LoRA adapter only (adapter_model.safetensors) for Qwen/Qwen3-4B-Base at revision 906504ac4b72e133a613011318afd8206fb5e8ec. It is an experimental research release, not a production model.

How it scores (joint-option format)

All options of one question are shown in one prompt as lettered lines; the model's score for an option is the next-token logit of its letter. Several option orders are scored and their per-order log-softmax over the letters is averaged, then renormalised to a probability vector. The exact prompt is l87_joint_prompt.py in this repo:

The following is a multiple choice question. Choose the single best answer.

<instructions>

Question: <state>
A. <option text>
B. <option text>
Answer:

Training used a random option order, LoRA r=16 / alpha=32 on all attention and MLP projections, lr 5e-5 (bf16, non-finite-gradient guard) for 8B/14B and 1e-4 (fp16) for the smaller models, 16 questions per step, one epoch over pool v5 (124,334 records), prompts left-truncated at 1,024 tokens.

Measured results (open selection data; run s88-p4-H100-20261014-110440)

fresh_sel .719, dev .721, zs_td acc .578 (balanced .359), sec_cve_open .425. These are selection-level numbers on the project's own open evaluation sets (scorer meta/final86/k89_score.py), not independent benchmark results. Pre-registered keep rules K1-K3 were not all met by any ladder arm (K3 fails for 4B, 8B, 14B); see docs/rev-large/results-8.9.md in the source repository. Always answering "investigate" scores .713 on zs_td, so its plain accuracy is a weak yardstick (balanced accuracy is given in brackets).

Training data

Pool v5 (124,334 records, 168 workflow groups): public datasets with their own annotations (some of which, e.g. UltraFeedback and HelpSteer, were LLM-annotated by their authors), plus the project's in-house sets from earlier stages: 780 synthetic support-ticket records written by Claude Haiku from deterministic specs (data/haiku-tickets/PROVENANCE.md in the source repo) and 724 hand-specified typed-decision examples. No labels were generated with Claude, GPT or another LLM for the 8.9 pool or for these adapters. Sources and licences as recorded by the builders are in pool-v5-manifest.json / pool-v3-manifest.json (licences are recorded, not gated; check them before any commercial use). No raw training records are in this repo. Evaluation sources (MMLU, ARC, CommonsenseQA, MedMCQA, ScienceQA, ...) were excluded by a reserved list. S1MB test cases were never used.

Limitations

• Accuracy on workflows unlike the training data is far lower than on in-distribution suites (about .55-.81 on unseen knowledge multiple choice depending on size).
• Probabilities are the model's softmax; the conformal coverage guarantees in the source repo hold only near the calibration data, and no abstention is shipped.
• The model reads at most about 1,024 tokens well (trained with left truncation at 1,024).
• Adapter only: the base weights are subject to the Qwen3 licence (Apache-2.0).

Source: https://github.com/codecraf8/rev (branch edge-master), state.csv for the hand-over index.