5ak3t/scm-max-14b-v0

SCM-max 14B v0 (experimental)

SCM ("Sovereign Classification Model", repository rev) makes typed, non-generative decisions in one forward pass: choice (pick one of K), noul (yes/no) and score (ordinal level) questions about a state. This repo holds a LoRA adapter only (adapter_model.safetensors) for Qwen/Qwen3-14B-Base at revision 0b0bd3732e2c374d483664439ea334928b65f304. It is an experimental research release, not a production model.

How it scores (joint-option format)

All options of one question are shown in one prompt as lettered lines; the model's score for an option is the next-token logit of its letter. Several option orders are scored and their per-order log-softmax over the letters is averaged, then renormalised to a probability vector. The exact prompt is l87_joint_prompt.py in this repo:

The following is a multiple choice question. Choose the single best answer.

<instructions>

Question: <state>
A. <option text>
B. <option text>
Answer:

Training used a random option order, LoRA r=16 / alpha=32 on all attention and MLP projections, lr 5e-5 (bf16, non-finite-gradient guard) for 8B/14B and 1e-4 (fp16) for the smaller models, 16 questions per step, one epoch over pool v5 (124,334 records), prompts left-truncated at 1,024 tokens.

Measured results (open selection data; run s88-p14c-H100x2-20261014-162050)

fresh_sel .810, dev .764, zs_td acc .645 (balanced .540), sec_cve_open .598. These are selection-level numbers on the project's own open evaluation sets (scorer meta/final86/k89_score.py), not independent benchmark results. Pre-registered keep rules K1-K3 were not all met by any ladder arm (K3 fails for 4B, 8B, 14B); see docs/rev-large/results-8.9.md in the source repository. Always answering "investigate" scores .713 on zs_td, so its plain accuracy is a weak yardstick (balanced accuracy is given in brackets).

Training data

Pool v5 (124,334 records, 168 workflow groups): public datasets with their own annotations (some of which, e.g. UltraFeedback and HelpSteer, were LLM-annotated by their authors), plus the project's in-house sets from earlier stages: 780 synthetic support-ticket records written by Claude Haiku from deterministic specs (data/haiku-tickets/PROVENANCE.md in the source repo) and 724 records from the train split of LocalLLaMA/typed-decisions (Apache-2.0). No labels were generated with Claude, GPT or another LLM for the 8.9 pool or for these adapters. Sources and licences as recorded by the builders are in pool-v5-manifest.json / pool-v3-manifest.json (licences are recorded, not gated; check them before any commercial use). No raw training records are in this repo. Evaluation sources (MMLU, ARC, CommonsenseQA, MedMCQA, ScienceQA, ...) were excluded by a reserved list. We did not use S1MB test splits for training, prompts or selection. An input-only audit of the final training pool against the S1MB English test inputs (s1mb-overlap-report-v2.json, which replaces the earlier s1mb-overlap-report.json) finds 0 exact matches and 41 approximate-match candidates (20 near, 21 weaker pooled; 30 of them are the shared 100 inputs of the three typed-decisions benchmarks). These are candidates, not confirmed use of evaluation inputs, and test targets were not compared. Several S1MB benchmarks share a source dataset with the training pool (train splits); that is declared in the results submission, and what cannot be audited (for example Qwen3 pretraining data) is unknown.

Limitations

• Accuracy on workflows unlike the training data is far lower than on in-distribution suites (about .55-.81 on unseen knowledge multiple choice depending on size).
• Probabilities are the model's softmax; the conformal coverage guarantees in the source repo hold only near the calibration data, and no abstention is shipped.
• The model reads at most about 1,024 tokens well (trained with left truncation at 1,024).
• Adapter only: the base weights are subject to the Qwen3 licence (Apache-2.0).

Source: https://github.com/codecraf8/rev (branch edge-master), state.csv for the hand-over index.