Model Card for ThatsNinja/glm-edge-falsifier-1.5b

A falsification-gated abstention model built on a frozen GLM-Edge 1.5B Chat base. The model generates multiple counterfactual narratives, a calibrated verifier discards those unlikely to hold, and it commits only to survivors — abstaining ("I don't know") when none remain. Block-sparse attention and latent KV compression target small-hardware efficiency.

Model Details

Model Description

GLM-Edge-Falsifier-1.5B is not a fine-tune of the base weights. The base GLM-Edge 1.5B Chat is kept frozen; the model is an inference-time architecture wrapped around it, consisting of:

1. Falsification fast loop (RFC0013) — at inference, $N$ candidate narratives are sampled (temperature 0.8, top-$p$ 0.95). Each is scored by two channels: a consistency channel (semantic clustering across samples) and a verifier channel (a calibrated MLP over on-policy attention features). A Popperian max-combination yields a combined falsification score $\phi(c)$. The surviving set is $S = {c : \phi(c) < \tau}$. If $S \neq \emptyset$, the model commits to $\arg\max_{c \in S} \hat{u}(c)$; otherwise it abstains ("I don't know") rather than guessing.
2. Correctness verifier (RFC0002/RFC0012) — a small MLP trained on on-policy features extracted from the base model's own generations, with truth-anchored gold-answer-match labels (GSM8K). Calibrated with temperature scaling (Guo et al. 2017).
3. Learned deactivation slow loop (RFC0013) — per-region Beta posteriors on adverse-outcome rates drive a block-sparse attention mask, deactivating attention regions once the posterior is confident and well-observed ($n \ge n_{\min}$). Implemented and self-test passing, but not yet wired to the HIP kernel for an end-to-end generation run — no generation results in this card use learned deactivation.
4. Block-sparse SDPA kernel (RFC0001/0003) — a custom HIP kernel (BLOCK_M=BLOCK_N=32, HEAD_DIM=128) implementing online-softmax block-sparse attention, validated against a CPU reference (max error 5.4e-7 at S=64/128). A monkey-patch hooks it into GLM-Edge's attention for experimentation, but no results in this card use the custom kernel — all generation results use PyTorch native SDPA. GQA-aware (16 Q heads, 4 KV heads). BF16 integration is in progress.
5. Latent KV compression (RFC0005, MLA-style) — down/up-projection adapters ($W_{kv_a}$, $W_{kv_b}$, RMSNorm) compress KV cache to a rank-512 latent. Projections trained, quality target met (eval cosine 0.9547, 2× KV compression).
6. FP8 block quantization (RFC0006) — per-128-block e4m3 quantization for weights and KV cache. Path A (Python dequant) validated; in-kernel dequant (Path B) is future work.
7. Process reward model (RFC0014/RFC0017) — a GRU-based step-level verifier (~15K params) trained on Math-Shepherd-style Monte-Carlo step labels (1000 problems, 7064 steps), replacing the token-level ORM verifier for hierarchical success measurement. Step-level ECE 0.0269 — breaks the 0.05 floor at the level N* stopping and tree search operate at. Includes a task head trained on solution_correct (task-level ECE 0.1048, roughly at parity with the ORM's 0.0910 task-level baseline). Note (2026-08-10): the 30-dim enriched pooling variant (mean+max+std, 30 features) was tested and produced worse calibration (step ECE 0.0339, task head ECE 0.1498) despite +1.4pp step accuracy — evidence that the precision-cost curve is U-shaped and we're past the optimal measurement spend $m^*$ at the step level. See the measurement-cost model (docs/theory/measurement_cost_model.md).
8. Conformal trajectory calibration (RFC0016) — split conformal over PRM step scores delivers distribution-free coverage guarantees: per-step coverage 0.9622 and trajectory coverage 0.9452 (target 0.90, α=0.1).
9. Falsification-guided tree search (RFC0015) — PRM-guided beam search with backtracking. Beats flat best-of-N at matched compute on hard domains: 61.7% vs 47.9% on GSM8K (n=100, +13.8pp). Ties on the easy domain SVAMP (71.4% vs 71.4%) — search earns its compute where the base model is weak (per Qin et al. 2025).
10. Live step-gated generation (RFC0018 draft) — TRUE live inference-time gating (steer_generate.py): KV-cached incremental decode; the PRM + conformal threshold scores each completed step and stops decoding immediately on a fired gate, so tokens after the gate never compute. Dev slice (n=10): the tau 0.45 gate saves 31% of tokens at 0pp accuracy loss; the conformal gate adds +10pp accuracy while saving 9.2% of tokens. Supersedes the post-hoc N* stopping mode, which computed all tokens and only truncated afterwards.

• Developed by: Jonah Sullivan (ThatsNinja)
• Funded by [optional]: Self-funded
• Shared by [optional]: ThatsNinja
• Model type: Decoder-only transformer with inference-time falsification gating and abstention
• Language(s) (NLP): English
• License: MIT for the falsifier architecture and code. The base GLM-Edge 1.5B Chat weights retain the GLM-4 model license.
• Finetuned from model [optional]: Not fine-tuned — wraps a frozen zai-org/glm-edge-1.5b-chat base. Only small adapter layers (MLA projections, verifier MLP) are trained.

Model Sources [optional]

• Repository: TheRock / correctness_model
• RFCs: correctness_model/docs/rfcs/ (RFC0001–RFC0018)
• Results: correctness_model/docs/RESULTS.md
• Demo [optional]: N/A (research prototype)

Uses

Direct Use

Research on falsification-based generation, calibrated abstention, and verifier-guided decoding. The model is not a general-purpose chatbot — it is optimized for correctness over fluency, trading coverage for precision. On the full GSM8K test set (n=1319) it commits to 53.8% of problems and abstains on 46.2%, achieving 68.6% accuracy on what it commits to (vs 36.0% greedy baseline).

Downstream Use [optional]

• As a verifier-augmented inference harness for other frozen base models (the architecture is base-model-agnostic in principle; in practice the feature extractor hooks are written for GLM-Edge's layer structure and would need adaptation for other model families).
• As a calibrated abstention layer for high-stakes QA where "I don't know" is preferable to a confident wrong answer.

Out-of-Scope Use

• General open-ended chat (the model is tuned for single-answer tasks with checkable correctness, not conversation).
• Tasks without a notion of answer correctness (creative writing, summarization quality is unevaluated).
• Safety-critical decisions without a human in the loop — abstention is structural, not a safety guarantee.
• Domains beyond grade-school math are only partially verified: SVAMP transfer is measured (committed 77.2%), but broader domains are not.

Bias, Risks, and Limitations

• Verifier overfitting: Trained on 1000 GSM8K problems with greedy generation only. The token-level ECE floor is ~0.06 (best sweep config 0.0597); the gap was structural at the token level and was addressed by hierarchical measurement (RFC0017) at the step level, not by tuning. The verifier likely overfits to GSM8K arithmetic patterns.
• Abstention is learned, not principled: The 46.2% abstention rate is an emergent property of $\tau=0.6$ and the verifier's calibration, not a guarantee of epistemic uncertainty. Conformal calibration (RFC0016) provides a distribution-free coverage guarantee for step/trajectory commit decisions; the raw abstention rate does not.
• Kernel is fp32, model is BF16: The HIP kernel is validated in fp32; BF16 integration is in progress. Kernel-integrated results use PyTorch native SDPA, not the custom kernel.
• Attention features require eager attention: GLM-Edge's default SDPA attention implementation does not expose attention weights (output_attentions=True returns an empty tuple). Feature extraction loads the base with attn_implementation='eager'.
• PRM task-level parity: The step-level verifier (RFC0014) is trained on 1000 problems and reaches step-level ECE 0.0269 (< 0.05). At the task level its ECE (0.1048) is at rough parity with the ORM baseline (0.0910), so the ORM verifier remains the production task-level channel for now.
• N* stopping result reclassified (2026-09-17): the earlier ~10% token reduction at ≤2pp accuracy loss was post-hoc accounting — generate_with_step_stopping() decoded all tokens and only truncated afterwards, so no compute was saved. The live gate (RFC0018, steer_generate.py) is the corrected mechanism: dev-slice (n=10) it saves 31% of tokens at 0pp accuracy loss (tau 0.45). Step discrimination (step accuracy 0.70) remains the binding constraint on how aggressive the gate can be.
• Measurement-cost model is a hypothesis: the claim that the precision-cost curve $\varepsilon(m)$ is U-shaped with a finite optimal measurement spend $m^*$ rests on one supporting data point (the 30-dim enriched-pooling negative result). It is a research direction, not an established result.
• Single-dataset evaluation: All metrics are on GSM8K (plus SVAMP transfer). MATH/MMLU adapters exist but no cross-domain training or evaluation on them has been conducted yet.
• Base model biases inherited: As a frozen-base wrapper, all biases, hallucination modes, and limitations of GLM-Edge 1.5B Chat are inherited unmodified.

Recommendations

Users (both direct and downstream) should be made aware that:

• The model abstains by design — empty output or "I don't know" is a first-class outcome, not an error.
• Committed answers are substantially more likely to be correct than greedy decoding (68.6% vs 36.0% on the full GSM8K test set), but this is measured only on GSM8K.
• The verifier should be retrained on the target domain before use outside grade-school math.
• For calibrated uncertainty, use the conformal channel (RFC0016), not the raw abstention rate; per-step coverage is 0.9622 at α=0.1 by construction.

How to Get Started with the Model

# 1. Install (ROCm / AMD Instinct) — use the STABLE rocm6.4 index.
#    The nightly multi-arch index installs a wheel that reports gfx942 in
#    get_arch_list() but fails every GPU op with hipErrorInvalidImage.
pip install torch torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/rocm6.4
pip install -r correctness_model/requirements-rocm.txt

# 2. Download the frozen base model
python -c "from huggingface_hub import snapshot_download; \
  snapshot_download('zai-org/glm-edge-1.5b-chat', \
  local_dir='correctness_model/models/glm-edge-1.5b-chat')"

# 3. Run falsification-based generation with the trained verifier
python correctness_model/src/python/falsify_generate.py \
  --gsm8k --num-problems 50 --num-samples 6 \
  --tau 0.6 --verifier correctness_model/checkpoints/verifier-v2-1k.pt

For the full training pipeline (MLA projections, verifier, feature collection) and remote AMD Instinct setup, see correctness_model/scripts/setup_remote.sh.

Training Details

Training Data

Verifier (RFC0002/RFC0012):

• Dataset: GSM8K train split (Cobbe et al. 2021), 1000 problems
• Labels: Truth-anchored gold-answer match (Track 1, RFC0012). The model generates greedily, the predicted answer is extracted and compared to the gold #### <number> answer. Label is outcome-based (ORM-style), broadcast across the generated sequence.
• Features: On-policy — extracted from the model's own generations via attention hooks (per-head attention entropy, output stats, cross-head consistency), closing the teacher-forcing distribution gap. Requires attn_implementation='eager' (SDPA does not expose attention weights).
• Cached as: Parquet in correctness_model/data/gsm8k_features_1k/

Process reward model / step verifier (RFC0014/RFC0017):

• Dataset: GSM8K, step-level labels via Math-Shepherd-style Monte-Carlo Hard Estimation (N=8 completions per step). 1000 problems collected (7064 steps, 63.2% positive).
• Features: Per-step mean-pooled token features + step metadata (12-dim step embedding), extracted with attn_implementation='eager'.
• Cached as: Parquet in data/step_labels/ (labels + features).

MLA projections (RFC0005):

• Dataset: GSM8K train prompts (in-distribution)
• Objective: Reconstruction MSE — compress each layer's projected K/V to a rank-$r$ latent and reconstruct; minimize $|K - \hat{K}|^2 + |V - \hat{V}|^2$. Base model frozen; only $W_{kv_a}$, $W_{kv_b}$, RMSNorm trained.
• Status: Trained at ranks 64/128/256/512. Rank 512 meets the quality target (eval cosine 0.9547 > 0.95, 2× KV compression).

Training Procedure

Preprocessing [optional]

• GSM8K gold answers extracted via #### <number> regex, normalized (strip commas, $, %, canonicalize numerics).
• Prompts built with the GLM-Edge chat template (build_prompt).
• Features normalized with FeatureNormalizer (z-score) fitted on the training split only.

Training Hyperparameters

Verifier MLP:

• Architecture: MLP, hidden_dim 256 (best sweep config), 9 input features: attention_entropy, attention_max_weight, output_variance, output_saturation, output_magnitude, cross_head_consistency, hidden_norm, token_entropy, token_confidence
• Loss: Binary cross-entropy
• Optimizer: Adam, lr 3e-3
• Epochs: 50
• Split: Sequence-level 70/15/15 (no token leakage between splits)
• Calibration: Temperature scaling (Guo et al. 2017), fitted on validation logits, evaluated on held-out test
• Training regime: BF16 mixed precision (base model), fp32 (verifier MLP)

Step verifier / PRM (RFC0014):

• Architecture: GRU, input_dim 12, hidden_dim 256, 2 layers (~15K params)
• Loss: Binary cross-entropy over per-step logits (masked padding)
• Optimizer: Adam, lr 1e-3
• Epochs: 200
• Split: Problem-level 70/15/15 (no step leakage between splits)
• Task composition: a task head (Linear on GRU final state) trained directly on solution_correct outperforms both geometric-mean and min composition of step probabilities for task-level scoring (task head ECE 0.1048 vs geomean 0.1284 vs min 0.1203 on 1000 problems). The geomean + temperature scaling variant was the best composition-based approach in the 200-problem pilot but is superseded by the task head at scale.

MLA projections:

• Architecture: Per-layer $W_{kv_a}: 1024 \to 512$, RMSNorm, $W_{kv_b}: 512 \to 1024$
• Loss: Reconstruction MSE (K + V)
• Optimizer: AdamW, lr 1e-3
• Init: $W_{kv_b}$ small-normal (std 0.02) for near-zero initial reconstruction
• Training regime: BF16 base (frozen), fp32 loss accumulation

Speeds, Sizes, Times [optional]

Component	Size	Notes
Base model (frozen)	2.945 GB (BF16)	GLM-Edge 1.5B, 1.472B params
Verifier MLP	~0.3 MB	9-dim input, hidden 256 (best sweep config)
Step verifier (PRM)	~60 KB	GRU, 12-dim input, hidden 256, 2 layers
MLA projections (28 layers)	~27 MB	rank-512, per-layer adapters (quality target met)
KV cache (full, per 1K tokens)	~0.054 GB	28 layers × 4 KV heads × 128 head_dim × 2 bytes (BF16) × 2 (K+V)
KV cache (MLA rank-512, per 1K tokens)	~0.027 GB	2× reduction vs full KV (stores rank-512 latent instead of separate K+V; validated, cosine 0.9547)
FP8 weight quant	1.45 GB	2× reduction vs BF16 weights (validated Path A)

Evaluation

Testing Data, Factors & Metrics

Testing Data

• GSM8K test split, full set (n=1319) for falsification evaluation; 150 sequences (39,690 tokens) held out for verifier test metrics.
• SVAMP (ChilleD/SVAMP, n=300) for cross-domain transfer.
• No external benchmarks (MMLU, TruthfulQA) evaluated yet.

Factors

• Commit vs abstain: The primary factor is whether the model commits or abstains, driven by the threshold $\tau$ and the surviving-set cardinality.
• Number of samples $N$: 6 narratives per problem (temperature 0.8, top-$p$ 0.95).
• Threshold $\tau$: 0.6 (the falsification cutoff).

Metrics

• Committed accuracy: accuracy over problems the model commits to (the precision-oriented metric this model optimizes for).
• Abstention rate: fraction of problems where the surviving set is empty.
• Greedy baseline accuracy: single greedy decode accuracy (control).
• Verifier F1 / ECE: held-out token-level F1 and expected calibration error for the verifier channel.

Results

Verifier (1000-problem, held-out test, 150 sequences, 39,690 tokens)

Metric	Value
Accuracy	0.7171
Precision	0.8021
Recall	0.4837
F1	0.6035
ECE	0.0893
Fitted temperature	0.990

Scaling training data from 200 to 1000 problems halved ECE (0.18 → 0.089) and raised precision to 0.80. F1 of 0.60 on a 41%-positive task is well above the 0.41 chance baseline. A subsequent hyperparameter sweep improved ECE further to 0.0597 (best config: hidden_dim 256, lr 3e-3, 50 epochs).

Falsification-based generation (full GSM8K test set, n=1319, $N=6$, $\tau=0.6$)

Metric	Value
Committed	710 (53.8%)
Abstained	609 (46.2%)
Committed accuracy	68.6%
Greedy baseline	36.0%

Committed accuracy beats the greedy baseline by 32 percentage points on the full test set. These results use the token-level ORM verifier (verifier-v2-1k.pt, ECE 0.0893) as the verifier channel — not the step-level PRM (which was trained later and has not been wired into falsify_generate.py). The consistency channel uses semantic clustering across the 6 samples. The combined falsification score is $\phi(c) = \max(\phi_{\text{verifier}}, \phi_{\text{consistency}})$; a candidate survives if $\phi(c) < 0.6$.

Component validation

Component	Metric	Result	Status
Block-sparse SDPA kernel	Max error vs CPU ref (S=64/128)	5.4e-7	✅ validated
FP8 weight quant (Path A)	Cosine sim vs BF16	0.9990	✅ math validated
FP8 weight quant (Path A)	Weight rel-err	2.25%	✅
FP8 weight quant (Path A)	Same top-1 prediction	yes	✅
FP8 KV cache (Path A)	Round-trip cosine sim	0.9996	✅
MLA projections (rank 512)	Eval cosine sim vs full attn	0.9547	✅ quality target met (>0.95)

Process reward model / step verifier (RFC0014/RFC0017)

Trained on 1000 problems of Math-Shepherd-style Monte-Carlo step labels (7064 steps, 63.2% positive). Task label is solution_correct (balanced) — the fair ORM-comparable label.

Metric	Value
Step-level accuracy	0.7005
Step-level ECE	0.0269 ✅ (breaks the 0.05 floor)
Task-level ECE (task head)	0.1048
Task head accuracy	0.7365
ORM task-level baseline (mean-of-tokens)	0.0910

RFC0017 is validated at the step level (ECE 0.0269 < 0.05) — the level that N* stopping and tree search operate at. At the task level the PRM is at rough parity with the ORM (0.1048 vs 0.0910); closing that gap is open work. Note the earlier "0.0597 ORM floor" was a token-level metric, not comparable to task-level ECE.

Summary

Falsification gating lifts committed accuracy from 36.0% to 68.6% on the full GSM8K test set (n=1319, $N=6$, $\tau=0.6$) with structural abstention, and transfers cross-domain to SVAMP (committed 77.2%, n=300). These results use the token-level ORM verifier (verifier-v2-1k.pt); the step-level PRM has not yet been wired into the falsification loop. Scaling the verifier from 200 to 1000 problems plus a hyperparameter sweep brought token-level ECE from 0.18 to 0.0597, and MLA projections meet the quality target at rank 512 (eval cosine 0.9547).

The token-level ECE has hit a structural floor at ~0.06: calibration-side levers (hyperparameters, richer calibrators) are exhausted. The remaining gap is on the feature/supervision side. RFC0017 (Hierarchical Success Measurement) addresses this by dividing success measurement across token/step/task levels; the step-level PRM reaches step-level ECE 0.0269 (below 0.05) but task-level ECE 0.1048 (at parity with the ORM baseline 0.0910, not yet better). The exquisite-token stopping rule ($N^*$, RFC0010) is not yet implemented in live generation — it is blocked on trustworthy calibration, and the step-level N* threshold sweep did not meet the combined target (≥20% token reduction AND ≤2pp accuracy loss; best result ~10% reduction at ≤2pp loss). Remaining hardware work: kernel integration into the model (BF16) and FP8 Path B (in-kernel dequant).

Model Examination [optional]

• Per-head attention entropy and cross-head consistency are among the 9 verifier features; their distributions differ between correct and incorrect generations (the basis for the F1 0.60 signal on the 1000-problem verifier, or 0.65 for the best sweep config). Note: these features require attn_implementation='eager' — GLM-Edge's default SDPA implementation returns an empty attention tuple.
• Temperature scaling fits a temperature near identity (0.990), and richer calibrators (vector, histogram) overfit — the ~0.06 ECE floor is structural (feature/supervision side), motivating RFC0017's hierarchical success measurement.
• Step-level measurement (RFC0017): the GRU step verifier trained on 1000 problems reaches step-level ECE 0.0269 — below the 0.05 floor, confirming that step-level labels + sequential context break the floor at the step level. Its task head (trained on solution_correct) reaches task-level ECE 0.1048, at rough parity with the ORM's 0.0910.
• Learned deactivation builds Beta posteriors per attention region; the self-test passes but the mask has not been wired to the HIP kernel for an end-to-end generation run.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: AMD Radeon RX 6900 XT (gfx1030, 16GB GDDR6, TDP 300W) for development; AMD Instinct MI300X (gfx942, 192GB HBM) for remote training.
• Hours used: Development: ~20 hrs local GPU (estimate). Remote (MI300X): ~12 hrs (estimate; MLA rank sweep, verifier scaling + sweep, falsification evals, step-label collection). Step-label collection alone was ~5.1 hrs for 1000 problems (306.5 min); total remote time is an estimate and may be higher.
• Cloud Provider: DigitalOcean (AMD Instinct GPU droplets).
• Compute Region: NYC2.
• Carbon Emitted: [More Information Needed] — the base model is frozen and only small adapters are trained, so the marginal training carbon is minimal compared to a full fine-tune. The dominant cost is Monte-Carlo step-label generation (inference, not training). Hour estimates are approximate and should be treated as order-of-magnitude.

Technical Specifications [optional]

Model Architecture and Objective

Base: GLM-Edge 1.5B Chat — 28 layers, hidden_size 2048, 16 query heads, 4 KV heads (GQA 4:1), head_dim 128, BF16, RoPE positional encoding.

Falsifier wrapper (the contribution):

The exquisite-token stopping rule (RFC0010) sets $N^*$ where marginal token utility equals average utility — past $N^*$, each token destroys value. This is the economic framing; mid-sequence early stopping at $N^*$ is future work (blocked on ECE < 0.05).

Compute Infrastructure

Hardware

• Development: AMD RX 6900 XT (gfx1030, RDNA2, 16GB GDDR6)
• Training: AMD Instinct MI300X (gfx942, CDNA3, 192GB HBM3) via DigitalOcean GPU Droplets

Software

• PyTorch 2.9.1+rocm6.4 (MI300X) / 2.12.0+rocm7.15 (gfx1030 local)
• transformers 5.15.0.dev0 (from source)
• accelerate 1.14.0, datasets 5.0.1
• HIP/ROCm (block-sparse kernel), CMake + Ninja build
• Python 3.11+

Citation

This model

@misc{glm-edge-falsifier-1.5b,
  author       = {Jonah Sullivan},
  title        = {GLM-Edge-Falsifier-1.5B: Falsification-Gated Abstention},
  year         = {2026},
  note         = {Research prototype},
  url          = {https://huggingface.co/ThatsNinja/glm-edge-falsifier-1.5b}
}

Prior work this build depends on

• Cobbe et al. 2021 — Training Verifiers to Solve Math Word Problems (arXiv:2110.14168). GSM8K; ORM-style outcome-supervised verification.
• Wang et al. 2023 — Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations (arXiv:2312.08935). Monte-Carlo Hard Estimation step labels (our H2 labeler).
• Lightman et al. 2023 — Let's Verify Step by Step (arXiv:2305.20050). Process supervision at the step level (PRM800K); motivates step-level credit assignment.
• Uesato et al. 2022 — Solving math word problems with process- and outcome-based feedback (arXiv:2211.14275). Early PRM vs ORM comparison.
• Guo et al. 2017 — On Calibration of Modern Neural Networks (arXiv:1706.04599). Temperature scaling.
• Gibbs & Candès 2021 — Adaptive Conformal Inference Under Distribution Shift (arXiv:2106.00170). Online conformal calibration (RFC0016 Layer 3).
• Angelopoulos et al. 2024 — Conformal Risk Control (arXiv:2209.07411) and related conformal prediction literature. Coverage-guarantee framework.
• Qin et al. 2025 — Tree Search for Language Model Agents (backtracking is not universally beneficial). Per-task justification for RFC0015 tree search; flat best-of-N as mandatory control.
• DeepSeek-AI 2024 — DeepSeek-V2 (arXiv:2405.04434) and DeepSeek-V3 (arXiv:2412.19437). Multi-head Latent Attention (MLA), basis for RFC0005 latent KV compression.
• Zelikman et al. 2022 — STaR: Bootstrapping Reasoning With Reasoning (arXiv:2203.14465). Self-improvement via generated rationales; related to our on-policy self-labeled collection.
• LeCun 2022 — A Path Towards Autonomous Machine Intelligence (JEPA / world-model architecture). Evaluated as a potential complement (latent-prediction scoring channel); not used in this build.

Base model

GLM-Edge 1.5B Chat (zai-org/glm-edge-1.5b-chat) — frozen base, GLM-4 model license. Not modified by this work.

Glossary [optional]

• Falsification (Popperian): A narrative is discarded if any channel marks it as unlikely to hold (max-combination), mirroring Popper's asymmetry — you can refute but not confirm.
• Surviving set $S$: The candidates that pass falsification at threshold $\tau$; the model commits to the highest-utility member.
• Abstention: Structural refusal to answer when $S = \emptyset$; a first-class output, not an error.
• Exquisite tokens (RFC0010): Tokens worth emitting — the optimal stopping point $N^*$ where marginal utility equals average utility.
• MLA (Multi-head Latent Attention): Compresses per-head K/V into a low-rank latent via down/up-projections, reducing KV cache memory (DeepSeek-V2/V3 architecture).
• FP8 (e4m3): 8-bit float format (1 sign, 4 exponent, 3 mantissa bits; max 448.0), used with per-128-block scaling for weight/KV quantization.
• GQA: Grouped-Query Attention — 16 query heads share 4 KV heads (4:1), already reducing KV cache vs MHA.
• ECE: Expected Calibration Error — the gap between predicted confidence and empirical accuracy; target < 0.05 for the $N^*$ stopping rule.

More Information [optional]

Architecture decision records: correctness_model/docs/rfcs/RFC0001–RFC0017. Scaling roadmap: correctness_model/docs/SCALING_PLAN.md. Reproducible results: correctness_model/docs/RESULTS.md. Theory: correctness_model/docs/theory/ (measurement-cost model, training pipeline design).

Model Card Authors [optional]

Jonah Sullivan (ThatsNinja)

Model Card Contact

Jonah Sullivan — jonah@thats.ninja