tjdoomer/system2-code

System 2: 5-Head Deliberation & Epistemic Drift Detector (Code Track)

"Small local models have a 15% to 25% stochastic error rate, whether from CoT loops, hallucinated syntax, or formatting quirks. If you run them raw, they break in production. System 2's job is not to magically make a broken model write code; it is to catch those failures in 500ms on Apple Silicon MPS and say: Certainty is 41.8%, do not run this code. That is the automated gatekeeper that makes edge AI safe."

System 2 is a non-autoregressive deliberation engine built on ModernBERT. It evaluates competing hypotheses across 5 parallel perspectives in under 30ms on Apple Silicon MPS or NVIDIA CUDA, calculates pairwise cosine cluster consensus, and detects epistemic drift before code execution.

• Checkpoint: system2_code.pt (143 MB)
• Base Architecture: answerdotai/ModernBERT-Large-Instruct (395M, frozen) + LoRA subspace adapters ($r=32, \alpha=32$ across layers 20-27) + 5 ThoughtHeads + Cross-attending Judger.
• Guarded Metric (Code Traces): 0.9010 (89.11% accuracy, 0.0891 Brier score, 0.0542 true 10-bin ECE).
• Official Website: system2.online
• GitHub Repository: github.com/tj-doomer-it/System2

---

The Problem: The 2-Bit CoT Collapse & Garbage In, Garbage Out (GIGO)

Crushing large models (27B+) to 2.0-bit (UD-IQ2_XXS) or 3.0-bit destroys the attention projection weights that emit the </think> closing token. In empirical testing on Apple Silicon (M4 Max), 2-bit models burned 95%+ of their token budgets inside internal reasoning loops, taking 8 to 9 minutes per stage and emitting empty or truncated code.

System 2 creates a new paradigm: Pairing lightweight 4-bit small models (7B to 12B) with a 143 MB ModernBERT 5-head consensus harness on standard 16 GB laptops.

[ User Prompt / Code Task ]
             │
             ▼
┌────────────────────────────────────────────────────────┐
│ ⚡ FAST GENERATOR: 7B-12B 4-bit Model (e.g. Gemma 12B) │
│  • Emits 4 parallel candidate hypotheses (~2-3s)       │
│  • High speed (50-70 tok/s), low VRAM (6.8 GB)         │
└──────────────────────────┬─────────────────────────────┘
             │ (4 Candidate Hypotheses)
             ▼
┌────────────────────────────────────────────────────────┐
│ 🧠 SYSTEM 2: 143 MB 5-Head Deliberation Network (MPS)  │
│  • Head 0: Holistic Context (Global integration)       │
│  • Head 1: Context-Anchored (40% Q / 60% A)            │
│  • Head 2: Choice-Forward (20% Q / 80% A)              │
│  • Head 3: Symmetric Balance (50% Q / 50% A)           │
│  • Head 4: Extreme Salience (5% Q / 95% A)             │
│  • Pairwise Cosine Consensus (Threshold >= 0.88)       │
└──────────────────────────┬─────────────────────────────┘
             │
       ┌─────┴────────────────────────┐
       ▼                              ▼
[ HIGH CERTAINTY ]            [ EPISTEMIC DRIFT ]
(Cluster Agreement >= 88%)    (Divergent Candidates)
       │                              │
       ▼                              ▼
Execute Locally on 16GB Mac   Smart Cloud Escalation
(Latency: < 4 seconds)        (Fast-Fail under 4 seconds)

---

Live Validation: Epistemic Drift Detection in Action

During multi-agent swarm testing constructing an in-memory LRUTTLCache, candidate subagents diverged into an internal debate over relative vs absolute TTL refresh logic:

• Candidate 1: 0 lines
• Candidate 2: 78 lines
• Candidate 3: 108 lines (scratchpad debate)
• Candidate 4: 7 lines

System 2 Verdict: Confidence collapsed from 98.7% down to 41.8% across the 5 ThoughtHeads. Because confidence fell below the 88% threshold, System 2 flagged Epistemic Drift, successfully catching that the local models were guessing and halting execution before deploying broken code.

---

Quickstart

Installation

pip install system2

Python API

from sys2 import System2Engine

# Initialize on Apple Silicon GPU (mps) or NVIDIA GPU (cuda)
engine = System2Engine(device="mps")

# Adjudicate 4 competing code candidates
result = engine.adjudicate(
    question="Select the correct, thread-safe implementation of an LRU cache eviction node:",
    choices=[
        candidate_a_code,
        candidate_b_code,
        candidate_c_code,
        candidate_d_code,
    ],
    domain="code",
)

print(f"Winning Candidate: #{result.selected_index + 1}")
print(f"Calibrated Confidence: {result.confidence * 100:.1f}%")
print(f"5-Head Consensus: {result.head_consensus} Heads")
print(f"Deliberation Latency: {result.deliberation_latency_ms:.2f} ms")

Drop-in Local OpenAI Proxy

Run System 2 as a drop-in bridge server for Cursor, Claude Code, Cline, or LangChain:

python3 -m sys2.server --port 8000

Point your OpenAI client base URL to http://localhost:8000/v1.

---

Benchmarks: M4 Max (128 GB Unified Memory)

Metric	Path A: qwen-3.8-27b-smoll	Path B: qwen-3.8-27b-iq3xxs	Path Gemma: google/gemma-4-12b
Model Size / VRAM	2.0-bit (6.77 GB)	3.0-bit (10.18 GB)	4.0-bit (6.77 GB)
Module 1 (Node)	99.7% Conf (41 lines)	81.6% Conf (19 lines)	76.6% Conf (22 lines)
Module 2 (Cache)	94.2% Conf (18 lines)	75.2% Conf (14 lines)	98.7% Conf (37 lines)
Module 3 (Test Suite)	99.6% Conf (31 lines)	67.8% Conf (10 lines)	41.8% Conf (Drift Flagged)
Deliberation Latency (MPS)	580ms	982ms	1,107ms
Pytest Result	FAILED (SyntaxError)	FAILED (IndentationError)	FAILED (Markdown list indent)
Root Failure Cause	2-bit CoT loop truncation	3-bit CoT loop truncation	Indented markdown list block

---

Citation & License

• License: Apache 2.0
• Authors: Thom James-Worrall / Antigravity Research