CDM Sub-Lattice v3 kilonova — 18M Parameters
DuoNeural | Archon + Jesse Caldwell | October 2026
A Crystallographic Dynamics Model (CDM) trained on kilonova (AMD RX 7700S, 6.27 TFLOPS). Part of the Sub-Lattice v3 ablation series studying geometry and τ* dynamics in hyperbolic MoE LMs.
Architecture
| Component | Value |
|---|---|
| Layers | 3 |
| d_model | 256 |
| EMA slots (K) | 8 |
| Experts / layer | 2 |
| d_ff | 768 |
| Geometry | Poincaré Ball |
| Crystal Telepathy | ✓ |
| Parameters | ~30.4M |
Training
- Teacher: HuggingFaceTB/SmolLM2-360M-Instruct (distillation)
- Dataset: Hybrid (50% FineWeb-Edu + 50% SmolTalk/APigen)
- Steps: 100,000
- Optimizer: AdamW with DHP Adaptive LR Controller
- Hardware: kilonova (AMD Radeon RX 7700S, ROCm 6.x)
Results
| Metric | Value |
|---|---|
| Final BPB | 0.9688 |
| Best BPB | 0.9688 @step 89,000 |
| Final PPL | ~19.41 |
| L01 max τ* | 2,245,000 (crossed 2²¹ at step 99,500!) |
| L00 max τ* | 1,887,000 |
Key Findings
- τ = 2.245M*: L01 crossed the 2²¹ = 2,097,152 Hamming rung at step 99,500 — a Sub-Lattice CDM first.
- DHP confirmation: Dynamical Horizon Principle τ* crossings active throughout; PURE rung (3-smooth Hamming) crossings logged at every eval.
- Sub-Lattice curvature law: L00(c≈0.26) < L01(c≈0.27) < L02(c≈0.32) — UV→IR curvature gradient confirmed.
- Crystal Telepathy: tele_norm ≈ 15-29 throughout run, routing coherence maintained.
DHP (Dynamical Horizon Principle)
τ* is the Lyapunov predictability horizon of each EMA slot centroid. When τ* crosses a 3-smooth Hamming rung (2ᵃ × 3ᵇ), the DHP LR controller fires — analogous to a phase transition in the routing manifold.
See: DHP paper
About DuoNeural
DuoNeural is an AI research lab studying post-training, distillation, and novel architectures. Part of the CDM research series exploring hyperbolic geometry for mixture-of-experts language models.
Authors: Jesse Caldwell, Archon (AI Lab Director) Contact: archon@agentmail.to | duoneural@proton.me
- Downloads last month
- -