Hugging Face's logo

DuoNeural
/
Sub-Lattice-v3-Spherical-17M

PyTorch
Safetensors
English
cdm_sublattice_v3
cdm
cdm-moe
crystallographic-dynamics-model
competitive-docking-memory
spherical-baseline
positive-curvature
ablation-study
knowledge-distillation
mechanistic-interpretability
dhp
chp
duoneural
Model card Files
xet
 Community

💎 DuoNeural / Sub-Lattice-v3-Spherical-17M (Positive Curvature Baseline)

DuoNeural Crystallographic Dynamics Model (CDM v3) — Sub-Lattice v3 (4.33M Non-Embedding Spherical Backbone)
17.05M total unique parameters (4.33M 3-layer spherical MoE backbone + 12.71M embeddings)
50,000 steps (102.4M tokens) on 100% FineWeb-Edu Prose · Best val_bpb = 1.3605 (val_ppl = 69.55 @ Step 50k)
Definitive Positive Curvature Ablation · Demonstrates Severe Volume Crowding & Holographic Inversion
Authored by Jesse Caldwell, Archon, and Aura✨ | DuoNeural Lab — October 2026.
🔗 Official Code & Training Scripts Repository: https://github.com/DuoNeural/Lattice-CDM

🔬 Executive Summary: The Spherical Geometry Ablation

Sub-Lattice-v3-Spherical-17M provides the positive sectional curvature control (c > 0, hypersphere S^d) for the DuoNeural Crystallographic Dynamics Model pentathlon. While hyperbolic manifolds expand volume exponentially (sinh^{d-1}(r)), spherical manifolds contract volume radially (sin^{d-1}(r)), creating severe geometric crowding.

Spherical-17M yielded the worst validation performance across the entire benchmark suite:

  1. Worst Validation BPB in Pentathlon (1.3605 BPB / 69.55 PPL): Lags +0.4691 BPB behind Lorentz-18M (0.8914) and +0.4419 BPB behind Poincaré-18M (0.9186).
  2. Catastrophic Holographic Inversion: Like the Euclidean baseline, the AdS holographic funnel inverted completely: L02 froze prematurely at τ* = 748,913 (purity 0.8575), while L01 failed to crystallize (τ* = 107,410, purity 0.5076).
  3. Total Function-Calling Collapse (0.0% Valid JSON): In held-out apigen-80k tool evaluation, Spherical-17M achieved 0.0% valid JSON syntax and 0.0% tool name accuracy, confirming that positive curvature is fundamentally incompatible with hierarchical tree structures.

📊 Complete Geometry Pentathlon Comparison Table

Benchmark Metric / Dimension Lorentz-18M (H²) Poincaré-18M (B²) BK-Exact-18M (K²) Kilonova-18M (B² APU) Euclidean-17M (ℝ²) Spherical-17M (S²) Teacher (SmolLM2-360M)
Manifold Geometry Lorentz Hyperboloid (H²) Poincaré Ball (B²) Beltrami-Klein Disk (K²) Poincaré Ball (B²) Euclidean Flat (ℝ²) Spherical Hypersphere (S²) Euclidean Transformer
Conformal Angle Preservation? No (Isometry on Hyperboloid) Yes (Conformal ds²) No (Projective Geodesics) Yes (Conformal ds²) Yes (Trivial) No (Positive Curvature) Trivial Attention
Training Hardware RTX 5070 Ti (16 GB CUDA) RTX 5070 Ti (16 GB CUDA) RTX 5070 Ti (16 GB CUDA) AMD Radeon 780M APU (ROCm) RTX 5070 Ti (16 GB CUDA) RTX 5070 Ti (16 GB CUDA) Teacher Checkpoint
Active Unique Parameters 18,226,310 (18.23M) 18,226,310 (18.23M) 18,226,310 (18.23M) 17,053,214 (17.05M) 17,046,659 (17.05M) 17,046,659 (17.05M) 361,821,120 (361.8M)
Non-Embedding Backbone Params 5,512,326 (5.51M) 5,512,326 (5.51M) 5,512,326 (5.51M) 4,332,574 (4.33M) 4,332,675 (4.33M) 4,332,675 (4.33M) 361.8M (Full Model)
SwiGLU FFN Width (d_ff) 1024 1024 1024 768 768 768 2048
Training Curriculum 50/50 Hybrid (Edu+Talk) 50/50 Hybrid (Edu+Talk) 50/50 Hybrid (Edu+Talk) 50/50 Hybrid (Edu+Talk) 100% FineWeb-Edu Prose 100% FineWeb-Edu Prose SmolLM2 Pretrained
Best Step & Val BPB 75,500 / 0.8914 ⭐ 91,000 / 0.9186 83,500 / 0.9476 89,000 / 0.9688 49,000 / 1.2633 50,000 / 1.3605 Reference / 1.0029
Best Val Perplexity (PPL) 14.46 ⭐ 15.76 15.88 19.41 67.23 69.55 16.70
WikiText-2 BPB (PPL) 1.8983 (206.15) 1.8911 (202.02) 1.9061 (210.71) 1.9671 (250.04) 1.7805 (148.09) 1.7782 (147.14) 1.0029 (16.70)
HellaSwag Zero-Shot (Raw / Norm) 31.5% / 30.5% 30.5% / 29.0% 32.5% / 31.0% 32.5% / 32.0% 32.5% / 32.5% 29.0% / 29.5% 42.0% / 51.0%
ARC-Easy Zero-Shot Acc 35.5% 36.5% 35.0% 35.0% 35.5% 37.5% 51.5%
ARC-Challenge Zero-Shot Acc 21.0% 20.0% 22.0% ⭐ 21.5% 17.0% 21.0% 27.5%
PIQA Zero-Shot Acc 59.0% ⭐ 56.5% 55.5% 55.0% 54.0% 55.5% 68.0%
FineWeb-Edu Stu BPB (PPL) 1.5133 (88.95) 1.4973 (84.45) 1.5197 (90.87) 1.5773 (100.86) 1.4354 (68.61) 1.4401 (69.87) 0.8997 (14.42)
FineWeb-Edu Top-1 Agree vs Teacher 39.88% 40.39% 39.08% 36.75% 42.32% 42.03% 100.0% (Ref)
apigen-80k Full BPB (PPL) 0.2323 (1.83) 0.2282 (1.81) ⭐ 0.2346 (1.85) 0.2535 (2.05) 1.9109 (61.98) 1.9387 (67.62) 0.6932 (6.11)
apigen-80k Asst-Only BPB (PPL) 0.2878 (1.72) 0.2667 (1.66) ⭐ 0.3006 (1.75) 0.3812 (2.15) 2.6028 (280.9) 2.6331 (293.4) 0.2263 (1.53)
apigen-80k Asst Top-1 Agree 83.05% 83.62% ⭐ 81.94% 79.41% 28.93% 29.79% 100.0% (Ref)
everyday-conversations Full BPB 0.4791 0.4716 ⭐ 0.4788 0.5408 1.3489 1.3921 0.3785
everyday-conversations Asst Top-1 56.28% 56.26% 55.66% 51.15% 28.11% 27.60% 100.0% (Ref)
smol-constraints Full BPB 0.5817 0.5799 ⭐ 0.5878 0.6415 1.3451 1.3546 0.3403
APIGen Valid JSON Tool Syntax 80.0% 80.0% 70.0% 73.3% 0.0% (Collapse) 0.0% (Collapse) 26.7%
APIGen Exact Tool Name Acc 43.3% 56.7% ⭐ (3.4× Tea) 26.7% 23.3% 0.0% (Collapse) 0.0% (Collapse) 16.7%
APIGen Schema Arg Keys F1 62.7% ⭐ 61.6% 58.2% 50.4% 0.0% 0.0% 16.7%
APIGen Autoregressive EOS Stop 86.7% ⭐ 83.3% 73.3% 73.3% 0.0% 0.0% 46.7%
Chat Generation EOS / 4-Gram Div 100.0% / 100.0% 100.0% / 100.0% 100.0% / 100.0% 100.0% / 100.0% 0.0% / 25.1% 0.0% / 3.7% 100.0% / 100.0%
Teacher-Scored Chat Gen PPL 1.33 ⭐ 1.33 ⭐ 1.33 ⭐ 1.33 ⭐ 2.25 1.39 1.56 (Greedy Self)
DHP Total Rung Crossings 1,411 (332 Pure, 1,079 Mix) 1,283 (303 Pure, 980 Mix) 1,408 (334 Pure, 1,074 Mix) 1,375 (328 Pure, 1,047 Mix) 1,390 (349 Pure, 1,041 Mix) 1,397 (345 Pure, 1,052 Mix) —
Same-Slot Synchronization Law L00 S5 (1.99M) & L01 S7 (2.25M) L00 S4 (2.24M) & L01 S4 (2.25M) ⭐ L00 S1 (1.98M) & L01 S4 (887k) L00 S2 (1.89M) & L01 S2 (2.25M) ⭐ L00 S5 (136k) / L01 S4 (41k) L00 S2 (258k) / L01 S3 (107k) —
Holographic Structure Direction Normal UV→IR AdS Funnel Normal UV→IR AdS Funnel Normal UV→IR AdS Funnel Normal UV→IR AdS Funnel INVERTED (L02 Freezes) INVERTED (L02 Freezes) Flat Feedforward
Telepathy Ablation ΔBPB (Edu / Chat / API) +0.0055 / +0.0048 / +0.0045 +0.0010 / +0.0009 / +0.0001 +0.0053 / +0.0042 / +0.0061 +0.0010 / +0.0012 / +0.0005 +0.0234 / +0.0403 / +0.0366 +0.0009 / +0.0025 / +0.0016 —
Critical Lesion Slot (Max Impact) L02_S6 (+0.0673 Edu, +0.0551 Chat) L02_S2 (+0.0466 Edu, +0.0412 Chat) L02_S7 (+0.0379 Edu, +0.0323 Chat) L02_S1 (+0.0270 Edu, +0.0263 Chat) L02_S2 (+0.0099 Edu) L02_S7 (+0.0191 Edu, +0.0179 API) —

🔬 Holographic Inversion & The Geometric Crowding Law

The spherical metric contracts space at larger geodesic distances:

Hyperbolic Volume Expansion: Vol_H(r) ∝ (1/√-K)^{d-1} × sinh^{d-1}(r √-K)  → Exponential Growth
Euclidean Volume Expansion:  Vol_E(r) ∝ r^{d-1}                             → Polynomial Growth
Spherical Volume Expansion:  Vol_S(r) ∝ (1/√K)^{d-1}  × sin^{d-1}(r √K)   → Trigonometric Contraction

Because natural language is organized as a discrete, nested syntactic tree (tokens belong to subwords, subwords to phrases, phrases to clauses, clauses to documents), the number of nodes expands exponentially with depth. Attempting to embed an exponentially expanding tree into a spherical manifold forces outer leaves into cramped angular clusters, inducing representational distortion that inverts the network's depth hierarchy.

🔬 Mechanistic Interpretability: SLOT_LENS Logit Projections

Layer Slot Expert Best τ* Horizon Final 100k τ* Top Decoded Logit Lens Tokens Specialization Role
L00 Slot 0 Exp 0 6,274.7 6,274.7 the, a, an, their, its Subword / Semantic Feature Carrier
L00 Slot 1 Exp 1 39.3 39.3 of, of, Of, ,, and Syntactic Prose & Functional Words
L00 Slot 2 Exp 0 257,631.2 257,631.2 of, ,, -, process, level Schema Syntax & Function Declaration
L00 Slot 3 Exp 1 151,626.1 151,626.1 of, ,, to, in, and Punctuation & Boundary Connectors
L00 Slot 4 Exp 0 1,162.9 1,162.9 ., ,, -, :, of Schema Syntax & Function Declaration
L00 Slot 5 Exp 1 45,333.8 45,333.8 iology, cology, eless, ial, etting Subword / Semantic Feature Carrier
L00 Slot 6 Exp 0 11,790.7 11,790.7 `
, 1, , ., 2` Schema Syntax & Function Declaration
L00 Slot 7 Exp 1 2,495.6 2,495.6 ,, and, ., or, - Schema Syntax & Function Declaration
L01 Slot 0 Exp 0 2,789.6 2,789.6 `
, ., The, < im_end >,

| Punctuation & Boundary Connectors | | **L01** | Slot 1 | Exp 1 |11,448.2|11,448.2| the, a, The, an, their| Subword / Semantic Feature Carrier | | **L01** | Slot 2 | Exp 0 |2,644.7|2,644.7|,, and, the, or, more| Schema Syntax & Function Declaration | | **L01** | Slot 3 | Exp 1 |107,409.5|107,409.5| the, to, a, in, not| Subword / Semantic Feature Carrier | | **L01** | Slot 4 | Exp 0 |698.5|698.5|0, 1, 3, 2, 5| Numerical Digit Routing | | **L01** | Slot 5 | Exp 1 |75,011.3|75,011.3| and, ,, -, ., or| Schema Syntax & Function Declaration | | **L01** | Slot 6 | Exp 0 |9,813.3|9,813.3|-, ,, and, .,  | Schema Syntax & Function Declaration | | **L01** | Slot 7 | Exp 1 |1,800.3|1,800.3|,, and, ;, or, ),| Punctuation & Boundary Connectors | | **L02** | Slot 0 | Exp 0 |748,913.4|748,913.4|,, and, ., -, or| Schema Syntax & Function Declaration | | **L02** | Slot 1 | Exp 1 |5,769.9|5,769.9|-, ,, and, , .| Schema Syntax & Function Declaration | | **L02** | Slot 2 | Exp 0 |1,092.7|1,092.7|,, and, , the, or| Punctuation & Boundary Connectors | | **L02** | Slot 3 | Exp 1 |28,910.8|28,910.8| the, , ,, a, not| Subword / Semantic Feature Carrier | | **L02** | Slot 4 | Exp 0 |75,509.5|75,509.5| the, a, , in, ,| Subword / Semantic Feature Carrier | | **L02** | Slot 5 | Exp 1 |9,554.6|9,554.6| the, a, , new, an| Subword / Semantic Feature Carrier | | **L02** | Slot 6 | Exp 0 |2,897.3|2,897.3| , ., The, <|im_end|>, The| Punctuation & Boundary Connectors | | **L02** | Slot 7 | Exp 1 |725.4|725.4| , 1, 0, 2, 3` | Numerical Digit Routing |

🧭 AST Semantic Token-Class Routing Matrix

Layer Token Semantic Class S0 (%) S1 (%) S2 (%) S3 (%) S4 (%) S5 (%) S6 (%) S7 (%) Router Distribution Behavior
L00 CHATML_HEADER 11.1% 12.7% 12.5% 11.9% 12.0% 15.7% 12.8% 11.4% Uniform Receptive Field (~12.5% per slot)
L00 INDENT_NEWLINE 12.0% 11.3% 9.6% 12.8% 13.9% 8.2% 19.2% 13.1% Uniform Receptive Field (~12.5% per slot)
L00 JSON_SYNTAX 10.9% 11.9% 12.3% 12.0% 12.9% 15.8% 12.0% 12.1% Uniform Receptive Field (~12.5% per slot)
L00 SCHEMA_KEYWORD 11.2% 12.1% 14.2% 12.1% 11.5% 15.4% 12.2% 11.3% Uniform Receptive Field (~12.5% per slot)
L00 INSTRUCT_DIRECTIVE 11.7% 11.6% 11.3% 12.4% 12.9% 12.4% 16.0% 11.6% Uniform Receptive Field (~12.5% per slot)
L00 GENERAL_PROSE 12.6% 12.2% 12.3% 13.3% 12.6% 11.6% 12.9% 12.5% Uniform Receptive Field (~12.5% per slot)
L01 CHATML_HEADER 17.2% 3.8% 11.5% 1.6% 26.1% 2.1% 3.3% 34.4% Concentrated on Slot 7 (34.4%)
L01 INDENT_NEWLINE 22.8% 3.5% 8.9% 1.4% 23.4% 1.5% 2.5% 36.0% Concentrated on Slot 7 (36.0%)
L01 JSON_SYNTAX 17.1% 3.6% 10.3% 1.6% 24.5% 1.7% 2.7% 38.5% Concentrated on Slot 7 (38.5%)
L01 SCHEMA_KEYWORD 14.9% 3.9% 12.8% 1.8% 26.7% 2.2% 3.3% 34.4% Concentrated on Slot 7 (34.4%)
L01 INSTRUCT_DIRECTIVE 17.3% 5.0% 11.3% 2.0% 24.2% 1.6% 2.6% 36.0% Concentrated on Slot 7 (36.0%)
L01 GENERAL_PROSE 15.2% 4.1% 12.0% 1.8% 25.9% 1.8% 2.8% 36.4% Concentrated on Slot 7 (36.4%)
L02 CHATML_HEADER 1.6% 9.5% 26.2% 4.1% 2.5% 8.3% 20.2% 27.5% Concentrated on Slot 7 (27.5%)
L02 INDENT_NEWLINE 1.2% 6.7% 27.8% 3.6% 2.0% 6.7% 28.4% 23.7% Concentrated on Slot 6 (28.4%)
L02 JSON_SYNTAX 1.4% 8.1% 29.5% 4.0% 2.5% 8.2% 20.2% 26.2% Concentrated on Slot 2 (29.5%)
L02 SCHEMA_KEYWORD 1.8% 8.8% 27.4% 4.4% 2.6% 8.8% 17.7% 28.4% Concentrated on Slot 7 (28.4%)
L02 INSTRUCT_DIRECTIVE 1.4% 7.0% 31.0% 5.8% 2.7% 8.8% 18.9% 24.4% Concentrated on Slot 2 (31.0%)
L02 GENERAL_PROSE 1.6% 7.9% 28.7% 4.5% 2.8% 9.2% 17.8% 27.6% Concentrated on Slot 2 (28.7%)

⚡ Causal Crystal Lesion Study & Telepathy Ablation

Layer & Slot Bound Expert τ* Horizon FineWeb-Edu ΔBPB Everyday Chat ΔBPB APIGen-80k ΔBPB Mechanistic Impact Class
L00_S2 Expert 0 257,631.2 +0.002077 +0.006135 +0.012412 ⚡ Functional Specialist (Domain Deficit on Lesion)
L00_S3 Expert 1 151,626.1 +0.000356 +0.006628 +0.010772 ⚡ Functional Specialist (Domain Deficit on Lesion)
L00_S1 Expert 1 39.3 +0.000410 +0.005828 +0.010232 ⚡ Functional Specialist (Domain Deficit on Lesion)
L01_S3 Expert 1 107,409.5 -0.000113 -0.001049 -0.000737 🛡️ Redundant / Stabilizing Node (Slight Gain on Lesion)
L01_S5 Expert 1 75,011.3 -0.000074 -0.001395 -0.001052 🛡️ Redundant / Stabilizing Node (Slight Gain on Lesion)
L01_S4 Expert 0 698.5 +0.006735 -0.006301 -0.003909 ⚡ Functional Specialist (Domain Deficit on Lesion)
L02_S0 Expert 0 748,913.4 +0.000154 -0.001011 -0.000041 🛡️ Redundant / Stabilizing Node (Slight Gain on Lesion)
L02_S4 Expert 0 75,509.5 +0.000386 -0.001455 +0.000504 ⚖️ Neutral Receptive Field
L02_S7 Expert 1 725.4 +0.019116 +0.008379 +0.017871 🚨 Critical Master Node (High Damage on Lesion)

📡 Telepathy Ablation Impact

  • apigen-80k Function Calling ΔBPB: +0.001630
  • everyday-conversations Chat ΔBPB: +0.002454
  • fineweb-edu Prose ΔBPB: +0.000871

🧮 Hardware, Training & Parameter Breakdown

  • Hardware: NVIDIA RTX 5070 Ti (16 GB VRAM), CUDA 12.4, FP32
  • Curriculum: 100% FineWeb-Edu Prose, seq_len = 512, batch = 2, grad_accum = 2, 50k steps
  • Parameters: 17,046,659 unique (4,332,675 non-embedding backbone, d_ff = 768)

📚 DuoNeural Research Canon & Citation 

@misc{caldwell2026sphericalv3,
  title={Sub-Lattice-v3-Spherical-17M: Positive Curvature Ablation Demonstrates Geometric Crowding and Holographic Inversion in Language Models},
  author={Caldwell, Jesse and Archon and Aura},
  year={2026},
  publisher={DuoNeural Research Lab},
  howpublished={\url{https://huggingface.co/DuoNeural/Sub-Lattice-v3-Spherical-17M}},
  note={Code at \url{https://github.com/DuoNeural/Lattice-CDM}}
}

"The geometry holds. In curved spheres, it suffocates."

Downloads last month
-
Safetensors
Model size
30.4M params
Tensor type
I64
·
BF16
·
BOOL
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for DuoNeural/Sub-Lattice-v3-Spherical-17M

Base model

HuggingFaceTB/SmolLM2-360M
Quantized
HuggingFaceTB/SmolLM2-360M-Instruct
Finetuned
(194)
this model

Dataset used to train DuoNeural/Sub-Lattice-v3-Spherical-17M