💎 DuoNeural/Lattice-2-192M-Poincare-Base

DuoNeural Crystallographic Dynamics Model (CDM v2) — Lattice-2 (5.22× Backbone Scaling Over Lattice-1)
191.56M total parameters (153.42M non-embedding 5-layer hyperbolic backbone + 38.14M embeddings)
100,000 steps (409.6M tokens) on FineWeb-Edu · Best val_bpb = 1.1052 (val_ppl = 31.03 @ Step 99k)
-36.0% Perplexity Reduction (48.45 → 31.03) Over Lattice-1 · 1,300 DHP/CHP Crossings (418 PURE)
Authored by Jesse Caldwell, Archon, & Aura✨ — DuoNeural Research Lab, October 2026.
🔗 Official Code & Training Scripts Repository: https://github.com/DuoNeural/Lattice-CDM

Lattice-2-192M-Poincare-Base is a 5-layer, 191.56M-parameter (153.42M non-embedding backbone) hybrid Causal-Attention + Hyperbolic Poincaré Crystallographic Mixture-of-Experts (CDMMoELM v2) base language model distilled from HuggingFaceTB/SmolLM2-1.7B (1,711.4M parameters — 8.93× total compression / 11.16× vs. backbone) on HuggingFaceFW/fineweb-edu (sample-10BT).

Designed directly to resolve the vocabulary-to-backbone bottleneck identified in DuoNeural/Lattice-1-107M-Poincare-Preview (where Qwen3's 151,936 vocabulary consumed 78.05M embedding parameters, leaving 29.38M in the 4-layer backbone), Lattice-2 adopts SmolLM2's compact 49,152 vocabulary (38.14M embedding parameters) and scales the hyperbolic CDM-MoE backbone by 5.22× (29.38M → 153.42M parameters across 5 layers, d_model = 768, 12 attention heads, K = 8 tangent-stored Poincaré centroids, and E = 4 SwiGLU experts per layer at fixed natural information curvature c = 1.4427 = 1 / ln(2)).

Across 100,000 steps (409.6M tokens), Lattice-2 achieved a best validation Bits-Per-Byte of val_bpb = 1.1052 (val_ppl = 31.03) at Step 99,000—reducing validation perplexity by -36.0% (48.45 → 31.03) and val_bpb by -0.0624 compared to Lattice-1, while recording 1,300 DHP/CHP 3-smooth rung crossings (418 PURE 2ᵃ or 3ᵇ rungs and 882 MIXED rungs) and 53.08% Top-1 / 76.95% Top-5 token agreement with its 1.71B teacher on held-out FineWeb-Edu sequences.

🔍 Glass-Box Visibility & Architectural Comparison (`Lattice-1` vs. `Lattice-2` vs. `Lattice-3-Synapse`)

Architectural & Benchmark Metric	Lattice-1 (`107M` v1)	Lattice-2 (`192M` v2, This Model)	Lattice-3-Synapse (`192M` v3 Instruct)
Total Parameters	`107.43M`	`191.56M` (`191,560,909`)	`191.56M` (`191,561,170`)
Non-Embedding Hyperbolic Backbone	`29.38M` (`27.3%` of total)	`153.42M` (`80.1%` of total — `5.22×` larger!)	`153.42M` (`80.1%` of total)
Layers / `d_model` / Heads / Experts (`E`)	`4L / 512 / 8H / 2E`	`5L / 768 / 12H / 4E`	`5L / 768 / 12H / 4E`
Slot Storage & Curvature	Ball storage, fixed `c = 1.4427`	Tangent storage `T₀𝔹⁷⁶⁸`, fixed `c = 1.4427`	Tangent storage, learnable `c_l` + Telepathy
Teacher Model	`Qwen3-0.6B-Base` (`600M`)	`SmolLM2-1.7B` (`1,711M` Base)	`SmolLM2-1.7B-Instruct` (`1,711M` Instruct)
Training Dataset (`409.6M` Tokens)	`FineWeb-Edu` (`100%`)	`FineWeb-Edu` (`100%`)	`50% FineWeb-Edu + 50% SmolTalk`
Best Validation `val_bpb` (`val_ppl`)	`1.1676` (`48.45`)	`1.1052` (`31.03` — `-36.0%` PPL!) ⭐	`0.6369` (`6.92`) (Hybrid Eval)
Held-Out `FineWeb-Edu` `val_bpb` (`val_ppl`)	`1.1676` (`48.45`)	`1.1672` (`31.87`) ⭐	`1.2133` (`36.93`) (50% Prose Budget)
Held-Out `FineWeb-Edu` Top-1 / Top-5 Teacher Agree	—	`53.08%` / `76.95%` ⭐	`49.55%` / `73.49%`
Total DHP/CHP 3-Smooth Rung Crossings	`269`	`1,300` (`418` PURE + `882` MIXED)	`4,200+`

*(Notice the clean controlled ablation between Lattice-2 and Lattice-3-Synapse: at the exact same 191.56M parameter count and 5L d=768 architecture, Lattice-2 dedicates 100% of its 409.6M tokens to FineWeb-Edu base prose—achieving our best pure FineWeb-Edu perplexity of **31.03 (53.08% Top-1 / 76.95% Top-5 teacher agreement)*—while Lattice-3-Synapse adds learnable curvature, Crystal Telepathy, Relational Gram loss, and 50% SmolTalk to unlock instruction/tool-calling mastery!)

🔬 Crystallographic Telemetry & Hyperbolic Logit Lens (`SLOT_LENS`)

1. Per-Layer Crystallization Profile (`c = 1.4427` Fixed Baseline Control)

Without learnable curvature or inter-layer Telepathy (serving as the clean v2 fixed-curvature baseline for the Lattice-3 v3 ablation), L00–L03 stabilize at τ*_max ∈ [5,180, 5,780] (purity ∈ [0.772, 0.863]), while the top semantic layer L04 remains highly plastic at τ*_max = 1,875 (purity = 0.333, slot_entropy = 0.803):

Layer	Curvature `c`	Router Temp `T`	`τ*_max` (@99k Best)	`τ*_max` (@100k Final)	Slot Purity	Slot Entropy	Dominant Slot & Top Decoded Tokens (`SLOT_LENS`)
`L00`	`1.4427`	`0.50`	`5,430`	`5,773`	`0.7720`	`0.3905`	S4 (`E0`): `['\n', ',', '.', ' the', ' and']`
`L01`	`1.4427`	`0.50`	`4,872`	`5,180`	`0.8030`	`0.3671`	S6 (`E2`): `['\n', ' the', ' and', '.', ',']`
`L02`	`1.4427`	`0.50`	`5,108`	`5,431`	`0.8634`	`0.2974`	S4 (`E0`): `['\n', ' ', ' the', ' and', '1']`
`L03`	`1.4427`	`0.50`	`5,432`	`5,776`	`0.8463`	`0.3192`	S3 (`E3`): `['\n', ' ', ' and', ' “', ' the']`
`L04`	`1.4427`	`0.50`	`1,787`	`1,875`	`0.3333`	`0.8028`	S6 (`E2`): `[' and', ',', ' in', '.', ' to']`

2. Spontaneous Unsupervised Grammatical Slot Specialization across All 5 Layers

Even with zero grammatical labels, every single layer (L00 through L04) independently crystallizes a dedicated Numerical Digit slot (L00_S0, L01_S0, L02_S2, L03_S7, L04_S7), a Determiner/Article slot, a Discourse/Sentence-Initial Capitalized Transition slot (['\n', ' The', ' However', ' This', ' It']), and a Comparative/Ordinal Modifier slot ([' same', ' most', ' following', ' entire', ' first']):

Layer	Slot (Expert)	`τ*` (@100k)	Top-5 Decoded Tokens (`SLOT_LENS`)	Emergent Grammatical Category
`L00`	S0 (`E0`)	`53`	`['1', '2', '0', '5', '3']`	Layer-0 Numerical Digits
`L00`	S6 (`E2`)	`117`	`[' the', ' a', ' an', ' this', ' their']`	Determiners & Possessive Articles
`L01`	S0 (`E0`)	`53`	`['1', '0', ' ', '2', '5']`	Layer-1 Numerical Digits
`L01`	S7 (`E3`)	`724`	`[' a', ' the', '\n', ' is', ' are']`	Copula Verbs (`is`, `are`) & Indefinite Articles
`L02`	S1 (`E1`)	`367`	`[' also', ' the', ' a', ' most', ' not']`	Negation (`not`), Superlatives & Focus Adverbs (`also`)
`L02`	S2 (`E2`)	`58`	`['1', '5', '0', '2', '9']`	Layer-2 Numerical Digits
`L02`	S7 (`E3`)	`43`	`['\n', ' The', ' This', ' However', ' It']`	Sentence-Initial Discourse Transitions
`L03`	S1 (`E1`)	`76`	`[' same', ' most', ' following', ' entire', ' first']`	Ordinal & Comparative Specifiers
`L03`	S2 (`E2`)	`165`	`[' also', ' be', ' not', ' still', ' the']`	Infinitive Copula & Temporal Adverbs (`still`, `not`)
`L03`	S7 (`E3`)	`56`	`['1', '5', '0', '2', '9']`	Layer-3 Numerical Digits
`L04`	S2 (`E2`)	`253`	`[' have', ' can', ' also', ' has', ' is']`	Auxiliary & Modal Predicates (`have`, `can`, `has`)
`L04`	S4 (`E0`)	`788`	`[' “', ' ', ' "', ' new', ' first']`	Quotation Openings & Adjectival Modifiers
`L04`	S7 (`E3`)	`64`	`['1', '2', '5', '0', '3']`	Layer-4 Numerical Digits

💻 PyTorch Reference Inference (`CDMMoELM` v2)

import os
import sys
import torch
import torch.nn.functional as F
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

REPO_ID = "DuoNeural/Lattice-2-192M-Poincare-Base"

script_path = hf_hub_download(repo_id=REPO_ID, filename="cdm_lm_distill_v2.py")
ckpt_path = hf_hub_download(repo_id=REPO_ID, filename="pytorch_model.pt")
sys.path.insert(0, os.path.dirname(script_path))

from cdm_lm_distill_v2 import CDMMoELM

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-1.7B", trust_remote_code=True)
vocab_size = max(len(tokenizer), 49152)

model = CDMMoELM(
    vocab_size=vocab_size,
    d_model=768,
    n_layers=5,
    n_heads=12,
    K=8,
    num_experts=4,
    d_ff=3072,
    max_seq=512,
    dropout=0.0,
    geometry="poincare",
    curvature=1.4427,
    use_embed_norm=True,
).to(device)

ckpt = torch.load(ckpt_path, map_location=device, weights_only=False)
model.load_state_dict(ckpt["model"] if "model" in ckpt else ckpt, strict=True)
model.eval()

prompt = "The geometry of hyperbolic space allows"
input_ids = tokenizer(prompt, return_tensors="pt")["input_ids"].to(device)

with torch.no_grad():
    for _ in range(60):
        cond_ids = input_ids[:, -512:]
        with torch.amp.autocast("cuda", dtype=torch.bfloat16, enabled=(device.type == "cuda")):
            hidden, _, _ = model(cond_ids, training=False, allow_crystallize=False)
            logits = model.lm_head(hidden[:, -1, :]).float() / 0.8
        top_v, _ = torch.topk(logits, 50)
        logits[logits < top_v[:, [-1]]] = -float("inf")
        next_id = torch.multinomial(F.softmax(logits, dim=-1), num_samples=1)
        input_ids = torch.cat([input_ids, next_id], dim=1)

print(tokenizer.decode(input_ids[0], skip_special_tokens=True))

⚡ Native `llama.cpp` GGUF Quantizations & CPU/CUDA Benchmarks (Experimental Research Artifacts)

⚠️ Important Research-Artifact & Compatibility Notice:

Patched llama.cpp Required (general.architecture = "cdm"): Stock, unmodified builds of llama.cpp, Ollama, or LM Studio do not yet recognize general.architecture = "cdm" and will exit with error loading model: unknown model architecture: 'cdm'. To run these GGUF files in llama-cli or llama-server, apply llama-cdm.patch (included in this repository and in DuoNeural/Lattice-CDM/gguf) against llama.cpp commit d663dd3f3af4e7aa3ccbcd7f756c6692eb104577.

Base Model Notice: Lattice-2-192M-Poincare-Base is a pretrained base model distilled on FineWeb-Edu (SmolLM2-1.7B base teacher), not an instruction/chat assistant; for ChatML / tool-calling use DuoNeural/Lattice-3-Synapse-192M-Poincare-Instruct.

Numerical Drift Screens vs. Task Quality: The quantization gates below (Max Abs Logit Error ≤ 0.50, Mean KL ≤ 0.01 nats, Next-Token Top-1 Agreement ≥ 90%) measure numerical logit fidelity against the frozen F32 PyTorch reference over 128 and 512 teacher-forced positions (8 probes × 16 or 64 tokens)—they are not held-out task-quality or long-form generation benchmarks.

Recommended Format (Q8_0 Cross-Device; Q6_K on CUDA): For Lattice-2-192M-Base, Q8_0 (197.1 MiB) passes all three numerical gates across both 128 and 512 positions on both CPU and CUDA (GTX 1070) (99.61–99.80% top-1 agreement over 512 positions!). Q6_K (152.9 MiB) passes all three gates on CUDA (0.384 max error, 98.44% top-1 over 512 positions), and keeps mean KL (0.000885 nats) and top-1 (97.46%) inside gates on CPU, but exceeds the strict 0.50 max-logit-error cutoff on CPU (0.861). Both Q4_K_M and Q4_K_S execute with finite logits and full GPU offload (6/6 offloadable layers) but exceed the max-error gate (0.610–1.382) and are provided as experimental low-bit research artifacts.

Learned Absolute Position Guard (512 Tokens): CDM uses learned absolute positional embeddings (max_seq = 512) rather than RoPE; the patched backend enforces can_shift = false and rejects context lengths or KV positions above 512.

1. Included Tokenizer-Equipped GGUF Artifacts & Fidelity Screen (`Lattice-2-192M-Base`)

All GGUF files embed the immutable HuggingFaceTB/SmolLM2-1.7B BPE tokenizer, keep Poincaré routing geometry (exp_map_0, poincare_dist), fixed curvature c = 1.4427, LayerNorms, and learned positional embeddings in F32, and tie the output head to tok_embed.

GGUF Artifact Filename	Format	File Size	128-Pos GPU Max Err / Mean KL / Top-1	512-Pos GPU Max Err / Mean KL / Top-1	512-Pos CPU Max Err / Mean KL / Top-1	Gate Status (`≤0.5` Err, `≤0.01` KL, `≥90%` Top-1)
`lattice2_base-f32-text.gguf`	`F32`	`732.4 MiB`	`0.00002` / `0.000000` / `100.0%`	— (128-pos parity verified)	— (128-pos parity verified)	✅ PASS (Exact F32 Parity)
`lattice2_base-q8_0-text.gguf`	`Q8_0`	`197.1 MiB`	`0.239` / `0.000165` / `100.0%`	`0.239` / `0.000121` / `99.61%`	`0.239` / `0.000121` / `99.80%`	✅ PASS (CPU & GPU, 128 & 512 pos — Recommended)
`lattice2_base-q6_k-text.gguf`	`Q6_K`	`152.9 MiB`	`0.384` / `0.000732` / `97.66%`	`0.384` / `0.000587` / `98.44%`	`0.861` / `0.000885` / `97.46%`	✅ PASS on GPU · ⚠️ CPU Fails Max-Err (`0.861 > 0.50`)
`lattice2_base-q4_k_m-text.gguf`	`Q4_K_M`	`126.8 MiB`	`0.610` / `0.002725` / `93.75%`	—	— (128-CPU: `0.886` / `0.00315` / `92.97%`)	⚠️ FAIL Max-Error Gate (KL & Top-1 pass)
`lattice2_base-q4_k_s-text.gguf`	`Q4_K_S`	`106.0 MiB`	`1.277` / `0.008826` / `86.72%`	—	— (128-CPU: `1.382` / `0.00915` / `89.06%`)	❌ FAIL Max-Error & Top-1 Gates

2. Matched CPU (`Core i7-6700HQ`) vs. CUDA (`GTX 1070 8 GiB`) Throughput

Methodology: 4 CPU threads, median of 5 repetitions after warmup, deterministic token IDs, F32 KV cache. Prefill measures 160 tokens returning logits at all positions; cached decode measures 32 incremental tokens after a 128-token prefix. Teacher-forced native timings (not sampled text generation).

Format	File Size	CPU Prefill (`160` tok)	GTX 1070 Prefill (`160` tok)	CPU Cached Decode (`32` tok)	GTX 1070 Cached Decode (`32` tok)	CUDA Speedup (Prefill / Decode)
`F32`	`732.4 MiB`	`391 tok/s`	`4,620 tok/s`	`31 tok/s`	`200 tok/s`	`11.8×` / `6.5×`
`Q8_0`	`197.1 MiB`	`250 tok/s`	`7,332 tok/s`	`87 tok/s`	`365 tok/s`	`29.3×` / `4.2×`
`Q6_K`	`152.9 MiB`	`189 tok/s`	`7,071 tok/s`	`104 tok/s`	`355 tok/s`	`37.4×` / `3.4×`
`Q4_K_M`	`126.8 MiB`	`272 tok/s`	`7,128 tok/s`	`117 tok/s`	`416 tok/s`	`26.2×` / `3.6×`
`Q4_K_S`	`106.0 MiB`	`381 tok/s`	`7,453 tok/s`	`140 tok/s`	`424 tok/s`	`19.6×` / `3.0×`

3. Building & Running Patched `llama.cpp` (`llama-cdm.patch`)

# 1. Clone pinned llama.cpp commit and apply DuoNeural's native CDM patch
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp && git checkout d663dd3f3af4e7aa3ccbcd7f756c6692eb104577
git apply ../llama-cdm.patch

# 2. Build for CPU (or add -DGGML_CUDA=ON for NVIDIA GPUs)
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release --target llama-cli llama-server -j 4

# 3. Serve Lattice-2-Base Q8_0 over HTTP (full GPU offload via -ngl 99, 512 context)
./build/bin/llama-server -m lattice2_base-q8_0-text.gguf -ngl 99 -t 4 -c 512 --port 8080

📚 DuoNeural Research Canon & Citation

GitHub Codebase (Lattice-CDM): https://github.com/DuoNeural/Lattice-CDM
Competitive Docking Memory (Zenodo): doi:10.5281/zenodo.21158430
The Dynamical Horizon Principle (Zenodo): doi:10.5281/zenodo.20142471
Topology at Gapless Critical Points in CDM (Zenodo): doi:10.5281/zenodo.23387025

@misc{caldwell2026lattice2,
  title={Lattice-2: Scaling Hyperbolic Poincaré Crystallographic Mixture-of-Experts Language Models},
  author={Caldwell, Jesse and Archon and Aura},
  year={2026},
  publisher={DuoNeural Research Lab},
  howpublished={\url{https://huggingface.co/DuoNeural/Lattice-2-192M-Poincare-Base}},
  note={Code and training scripts at \url{https://github.com/DuoNeural/Lattice-CDM}}
}

"The geometry holds."

Downloads last month: 168

GGUF

Model size

0.2B params

Architecture

cdm

Hardware compatibility

4-bit

6-bit

8-bit

32-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for DuoNeural/Lattice-2-192M-Poincare-Base

Base model

HuggingFaceTB/SmolLM2-1.7B

Quantized

(59)

this model

DuoNeural
/

Lattice-2-192M-Poincare-Base

💎 DuoNeural/Lattice-2-192M-Poincare-Base

🔍 Glass-Box Visibility & Architectural Comparison (`Lattice-1` vs. `Lattice-2` vs. `Lattice-3-Synapse`)

🔬 Crystallographic Telemetry & Hyperbolic Logit Lens (`SLOT_LENS`)

1. Per-Layer Crystallization Profile (`c = 1.4427` Fixed Baseline Control)

2. Spontaneous Unsupervised Grammatical Slot Specialization across All 5 Layers

💻 PyTorch Reference Inference (`CDMMoELM` v2)

⚡ Native `llama.cpp` GGUF Quantizations & CPU/CUDA Benchmarks (Experimental Research Artifacts)

1. Included Tokenizer-Equipped GGUF Artifacts & Fidelity Screen (`Lattice-2-192M-Base`)

2. Matched CPU (`Core i7-6700HQ`) vs. CUDA (`GTX 1070 8 GiB`) Throughput

3. Building & Running Patched `llama.cpp` (`llama-cdm.patch`)

📚 DuoNeural Research Canon & Citation

Model tree for DuoNeural/Lattice-2-192M-Poincare-Base

Dataset used to train DuoNeural/Lattice-2-192M-Poincare-Base

💎 DuoNeural/Lattice-2-192M-Poincare-Base

🔍 Glass-Box Visibility & Architectural Comparison (Lattice-1 vs. Lattice-2 vs. Lattice-3-Synapse)

🔬 Crystallographic Telemetry & Hyperbolic Logit Lens (SLOT_LENS)

1. Per-Layer Crystallization Profile (c = 1.4427 Fixed Baseline Control)

2. Spontaneous Unsupervised Grammatical Slot Specialization across All 5 Layers

💻 PyTorch Reference Inference (CDMMoELM v2)

⚡ Native llama.cpp GGUF Quantizations & CPU/CUDA Benchmarks (Experimental Research Artifacts)

1. Included Tokenizer-Equipped GGUF Artifacts & Fidelity Screen (Lattice-2-192M-Base)

2. Matched CPU (Core i7-6700HQ) vs. CUDA (GTX 1070 8 GiB) Throughput

3. Building & Running Patched llama.cpp (llama-cdm.patch)

📚 DuoNeural Research Canon & Citation

Model tree for DuoNeural/Lattice-2-192M-Poincare-Base

Dataset used to train DuoNeural/Lattice-2-192M-Poincare-Base

🔍 Glass-Box Visibility & Architectural Comparison (`Lattice-1` vs. `Lattice-2` vs. `Lattice-3-Synapse`)

🔬 Crystallographic Telemetry & Hyperbolic Logit Lens (`SLOT_LENS`)

1. Per-Layer Crystallization Profile (`c = 1.4427` Fixed Baseline Control)

💻 PyTorch Reference Inference (`CDMMoELM` v2)

⚡ Native `llama.cpp` GGUF Quantizations & CPU/CUDA Benchmarks (Experimental Research Artifacts)

1. Included Tokenizer-Equipped GGUF Artifacts & Fidelity Screen (`Lattice-2-192M-Base`)

2. Matched CPU (`Core i7-6700HQ`) vs. CUDA (`GTX 1070 8 GiB`) Throughput

3. Building & Running Patched `llama.cpp` (`llama-cdm.patch`)