MonumentalSystems/harmonic-gpt-spherical-kan-128m

Harmonic GPT Spherical KAN 128M

An experimental 128.26M-parameter causal byte language model from the Harmonic GPT geometry A/B. This is the final step-39,063 checkpoint, selected because its generations were preferred qualitatively during review. It is not the run's metric-best checkpoint: final validation BPB was 1.2693, while the best observed validation BPB was 1.2102.

The model uses a fixed QPSK + Hamming wire encoder, 18 unified blocks, seven heads, eight Funk-Hecke/SpectralKAN channels per head, bounded decayed recurrent state, and projection-free spherical Lohe mean-field coupling. Its hidden width is 1,330 (seven oscillator groups of 190 dimensions). The minimal readout exposes raw oscillator state, alignment, and order-parameter observables to a 256-byte head.

Checkpoint and data

• Parameters: 128,260,556
• Training exposure: 2,560,032,768 bytes
• Context used in training: 2,048 bytes
• Optimizer: split-decay RotationalAdamW
• Dropout: 0
• Training stream: ordered document-corpus-v3
• Checkpoint precision in this repository: BF16

This is a base continuation model, not an instruction-tuned assistant. The demo formats chat turns as Q: ...\nA: because that was the run's original inference probe format. Byte value zero is used as a conservative generation stop heuristic; the checkpoint was not trained with the later explicit chat end-of-response delimiter.

Native bounded-state inference

modeling_spherical_kan.py is a standalone inference implementation. Its step() path preserves the trained SpectralKAN recurrence and spherical Lohe update while retaining O(1) state with respect to generated sequence length. Packaged BF16 logits were checked against the original checkpoint's BF16 step() path with maximum absolute delta 0.0.

Important limitations

This is a research artifact, not a reliable general assistant. It can produce plausible but incorrect, repetitive, malformed, or nonsensical text. Do not use it for medical, legal, financial, safety-critical, or production decisions.

The model is released under CC BY-NC 4.0 for research use. The training corpus is an internally assembled, filtered, globally deduplicated research mixture of curated, reference, long-form, mathematical, reasoning, scientific/code, and document-QA sources; downstream users remain responsible for reviewing source-specific terms.

Research context

The checkpoint is the spherical geometry sibling of the modern Transformer and planar Givens controls. Harmonic GPT treats benchmark scores as diagnostics of self-organizing oscillator computation rather than as claims of production-model competitiveness.