Note: Method/findings card published ahead of the weights. KOLM-Beta-T-1.3B weights will be uploaded to this repo on release. Part of the KOLM project (Kuramoto-oscillator language models).
Model Card: KOLM-Beta-T (Transplant line)
KOLM-Beta-T ("T" = Transplant) converts pretrained transformer weights into a fully-oscillatory KOLM-Alpha-family architecture β synchronization-based routing, oscillator processing, zero transformer computation β and heals the converted model with a short recovery run, skipping pretraining entirely. This card records the method, the controlled evidence that the transplant confers a large head start, and the artifact lineage. Findings files: Spike1_andSpike2_findings.md, HEADSTART_FINDINGS.md. Naming per NAMING.md.
Method (established across Spikes 1β3 + head-start test, 2026-07-13/14)
- Exact-copy half. Embeddings, RMS norms, and attention q/k/v transfer exactly: GQA expanded to MHA (repeat KV heads) and a per-head row permutation maps the donor's half-split RoPE onto the oscillator router's interleaved convention. Verified parity vs donor attention: 1.5e-06 relative error.
- Amplitude (Stuart-Landau) output path. Phase-only (unit-norm)
settle destroys signal magnitude (clone error 0.135 β fail). Restoring
the oscillators' amplitude degree of freedom (
(ΞΌ β |x|Β²)x) fixes the attention output path to 0.0099 (13.6Γ, strong pass): the entire attention side of a transformer layer converts at sub-1% error. - The thinker (MLP) does not clone directly. A trained SwiGLU resists per-layer imitation by settle dynamics of any tested flavor (phase-only 0.632 / amplitude 0.662 / + parametric-frequency mixer 0.582, vs a plain-linear floor of 0.678). Parametric frequency modulation (input sets each oscillator's rotation speed) is the only mechanism that separated from baseline and is adopted in the block.
- Imperfect transplant + recovery beats from-scratch decisively (the head-start result, below): the thinkers do not need to be right, only to be a far-better-than-random start.
Head-start result (controlled A/B, 2026-07-14)
Donor SmolLM2-135M β converted 235M full-oscillator model. Identical architecture, corpus (wikitext-103), token budget, and fair init for both arms; the only difference is transplanted vs random weights.
| Recovery tokens | Converted | Random (from scratch) |
|---|---|---|
| 0 | 5.33 | 10.95 |
| 1M | 4.03 | 6.24 |
| 5M | 3.72 | 5.63 |
| 7M | 3.62 | 5.45 (retired) |
| 40M | 3.19 β donor-level | β |
The untrained transplant already beats the from-scratch model after 7M tokens of training; the transplant reaches donor-level perplexity in 40M tokens. Scope: one scale, one corpus, perplexity only β behavior preservation (chat/coding/tool use) is the explicit target of the next rung, which adds donor-graded healing (the frozen donor scores every prediction via KL on full logits) and staged SFT.
Models in this line
| Model | Status |
|---|---|
| KOLM-Beta-T-235M (SmolLM2-135M donor) | pre-recovery checkpoint archived; healed weights lost to a save-ordering bug (curves preserved; fix in place: checkpoint every val step) |
| KOLM-Beta-T-1.3B (Qwen2.5-1.5B-Instruct donor) | in progress β donor-graded healing + chat SFT; weights to be uploaded here |
Intended use & limitations
Research line, pre-release. The 235M evidence is perplexity-only at one scale; recovery-token scaling (Β±3Γ) and behavior preservation are open until the 1.3B run reports. Converted models inherit their donor's knowledge, biases, and license conditions (donors used are Apache-2.0). The resulting models are pure oscillator networks with a settle-depth compute dial; no transformer computation remains after conversion.