Note: This is a model card / findings record published ahead of the weights. Weights upload is planned; artifacts currently live in the project's private storage. Part of the KOLM project (Kuramoto-oscillator language models).

Model Card: KOLM-Hybrid-GSK (61M)

GSK = Grow → Sparsify → K-upconvert — an architecture-surgery pipeline executed on a trained KOLM-Hybrid-1 model, demonstrating that oscillator LMs tolerate staged post-hoc surgery: function-preserving width growth, low-rank coupling sparsification with fast recovery, settling-depth up-conversion, and chat tuning. Run 2026-07-12 on one A10G (~1.6h end-to-end). Naming per NAMING.md.

Model details

• Architecture: KOLM-Hybrid-1 (causal attention + Kuramoto-oscillator FFN), 16 layers, d=384, frustrated coupling, 32k vocabulary (FVT).
• Starting point: KOLM-Hybrid-1-42M-K1 checkpoint (native_beta32k_k1.pt, val 1.7162 on beta32k).
• Pipeline stages:
  1. G — Grow: oscillator banks widened H 320 → 640 across all 16 layers (54.3M → 80.2M params), function-preserving by construction. Verified: max |Δlogits| = 4.09e-14 (bit-level identity).
  2. S — Sparsify: dense H×H couplings J factorized to rank 160 (80.2M → 60.96M params), then 1,500 recovery steps at K=1.
  3. K — Up-convert: 1,500 further steps at K=2.
  4. Chat: 2,000 steps on the chat mixture at K=2.
• Training config: batch 48 × ctx 256, bf16, A10G. Note: grad-steps 1 (truncated backprop through settling) — this run predates the project's full-BPTT hard rule.

Results

Stage	Params	Val loss	Reference
donor K1 checkpoint	54.3M	1.7162	K-sweep native K1
after Grow (identity)	80.2M	1.7162 (bit-identical)	Δlogits 4.09e-14
after Sparsify + recovery (K=1, 18M tokens)	61.0M	1.7136	beats donor at rank-160 J
after K-upconvert (K=2, 18M tokens)	61.0M	1.7037	native-K2 control: 1.6848
after chat tune (K=2, 25M tokens)	61.0M	3.6659 (chatmix val)	—

Findings

1. Width growth is exactly function-preserving (4e-14), extending the depth-growth result of the KOLM-Hybrid-1 paper (§3.3) to width — the grown model inherits the donor bit-for-bit and only trains new capacity.
2. Coupling matrices sparsify cheaply. Cutting J to rank 160 (25% of dense) recovered within 18M tokens to slightly better than the dense donor (1.7136 vs 1.7162) at 61M vs 80M params — evidence the dense couplings of a trained hybrid are heavily redundant.
3. K-up-conversion approaches but does not match native training at the target depth: 1.7037 after 18M tokens vs 1.6848 for a model trained at K=2 from the FVT checkpoint with a 147M-token budget. Consistent with the paper's finding that settle depth is negotiable but each depth has its own optimum shape.
4. Chat tuning behaves as in other hybrids (fluent register, weak facts).

Artifacts

File	Where
native_gsk.pt (post-K, pre-chat, 61M)	S3 kolm-beta/results/
native_gsk_chat.pt (chat-tuned)	local + S3
k1_h640.pt (post-grow 80.2M)	local
curve_gsk_s.csv, curve_gsk_k.csv, curve_gsk_chat.csv, run_gsk.log	S3

Intended use & limitations

Research artifact for studying architecture surgery on oscillator LMs. Not a usable assistant: trained on ~2% of a Chinchilla-optimal budget on easy corpora; factual recall out of scope. Single-seed results. Truncated BPTT (grad-steps 1) means numbers are not directly comparable to the later full-BPTT KOLM-Alpha line.