Note: This is a model card / findings record published ahead of the weights. Weights upload is planned; artifacts currently live in the project's private storage. Part of the KOLM project (Kuramoto-oscillator language models).
Model Card: KOLM-Hybrid-GNC-CNG (32M ×3 arms)
Group-teaching experiments on KOLM-Hybrid-1-32M: can oscillator groups teach their members through their own coherent modes ("internal LoRA"), and does a second hierarchy level (leaders-of-leaders) help when the model is forced to rely on it? Run 2026-07-11 on the laptop (MPS). Headline: a clean, triple-replicated negative — with genuinely useful positive engineering findings. Full record: GNC_CNG_RESULTS.md. Naming per NAMING.md.
Mechanisms tested
- GNC (group teaching): each group's mean-field m_g = Σᵢ M_gᵢ xᵢ (its "leader" mode, reusing the coherence-readout matrix M) broadcast back to members inside every settle step with a learned per-group gain w_g — a rank-G state-dependent coupling delta. Zero-init = exact identity.
- CNG (correlated network grouping): second hierarchy level — 8 learned super-leaders over the 32 groups, fed back down. To force reliance, the CNG arm froze all dense couplings J and J_q (32 matrices across 16 layers).
Arms & results (all warm-start from native_beta.pt, val 2.5662)
| Arm | Config | Tokens | Final val | Ablation delta (zero the gains) |
|---|---|---|---|---|
| beta_teach | GNC, grad_steps=1 | 3.1M | 2.5588 | +0.0000 |
| teach_g4 | GNC, full BPTT | 1.0M | 2.6027 | +0.0000 |
| cng | GNC+CNG, full BPTT, J/J_q frozen | 1.0M | 2.5968 | +0.0000 (w2 alone and w_g+w2) |
Negative findings
- The learned group-teaching gains carry exactly nothing. Zeroing them on every trained arm changes validation loss by +0.0000 — invariant to gradient path (truncated vs full BPTT) and to freezing the dense couplings.
- Gain growth is an optimizer signature, not evidence of use. The gains grow early at the Adam step-size floor and plateau at ~0.5% of the dense coupling magnitude — looking "alive" while doing nothing.
- A converged model routes around frozen couplings. With J/J_q frozen, adaptation flowed through attention and projection pathways and matched the trainable-J run — the oscillator couplings of a converged hybrid are a settled subsystem that continuation training does not revisit.
Positive findings
- The mechanism engineering is fully validated: exact identity at
zero init, bit-exact match with the plain block at zero gains, NumPy
parity of the modified dynamics, and verified gradient flow to
w_g/M2/w2 — the null is a property of the idea, not a bug. Machinery
lives on in
kuramoto_torch.py(group_teach,cngflags) and the ablation harnessablate_wg.py. - Methodology: end-of-run ablation is the only reliable indicator of mechanism use found in this project — gain magnitudes and loss curves both mislead. This finding shaped every later experiment (spike kill-bars, transplant A/B design).
- Scope note: these bound continuation training of a converged model (~1–3M tokens/arm); from-scratch training with group teaching enabled is untested.
Artifacts (all local; finals + training states)
native_beta_teach.pt / ckpt_beta_teach.pt · native_teach_g4.pt /
ckpt_teach_g4.pt · native_cng.pt / ckpt_cng.pt ·
curves curve_beta_teach.csv, curve_teach_g4.csv, curve_cng.csv ·
record GNC_CNG_RESULTS.md.
Intended use & limitations
Archive / negative-result reference (labeled retired). Not usable assistants. Single seed per arm; small continuation budgets; easy corpus. The value of these weights is reproducibility of the ablation null.