Note: This is a model card / findings record published ahead of the weights. Weights upload is planned; artifacts currently live in the project's private storage. Part of the KOLM project (Kuramoto-oscillator language models).

Model Card: KOLM-Hybrid-GNC-CNG (32M ×3 arms)

Group-teaching experiments on KOLM-Hybrid-1-32M: can oscillator groups teach their members through their own coherent modes ("internal LoRA"), and does a second hierarchy level (leaders-of-leaders) help when the model is forced to rely on it? Run 2026-07-11 on the laptop (MPS). Headline: a clean, triple-replicated negative — with genuinely useful positive engineering findings. Full record: GNC_CNG_RESULTS.md. Naming per NAMING.md.

Mechanisms tested

  • GNC (group teaching): each group's mean-field m_g = Σᵢ M_gᵢ xᵢ (its "leader" mode, reusing the coherence-readout matrix M) broadcast back to members inside every settle step with a learned per-group gain w_g — a rank-G state-dependent coupling delta. Zero-init = exact identity.
  • CNG (correlated network grouping): second hierarchy level — 8 learned super-leaders over the 32 groups, fed back down. To force reliance, the CNG arm froze all dense couplings J and J_q (32 matrices across 16 layers).

Arms & results (all warm-start from native_beta.pt, val 2.5662)

Arm Config Tokens Final val Ablation delta (zero the gains)
beta_teach GNC, grad_steps=1 3.1M 2.5588 +0.0000
teach_g4 GNC, full BPTT 1.0M 2.6027 +0.0000
cng GNC+CNG, full BPTT, J/J_q frozen 1.0M 2.5968 +0.0000 (w2 alone and w_g+w2)

Negative findings

  1. The learned group-teaching gains carry exactly nothing. Zeroing them on every trained arm changes validation loss by +0.0000 — invariant to gradient path (truncated vs full BPTT) and to freezing the dense couplings.
  2. Gain growth is an optimizer signature, not evidence of use. The gains grow early at the Adam step-size floor and plateau at ~0.5% of the dense coupling magnitude — looking "alive" while doing nothing.
  3. A converged model routes around frozen couplings. With J/J_q frozen, adaptation flowed through attention and projection pathways and matched the trainable-J run — the oscillator couplings of a converged hybrid are a settled subsystem that continuation training does not revisit.

Positive findings

  1. The mechanism engineering is fully validated: exact identity at zero init, bit-exact match with the plain block at zero gains, NumPy parity of the modified dynamics, and verified gradient flow to w_g/M2/w2 — the null is a property of the idea, not a bug. Machinery lives on in kuramoto_torch.py (group_teach, cng flags) and the ablation harness ablate_wg.py.
  2. Methodology: end-of-run ablation is the only reliable indicator of mechanism use found in this project — gain magnitudes and loss curves both mislead. This finding shaped every later experiment (spike kill-bars, transplant A/B design).
  3. Scope note: these bound continuation training of a converged model (~1–3M tokens/arm); from-scratch training with group teaching enabled is untested.

Artifacts (all local; finals + training states)

native_beta_teach.pt / ckpt_beta_teach.pt · native_teach_g4.pt / ckpt_teach_g4.pt · native_cng.pt / ckpt_cng.pt · curves curve_beta_teach.csv, curve_teach_g4.csv, curve_cng.csv · record GNC_CNG_RESULTS.md.

Intended use & limitations

Archive / negative-result reference (labeled retired). Not usable assistants. Single seed per arm; small continuation budgets; easy corpus. The value of these weights is reproducibility of the ablation null.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support