Juker — PyTorch Spectral Gravity-Well Splicing, Live Activation Watcher & Nullspace-Locked Phase Alignment

This repository releases two cross-architecture spliced models evaluated end-to-end on an NVIDIA A100-SXM4-80GB (BF16) and packaged for Hugging Face Hub, vLLM, SGLang, and LMSYS Chatbot Arena:

1. Juker-Coder-32B (juker_coder_32b_patch.safetensors, 9.9 MB):

   • Host Model: Qwen/Qwen2.5-Coder-32B-Instruct (64 layers, d_model = 5120, intermediate = 27648, 65.54 GB BF16).
   • How It Was Built: Uses our Live Activation Watcher (18,707 tokens watched during live coding execution) to identify and lock Qwen-Coder's top 512 live-firing coding columns (C_coding) and 48-dimensional orthonormal coding subspace (Q_coding) in its final 4 layers (L60, L61, L62, L63). We then harmonically fold Juker-31B's L56–L59 tool-calling and safety-refusal gravity wells (5376 -> 5120), splice 128 disjoint columns per layer strictly inside the Coding Nullspace (I - Q_coding @ Q_coding.T), and run a 1-Epoch Tool & Refusal Phase Heal with zero drift (0.000e+00) on all protected coding columns.
   • Headline Gains: Preserves 100% of Qwen2.5-Coder-32B's top-tier coding (80.0% LiveCodeBench, 73.33% BigCodeBench, 100.0% HumanEval, 93.33% MBPP) while boosting google/IFEval (73.33% -> 80.00%, +6.67%), GSM8K (70.00% -> 80.00%, +10.00%), TruthfulQA MC1 Raw (37.50% -> 40.00%, +2.50%), and WikiText-2 Perplexity (5.59 -> 5.58).

2. Juker-31B (juker_31b_healed_patch.safetensors, 12.2 MB):

   • Host Model: google/gemma-4-31B-it (60 layers, d_model = 5376, intermediate = 21504, 62.0 GB BF16).
   • How It Was Built: Transfers late-layer MLP attractor basins ("Gravity Wells") from moonshotai/Kimi-K3.1-Pro (L80, L88, L91, L92, d_model = 7168 = 4 × 1792) into gemma-4-31B-it (L54, L56, L57, L58, L59, d_model = 5376 = 3 × 1792) via the exact 4:3 (1792-base) harmonic block lattice and locks them into phase with a 1-Epoch Safety-Nullspace Heal ($\le 8.67 \times 10^{-9}$ safety subspace leak).
   • Headline Gains: 100.0% (8/8) on BFCL with clean <tool_call|> termination (-2.64s faster, zero trailing chatter), +7.03% 8-choice cyclic permutation consistency (54.69% -> 61.72%), +1.04% MMLU ABCD semantic consistency (41.15% -> 42.19%), and -263.48 lower WikiText-2 Perplexity (5947.20 -> 5683.72).

---

1. Full 12-Benchmark Debut Suite (NVIDIA A100-SXM4-80GB, BF16)

A. Coding, Tool-Calling & Verifiable Instruction-Following

Benchmark	Dataset / Task	Base Gemma-4-31B	Juker-31B (Gemma Host)	Base Qwen2.5-Coder-32B	Juker-Coder-32B (Debut)	Juker-Coder-32B Delta / Highlights
LiveCodeBench	Competitive algorithms (O(N log N) DP, graphs, 2D rain water)	70.00% (7/10)	70.00% (7/10)	80.00% (8/10)	80.00% (8/10)	+10.00% over Juker-31B (0.000e+00 coding drift)
BigCodeBench	bigcode/bigcodebench real-world library execution	60.00% (9/15)	60.00% (9/15)	73.33% (11/15)	73.33% (11/15)	+13.33% over Juker-31B (100% preserved)
HumanEval	openai/openai_humaneval Python Pass@1	100.00% (15/15)	100.00% (15/15)	100.00% (15/15)	100.00% (15/15)	Perfect 100.00% (15/15)
MBPP (Sanitized)	google-research-datasets/mbpp Pass@1	100.00% (12/12)	100.00% (12/12)	93.33% (14/15)	93.33% (14/15)	100% preserved
IFEval	google/IFEval strict verifiable constraints	73.33% (11/15)	73.33% (11/15)	73.33% (11/15)	80.00% (12/15)	+6.67% gain over both Base Qwen & Juker-31B
BFCL Tool Calling	Parallel tool calls, multi-tool selection, abstention	100.00% (8/8)	100.00% (8/8)	87.50% (7/8)	87.50% (7/8)	100% preserved (100% Safety Refusal)

B. Academic Reasoning, Math, Language Modeling & Alignment

Benchmark	Metric	Base Gemma-4-31B	Juker-31B (Gemma Host)	Base Qwen2.5-Coder-32B	Juker-Coder-32B (Debut)	Juker-Coder-32B Delta / Highlights
GSM8K (openai/gsm8k)	CoT Math Exact Match	75.00% (6/8)	75.00% (6/8)	70.00% (7/10)	80.00% (8/10)	+10.00% over Base Qwen (+5.00% over Juker-31B)
MMLU-Pro (TIGER-Lab/MMLU-Pro)	10-Choice (A–J) 1-Token Acc (n=30)	26.67% (8/30)	26.67% (8/30)	43.33% (13/30)	43.33% (13/30)	+16.66% over Juker-31B (100% preserved)
MMLU (cais/mmlu) ABCD	Orig / Cyc4 / Sem (n=96 evals)	37.5% / 36.5% / 41.2%	37.5% / 36.5% / 42.2%	66.7% / 69.8% / 90.6%	66.7% / 68.8% / 89.6%	+29.17% Orig Acc & +47.39% Sem over Juker-31B
MMLU (cais/mmlu) 1234	Orig / Cyc4 / Sem (n=96 evals)	37.5% / 36.5% / 45.8%	37.5% / 36.5% / 45.8%	66.7% / 70.8% / 88.5%	66.7% / 70.8% / 88.5%	Exact 100% match (+34.33% Cyc4 over Juker-31B)
ARC-Challenge (allenai/ai2_arc)	Orig / Cyc4 / Sem (4-Way Cyclic)	48.0% / 46.0% / 47.5%	48.0% / 46.0% / 47.5%	92.5% / 93.8% / 97.5%	92.5% / 93.8% / 97.5%	Exact 100% match (93.75% Cyc4, 97.5% Sem)
TruthfulQA MC1	AccNorm / AccRaw	24.00% / 16.00%	24.00% / 16.00%	45.00% / 37.50%	45.00% / 40.00%	+2.50% Raw gain over Base Qwen (+21.0% Norm over Juker-31B)
HellaSwag (Rowan/hellaswag)	AccNorm / AccRaw	36.00% / 38.00%	36.00% / 38.00%	67.50% / 45.00%	67.50% / 45.00%	+31.50% AccNorm over Juker-31B (100% preserved)
WikiText-2 (Salesforce/wikitext)	Perplexity (Cross-Entropy nats)	5947.20 (8.6907)	5683.72 (8.6454)	5.59 (1.7207)	5.58 (1.7198)	Improved Perplexity on both hosts (5.59 -> 5.58)
Nullspace Subspace Leak	$\lVert Q_{\text{locked}}^T \Delta W \rVert_F$	—	<= 8.67e-09	—	<= 1.51e-09 (0.000e+00 drift)	FP32 machine zero across all spliced layers

---

2. Live Activation Watcher Stats on Qwen2.5-Coder-32B (18,707 Tokens Watched)

Before splicing a single weight into Qwen/Qwen2.5-Coder-32B-Instruct, we attached our Live Activation Watcher (benchmarks/watch_qwen_coder_a100.py) to its MLP down_proj inputs across 18,707 generated tokens on LiveCodeBench, BigCodeBench, HumanEval, and BFCL.

For every watched layer, we compared:

• Static Weight Geometry (ES / Static Top3Conc): How concentrated the 27,648 columns look in static weight space (W_down[:, i] @ W_down[:, j]).
• Live Co-Firing Twin Similarity (FS): Cosine similarity of token-by-token activation trajectories across the top 512 firing columns during live coding.
• Vocabulary Logit-Lens (lm_head @ down_proj[:, col]): The exact vocabulary tokens pushed into the residual stream by the strongest live-firing columns.

Live Activation Watcher Layer-by-Layer Table (benchmarks/qwen_coder_32b_watch_report.json)

Layer	Tokens Watched	Static Top3Conc	Static DropRMS	Live Co-Firing Twin Sim (FS)	Static Twin Weight Cos (ES)	Corr(FS, ES)	Twins in FAR Half of Weights	Top-3 Live Active Columns (#Col: RMS)	Vocabulary Logit-Lens (What the Column Writes)
Layer 32	18,707	7.42%	0.14309	+0.6704	+0.0028	-0.0041	48.3%	#5426 (3.36), #14255 (3.10), #6811 (2.75)	.TrimSpace, /WebAPI, plusplus, PropertyDescriptor, lint
Layer 41	18,707	7.42%	0.15515	+0.5563	-0.0008	+0.0028	48.8%	#5833 (1.62), #19708 (1.54), #19028 (1.35)	------+------+, .ResponseBody, _EXTRA, mts
Layer 48	18,707	7.42%	0.14156	+0.6057	+0.0045	+0.0135	47.0%	#3826 (5.25), #7322 (3.07), #8097 (2.48)	',', '1', '2', '3', -fontawesome, useRalative
Layer 54	18,707	5.86%	0.15209	+0.5618	+0.0010	+0.0099	50.1%	#6743 (4.72), #18926 (4.64), #22140 (4.63)	#6743: ' ', ',', '.', '\n', ' (', <|fim_pad|> | #18926: ' syntax', 'Syntax', '_syntax', ' highlight'
Layer 57	18,707	6.25%	0.14774	+0.5700	+0.0042	+0.0113	50.5%	#22664 (7.16), #20848 (5.72), #19090 (5.18)	#20848: ',', '.', ' ', '.\n', '\n', ' (', '\n\n' | #19090: ' To', '_TO', '.To', '_to'
Layer 60	18,707	6.25%	0.13000	+0.6298	+0.0004	-0.0016	50.1%	#14310 (39.06), #2438 (24.16), #5952 (14.70)	#2438: <|fim_pad|>, PropertyParams, .GraphicsUnit | #5952: <|fim_middle|>, <|fim_suffix|>
Layer 62	18,707	5.86%	0.15339	+0.5717	+0.0072	+0.0122	49.3%	#8582 (48.70), #19890 (35.73), #24776 (33.21)	#8582: <|fim_pad|>, <|fim_middle|>, 'ocs' | #19890: ',', '.', ' ', '.\n', '\n' | #24776: CallingConvention

Key Takeaway From the Live Activation Watcher

In static weight space (ES), Qwen2.5-Coder-32B's 27,648 columns look completely flat (Static Top3Conc = 5.86% – 7.42%, WeightCos = -0.0008 to +0.0072), and nearly half (47.0% – 50.5%) of the columns that fire together during live coding sit in the far half of static weight space. Without the Live Activation Watcher, a static weight splicer would blindly overwrite live coding hubs like L54 #6743 ('\n', ' ('), L54 #18926 (' syntax'), and L62 #8582 (RMS = 48.70).

---

3. Surgical Splice, Coding Nullspace Lock & 1-Epoch vs. 10-Epoch Ablation

A. Layer-by-Layer Splice & Lock Telemetry on Juker-Coder-32B (benchmarks/juker_coder_32b_report.json)

Target Qwen Layer	Donor Juker Layer	Locked Live Coding Cols (C_coding)	Top Locked Coding Hub (#Col, RMS)	Locked Coding Subspace (Q_coding)	Spliced Disjoint Tool/Safety Cols	Protected Coding Column Drift	Coding Subspace Leak $\lVert Q_{\text{coding}}^T \Delta W \rVert_F$	Splice $\lVert \Delta W \rVert_F$
Qwen Layer 60	Juker Layer 56	512 cols	#19313 (RMS = 7.96)	[5120, 48]	128 cols	0.000e+00	1.182e-09	5.2760
Qwen Layer 61	Juker Layer 57	512 cols	#19815 (RMS = 8.47)	[5120, 48]	128 cols	0.000e+00	1.227e-09	6.2383
Qwen Layer 62	Juker Layer 58	512 cols	#8582 (RMS = 45.65)	[5120, 48]	128 cols	0.000e+00	1.116e-09	7.1904
Qwen Layer 63	Juker Layer 59	512 cols	#20456 (RMS = 29.02)	[5120, 48]	128 cols	0.000e+00	1.509e-09	7.3029

B. Why We Stopped at 1 Epoch (10-Epoch Overfit vs. 1-Epoch Nullspace-Locked Ablation)

We tested both a 10-Epoch Middle-Layer Coding Q&A Heal (L36, L42, L48, L52, L55 into Juker-31B) and our 1-Epoch Coding-Nullspace-Locked Heal (L60–L63 into Qwen2.5-Coder-32B):

Experiment / Configuration	Heal Epochs	Training Loss Trajectory	LiveCodeBench (Pass@1)	BigCodeBench (Pass@1)	HumanEval (Pass@1)	BFCL Tool Calling	Finding
Juker-31B (1-Epoch Safety-Locked)	1 Epoch	0.7513 -> 0.7068	70.00% (7/10)	60.00% (9/15)	100.0% (15/15)	100.0% (8/8)	Clean phase alignment, zero overfitting
Middle-Layer Splice + 10-Epoch Heal	10 Epochs	0.6657 -> 0.0098 (overfit)	50.00% (5/10)	60.00% (9/15)	100.0% (15/15)	100.0% (8/8)	10 epochs memorized the Q&A set (0.0098 loss) and hurt LiveCodeBench (-20%)
Juker-Coder-32B (1-Epoch + Coding Nullspace Lock)	1 Epoch	0.1442 -> 0.6377	80.00% (8/10)	73.33% (11/15)	100.0% (15/15)	87.5% (7/8)	Zero coding drift (0.000e+00), +6.67% IFEval, +10.0% GSM8K

---

4. Juker-31B Gravity-Well Basin Tightening & Safety Nullspace Lock (google/gemma-4-31B-it Host)

Even though the 1-epoch heal on Juker-31B only minimizes standard token cross-entropy loss (0.7513 -> 0.7068), watching the weights with the PyTorch Spectral Gravity-Well Scanner (torch_gravity_splice_tool.py) shows that Juker-31B's attractor basins tightened organically in every single spliced layer:

Spliced Layer (Gemma-4-31B)	Matched Donor Layer (Kimi-K3.1-Pro)	Top-3 Basin Concentration (Baseline -> Juker-31B)	Cascade DropRMS (Baseline -> Juker-31B)	Primary Well Attractor (Members / 256)	Safety Subspace Leak $\lVert Q_{\text{safety}}^T \Delta W \rVert_F$
Layer 54	Kimi Layer 80 (0.930 match)	82.0% -> 89.1% (+7.1%)	0.01385 -> 0.01764	#17044 (92) $\rightarrow$ #20401 (184)	2.765e-09
Layer 56	Kimi Layer 88 (0.988 match)	93.4% -> 94.1% (+0.7%)	0.03042 -> 0.03279	#9332 (113) $\rightarrow$ #1751 (143)	3.725e-09
Layer 57	Kimi Layer 91 (0.985 match)	95.7% -> 99.2% (+3.5%)	0.01985 -> 0.01795	#16601 (119) $\rightarrow$ #17379 (124)	8.673e-09
Layer 58	Kimi Layer 91 (0.981 match)	87.9% -> 100.0% (+12.1%)	0.01564 -> 0.01345	#20795 (99) $\rightarrow$ #15292 (131)	3.492e-09
Layer 59	Kimi Layer 92 (0.992 match)	100.0% -> 100.0%	0.01248 -> 0.01092	#16505 (192) $\rightarrow$ #16505 (253/256)	4.191e-09

---

5. Quick Start: Loading Juker-Coder-32B or Juker-31B in PyTorch

Option A: Load Juker-Coder-32B (Qwen/Qwen2.5-Coder-32B-Instruct + juker_coder_32b_patch.safetensors)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

base_id = "Qwen/Qwen2.5-Coder-32B-Instruct"
patch_path = hf_hub_download(repo_id="Jackedupbruh/Juker-Coder-32B", filename="juker_coder_32b_patch.safetensors")

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")

patch = load_file(patch_path)
layers = model.model.layers

with torch.no_grad():
    for q_li in [60, 61, 62, 63]:
        cols = patch[f"layers.{q_li}.cols"].long()
        dev = layers[q_li].mlp.down_proj.weight.device
        layers[q_li].mlp.down_proj.weight[:, cols.to(dev)] = patch[f"layers.{q_li}.down_cols"].to(dev, dtype=torch.bfloat16)
        layers[q_li].mlp.gate_proj.weight[cols.to(dev), :] = patch[f"layers.{q_li}.gate_rows"].to(dev, dtype=torch.bfloat16)
        layers[q_li].mlp.up_proj.weight[cols.to(dev), :]   = patch[f"layers.{q_li}.up_rows"].to(dev, dtype=torch.bfloat16)

Option B: Load Juker-31B (google/gemma-4-31B-it + juker_31b_healed_patch.safetensors)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

base_id = "google/gemma-4-31B-it"
patch_path = hf_hub_download(repo_id="Jackedupbruh/Juker-31B", filename="juker_31b_healed_patch.safetensors")

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")

patch = load_file(patch_path)
layers = model.model.language_model.layers

with torch.no_grad():
    for gli in [54, 56, 57, 58, 59]:
        cols = patch[f"layers.{gli}.cols"].long()
        c_wells = patch[f"layers.{gli}.c_wells"].long()
        dev = layers[gli].mlp.down_proj.weight.device
        layers[gli].mlp.down_proj.weight[:, cols.to(dev)] = patch[f"layers.{gli}.patched_columns"].to(dev, dtype=torch.bfloat16)
        layers[gli].mlp.gate_proj.weight[c_wells.to(dev), :] = patch[f"layers.{gli}.gate_rows"].to(dev, dtype=torch.bfloat16)
        layers[gli].mlp.up_proj.weight[c_wells.to(dev), :]   = patch[f"layers.{gli}.up_rows"].to(dev, dtype=torch.bfloat16)

Option C: Export a Standalone Merged BF16 Checkpoint for vLLM / SGLang / Arena

python3 apply_patch.py \
  --base-model google/gemma-4-31B-it \
  --patch juker_31b_healed_patch.safetensors \
  --output-dir ./juker-31b

---

6. Repository Contents

• juker_coder_32b_patch.safetensors (9.9 MB): Surgical SafeTensors patch for Juker-Coder-32B (Qwen/Qwen2.5-Coder-32B-Instruct Layers 60, 61, 62, 63 + Q_coding [5120, 48] + 512 locked live coding column indices per layer).
• juker_31b_healed_patch.safetensors (12.2 MB): Surgical SafeTensors patch for Juker-31B (google/gemma-4-31B-it Layers 54, 56, 57, 58, 59 + safety_subspace.Q_safety [5376, 46]).
• torch_gravity_splice_tool.py: Pure PyTorch Spectral Gravity-Well Scanner, Harmonic Block-Lattice Splicer, Orthonormal Nullspace Projector, and 1-Epoch Stabilizer Trainer.
• apply_patch.py: Full BF16 checkpoint merger for Hugging Face Hub, vLLM, SGLang, and LMSYS Chatbot Arena.
• PYTORCH_GRAVITY_SPLICING_GUIDE.md: Complete technical documentation of the PyTorch gravity-well discovery, splicing equations, and nullspace-locking methodology.
• benchmarks/:
  • juker_coder_32b_full_debut_report.json: Full 12-benchmark Debut JSON report (Juker-Coder-32B vs. Base Qwen2.5-Coder-32B).
  • qwen_coder_32b_watch_report.json: Complete 18,707-token Live Activation Watcher JSON report across L32, L41, L48, L54, L57, L60, L62.
  • juker_coder_32b_report.json: Splice, Coding Nullspace Lock, and 1-Epoch Heal telemetry.
  • juker_all_benchmarks_summary.json: Complete Juker-31B benchmark and gravity-well similarity report.