qwen3-14b-code-agent-wsurgery — weight-space surgery patch

A surgical weight patch for harshagowtham/qwen3-14b-code-agent (fine-tune of Qwen/Qwen3-14B), built by weight-space surgery — no retraining. It trims 30% of the fine-tune's drift on the 13 highest-energy residual-stream writers while keeping 70% of the learned delta:

W_surgery = W_base + 0.70 * (W_ft - W_base)

applied to the top-8 self_attn.o_proj and top-5 mlp.down_proj tensors (13 tensors, 1.31 GB bf16) selected by measured delta energy across all 40 layers. Full analysis: SURGERY_REPORT.md.

The patch is not a full model — it carries only the 13 modified tensors. It applies to a local snapshot of the finetuned model by byte-exact splicing into the safetensors shards (identical byte length, sha256 verified before and after every write, no torch required).

Quickstart with llm_engine (recommended)

# pip: see https://huggingface.co/harshagowtham/qwen-agent-scripts
from llm_engine import LocalLLM

# A) patch auto-detected and applied on load:
llm = LocalLLM("harshagowtham/qwen3-14b-code-agent",
               adapter="harshagowtham/qwen3-14b-code-agent-mcp-lora",
               hf_token="hf_...")

# B) or this repo directly as the model (one argument):
llm = LocalLLM("harshagowtham/qwen3-14b-code-agent-wsurgery", hf_token="hf_...")

Standalone apply (stdlib + huggingface_hub only)

# 1) download the finetuned model once
huggingface-cli download harshagowtham/qwen3-14b-code-agent \
    --local-dir ./ft-snapshot --token hf_...

# 2) verify (no writes)
python apply_patch.py --model-dir ./ft-snapshot \
    --patch-repo harshagowtham/qwen3-14b-code-agent-wsurgery \
    --token hf_... --dry-run

# 3) apply
python apply_patch.py --model-dir ./ft-snapshot \
    --patch-repo harshagowtham/qwen3-14b-code-agent-wsurgery \
    --token hf_...

# 4) load the patched model with plain transformers
python -c "
from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained('./ft-snapshot', torch_dtype='bfloat16')
print('patched model loaded:', m.config.model_type)
"

A .wsurgery_applied marker file appears in the model dir; re-running is a no-op. Removing the marker re-applies (after restoring a pristine snapshot — the applier verifies original bytes and refuses mismatches).

Repository layout

apply_patch.py            # standalone applier (this repo, no torch needed)
SURGERY_REPORT.md         # full diagnosis + selection + verification report
wsurgery/
  manifest.json           # format, alpha, per-tensor shard/offset/sha256/shape
  model_layers_*_o_proj_weight.bin     # 8 patched attention writers (52.4 MB each)
  model_layers_*_down_proj_weight.bin  # 5 patched MLP writers (178.3 MB each)

Manifest format (v1)

{
  "format": "wsurgery-patch",
  "base_model": "harshagowtham/qwen3-14b-code-agent",
  "base_model_upstream": "Qwen/Qwen3-14B",
  "alpha": 0.7,
  "rule": "W_surgery = W_base + alpha*(W_ft - W_base)",
  "tensor_count": 13,
  "tensors": [
    {
      "name": "model.layers.39.self_attn.o_proj.weight",
      "shard": "model-00007-of-00008.safetensors",
      "offset": 12345678,
      "nbytes": 52428800,
      "shape": [5120, 5120],
      "dtype": "bfloat16",
      "sha256_original": "...",   // pristine finetuned bytes (checked before write)
      "sha256_patched": "...",    // patched bytes (checked after write)
      "file": "wsurgery/model_layers_39_self_attn_o_proj_weight.bin"
    }
  ]
}

Safety properties

• Refuses to write if the bytes at the target offset do not hash to sha256_original (i.e. the snapshot is not the expected finetuned model).
• Post-write verification re-reads and re-hashes every spliced tensor.
• Idempotent via the .wsurgery_applied marker + manifest hash.
• safetensors headers are untouched — the file layout is unchanged, only tensor payload bytes differ.

Related

• harshagowtham/qwen3-14b-code-agent — the fine-tuned base of this patch
• harshagowtham/qwen-agent-scripts — the engine that auto-applies it
• harshagowtham/qwen3-14b-code-agent-v2 — everything pre-merged (patch + MCP LoRA)

License

Apache-2.0 (patch data and code). The patched weights derive from harshagowtham/qwen3-14b-code-agent / Qwen/Qwen3-14B (Apache-2.0).