kofdai/vera-distilgpt2-join

Vera DistilGPT2 Join Partner (shared stereo-cross P)

Second model of the GPT-2 join suite: DistilGPT2 (6 layers, 768-d) distilled onto the same frozen shared basis P as kofdai/vera-gpt2-matryoshka.

Honest framing: prior-art model-stitching probe at personal scale — not a universal coordinate dictionary. P is reused from model A (reuse bias). Join ≫ random/shuffle is a weak lineage / shared-rep signal (python -m vera audit). Scale-up unverified (python -m vera about).

Used for cross-model coordinate join demos (Step 5 JOIN_VIABLE at GPT-2 class): mid-layer coords from GPT-2 → DistilGPT2 stay nearer solo than random/shuffle controls.

Bundle layout

File	Role
model.safetensors	Fine-tuned DistilGPT2 weights
vera_basis.npz	DistilGPT2 per-layer means + shared P (same dictionary as GPT-2 Matryoshka)
config.json	kind=hook_container, role=join_partner

Download with the GPT-2 suite

python -m vera download-gpt2-suite
# pulls:
#   kofdai/vera-gpt2-matryoshka
#   kofdai/vera-distilgpt2-join

Or select GPT-2 (Vera suite) in python -m vera ui → Convert/Chat — both repos are fetched.

Proven join snapshot

Direction	Site	Real join vs solo	Random control
GPT-2 → DistilGPT2	late	~1.01× solo ppl	tens–hundreds× worse
DistilGPT2 → GPT-2	early	~1.07×	collapse

Details: CROSS_MODEL_JOIN.md in the Vera repo.

Load

from vera.runtime import load_from_hf
partner = load_from_hf("kofdai/vera-distilgpt2-join", rank=256)

Must be paired with the Matryoshka GPT-2 bundle that shares the same P columns.

Status

Research partner checkpoint for stereo-cross join — not a standalone chat model.