YeMoKoo/hongsamjin

abclore — CLoRE and baseline checkpoints (weights only)

No optimizer state, data caches, replay memories or trainer state: these checkpoints are for evaluation, not for resuming training.

flame_wcc/ — FLAME-MoE, Wiki → Code → Conversation (Megatron torch_dist)

Each stage dir holds iter_XXXXXXX/ plus latest_checkpointed_iteration.txt and run_metadata.json. Load with --load <stage dir> --no-load-optim --no-load-rng (optimizer entries were removed; every model tensor is bit-identical to the training checkpoint).

path	what	final probe acc (wiki / code / conv)
clore/{wiki,code,conversation}	CLoRE (mass reservoir), hybrid FFN + QKVO LoRA experts, 8 → 16 → 24	46.20 / 67.48 / 38.05, AA 50.58, FM 0.26
baselines/wiki_source_ffn_e8	shared Wiki start point of the baselines (FFN-only, 8 experts)	wiki 46.00
baselines/moe_lpr_g0.1/{code,conversation}	MoE-LPR, γ = 0.1, task labels (after router review, iter 2160)	AA 49.80
baselines/moe_lpr_g0.1/{code,conversation}_prereview	same, before the review (forgetting reference)
baselines/lifelong_moe_kl1.0/{code,conversation}	Lifelong-MoE, λ_KL = 1.0	AA 37.28

trace/ — TRACE (8 tasks, sparse-15 evaluation)

Every run dir has rounds 0 … 7 (after task 1 … 8; 7 = final), evaluation/order*/results-*.json, sparse15_summary.json, the train command and run config.

path	AA	F
llama31/baselines/seq_lora	58.44	8.35
llama31/baselines/ewc	58.69	7.48
llama31/baselines/olora	51.81	6.55
llama31/baselines/slora_r64 (S-LoRA, merge-scaling fix; orderN/ = after task N; evaluation/orderN/<task>/infer.jsonl)	56.08	10.56
llama31/baselines/moe_lpr_g0.1	55.91	0.41
llama31/baselines/lifelong_moe_kd1.5	35.11	16.00
llama31/clore_ablation/no_reservoir (reservoir off, random new rows)	62.79	0.58
llama31/clore_ablation/learned_bos (learned <BoS_task> generation token; cond/ = token embeddings + masks)	60.02	6.69
qwen3_8b/baselines/mtl (final only)	65.72	–
qwen3_8b/baselines/moe_lpr_g0.1 (prereview/ = before router review)	59.62	5.50
qwen3_8b/baselines/lifelong_moe_kd1.5	43.20	9.77

AA / F from each run's sparse15_summary.json (final_average, -BWT); S-LoRA from RESULT.md. Base models: Llama-3.1-8B-Instruct, Qwen3-8B (think-off, --conv-mode qwen3).