bernardw/qwen2.5-coder-1.5b-codereview-taid-q4-mlc

Qwen2.5-Coder-1.5B Code-Review (TAID) — MLC q4f16_1 (browser / WebGPU)

4-bit (q4f16_1) MLC build for in-browser WebLLM/WebGPU deployment. Layout: mlc-chat-config.json + ndarray-cache.json + params_shard_*.bin. The student architecture is stock Qwen2.5-Coder — load with the matching prebuilt WebLLM model lib. Held-out F1 0.482. Use the prebuilt Qwen2.5-Coder-1.5B model lib (~0.84 GB download).

Method. Cached-logit knowledge distillation on Apple MLX: run the teacher once, cache top-k=50 logits over its response positions, train a LoRA student (rank 16 / 16 layers, lr 1e-4, seq 768) against them, fuse. Held-out eval eval_set_100 (100 labeled chunks, 73 buggy / 27 clean; py/js/c/go).

Prompt contract. Given a numbered code chunk, emit one JSON object {"findings":[{category, subtype, severity, confidence, title, body, evidence, line}]}; evidence is a verbatim source substring, line 1-based. Use repetition_penalty ~1.15 to keep JSON valid.

Project findings (honest). Teacher selection dominated: a 3B distilled from Gemma-2-9B ties the Qwen-7B teacher (F1 0.509); per-category teacher routing added nothing (-0.003); 1.6x more in-distribution data did not raise the ceiling. TAID (logit KD) > SFT with the Qwen-7B teacher (0.478 vs 0.410) but both over-train past ~3000 steps on this small corpus.