code-daemon-enrich-v1

A distilled Qwen3-0.6B worker that writes the short, structured labels in the UltraCode code-intelligence pipeline: RAPTOR L0/L1 cluster labels, community labels, and link-selection picks. It replaces a 7B teacher on exactly the high-volume, prefill-bound, short-output stages where a sub-billion-parameter model is enough — running those stages on the daemon's dedicated .enrich worker at a fraction of the main LLM's per-call prefill cost.

This is a purpose-built pipeline component, not a general assistant. It only does the four label tasks below; outside that distribution its behaviour is undefined.

What it is

• Base: Qwen/Qwen3-0.6B (Apache-2.0) — 28 layers, ChatML, 151 936-token vocab.
• Teacher: Qwen/Qwen2.5-7B-Instruct.
• Method: sequence-level knowledge distillation (SeqKD) — LoRA SFT on teacher (prompt → response) traces, 3 epochs, prompt tokens masked; then merged into the base and exported to GGUF.
• Format: code-daemon-enrich-v1-Q8_0.gguf (~800 MB, Q8_0, llama.cpp). Q8_0 keeps the tiny model's logits crisp for short noun-phrase / single-token outputs.
• Tokenizer: Qwen2 family — so this model can double as a speculative-decoding draft for a Qwen3 target (shared tokenizer ⇒ aligned prefixes).

What it does — the four label tasks

System prompt bucket	Output	Example
SYS_RAPTOR_LABEL_L0 / L1	a cluster label "<topic>: name1, name2, name3"	"AST extractor methods: extractFnDecl, extractClassDecl, harvest"
SYS_COMMUNITY_LABEL	a graph-community label (noun phrase)	"OpenVINO pipeline state"
SYS_LINK_SELECT	candidate ids to link (c<N> picks or none)	"c2, c5"

All outputs are short (labels ≈ noun phrases; link-select emits ids). Long-form / multi-paragraph summaries are a different model (the long-output branch) — not this one.

Evaluation

Held-out set via a deterministic content-hash split (no RNG; the eval set is provably never in training). Metrics vs the teacher's recorded output on the same held-out prompts:

Bucket	ROUGE-L	Exact	Notes
LINK_SELECT	0.79	0.73	structured id-emit — the 0.6B matches the 7B on this task
COMMUNITY	0.40	0.21	usable label quality
L0 labels	0.39	0.04	token metrics undersell it — a valid paraphrase scores low on exact-token overlap

For L0, an embedding-cosine check (all-MiniLM-L6-v2, student vs teacher label) gives sem-cos ≈ 0.74 — a solid paraphrase, i.e. the label bulk semantically tracks the teacher even where ROUGE-L looks weak. Honest read: LINK is a clear win; L0/COMMUNITY are deployable for display/navigation labels, with a residual gap to the 7B's exactness on L0 (the next lever is on-policy distillation / more data).

Built for speed

The stages this model serves are prefill-bound with short outputs — the regime where shrinking the model (not speculative decoding) is the right lever. In the UltraCode daemon it loads into a dedicated .enrich worker (~0.9 GB VRAM, Q8_0 + KV) that co-resides with the main LLM, so the label stages run on the 0.6B while paragraph/prose stages stay on the larger model. Measured prefill throughput on the label batches: ~8–12k tok/s.

Usage (llama.cpp)

# ChatML; system prompt = one of the buckets above, user turn = the cluster/context.
llama-cli -m code-daemon-enrich-v1-Q8_0.gguf -c 8192 \
  -p '<|im_start|>system
Write a short label for this code/doc cluster. Format: "<topic phrase>: <name1>, <name2>, <name3>".<|im_end|>
<|im_start|>user
extractFnDecl, extractClassDecl, harvest, namedChild — an AST walker module.<|im_end|>
<|im_start|>assistant
'

Greedy decoding (temperature 0) is recommended — the outputs are factual labels.

Training data

Teacher (prompt, response) traces generated by Qwen2.5-7B-Instruct running the UltraCode knowledge-graph pipeline over a mixed multi-language code corpus (Zig, C#, TypeScript, Kotlin, Python, JavaScript). ~4,800 enrich-bucket traces after dedup. No third-party labeled dataset is used.

License & attribution

Apache-2.0 — matches the Qwen3-0.6B base and the Qwen2.5-7B-Instruct teacher (both Alibaba / Qwen team, Apache-2.0). Not legal advice. Base and teacher © the Qwen team; please also honour their model cards.