pointbreaklab/knot-scribe

Knot Scribe

On-device model that writes your git commit messages. Give it a diff, get a correctly-typed Conventional-Commit message back. Runs 100% on your machine, no API, no code leaving your device.

Part of the Knot AI family, alongside Knot Delta (which reads a change and writes a grounded report). Knot Scribe is the model behind Knot's hands-free auto-commit, released standalone here.

Renamed: this repo was formerly pointbreaklab/knot-ai. The old URL redirects here.

A LoRA fine-tune of Qwen2.5-Coder-7B-Instruct (Apache-2.0), trained only on public commit history. No user data, no private code.

Benchmark — v5

Sixty hand-reviewed commits (gold-v1), balanced across all eleven conventional-commit types. Head-to-head of the previous release (v4) and the current model (v5).

metric	v4	v5
Type accuracy	48.3%	60.0%
Scope accuracy	68.3%	75.0%
Title word-overlap	27.2%	19.1%
Exact title	6.7%	0.0%
Generation errors / 60	7	4

Type accuracy across releases: v3 40% → v4 55% → v5 60%. v3/v4 were scored on an earlier JS/TS-only set; v5 on the harder balanced set above, where v4 re-scores 48.3% — so v5's real gain over v4 is +11.7 points.

Honest read: v5 lifts type accuracy and scope and nearly halves the format errors. Title word-overlap and exact-title fall because v5 paraphrases: it writes its own titles, often more descriptive than the single human reference it is scored against, which exact string-match penalizes. build and revert remain hard.

Use it

Ollama

ollama run pointbreaklab/knot-scribe "$(git diff --staged)"

llama.cpp (base + this LoRA)

llama-server --model qwen2.5-coder-7b-instruct-q4_k_m.gguf \
             --lora knot-scribe-lora.gguf

Output is JSON {"title": "...", "body": "..."} — a correctly-typed Conventional-Commit subject and body.

Training

• Base: Qwen2.5-Coder-7B-Instruct · Apache-2.0
• Method: QLoRA (4-bit), completion-only SFT (loss on the message only)
• Data: diff → message pairs mined from permissive open-source commit history (JS/TS, Python, Rust, Go)
• Privacy: public commit history only — no user data, no private code
• License: Apache-2.0 (derivative of Qwen2.5-Coder-7B)