gerbil-qwen3-coder-30b-gguf

GGUF quants of jaimef21/gerbil-qwen3-coder-30b-bf16, fine-tuned from Qwen3-Coder-30B-A3B-Instruct for Gerbil Scheme generation.

Training pipeline and tooling: https://github.com/ober/gerbil-lora

Files

File	Quant	Size	Notes
gerbil-qwen3.q8_0.gguf	Q8_0	~32 GB	High fidelity; indistinguishable from F16 for most use
gerbil-qwen3.q4_k_m.gguf	Q4_K_M	~17 GB	Runs comfortably in 24 GB VRAM

Usage with llama.cpp

./llama-cli -m gerbil-qwen3.q4_k_m.gguf -p "Write a Gerbil function to..."

Usage with Ollama

Build a Modelfile locally:

FROM ./gerbil-qwen3.q4_k_m.gguf

SYSTEM "You are an expert in Gerbil Scheme, a dialect of Scheme built on Gambit. You provide accurate, idiomatic Gerbil code with correct imports, function names, and arities. Module paths use the :std/* form (e.g. :std/sort, :std/iter, :std/text/json). The idiomatic definition form is `def`, not `define`."

PARAMETER temperature 0.2
PARAMETER num_ctx 32768
PARAMETER stop "<|im_end|>"

Training pipeline

Three-stage LoRA fine-tune (r=32, α=64, fused-MoE expert targets):

1. CPT — Continued pre-training on Gerbil source corpus (lr 2e-5, 2 epochs)
2. SFT — Supervised fine-tune on instruction/response pairs (lr 1e-4, 2 epochs)
3. DPO — Direct preference optimization on wrong→right pairs (lr 5e-6, 3 epochs)

DPO eval (vs base Qwen3-Coder-30B-A3B-Instruct)

Metric	Base	Trained	Δ
Holdout task score	31	39	+8
Anti-idioms hit	1	0	-1
Code blocks wrapped	9	14	+5
tok_lean_sum (P(chosen) > P(rejected))	-4.17	+4.03	+8.19
wins chosen / rejected (n=66)	47 / 19	52 / 13	+5 / -6