minsore/quill-1-preview

🟢 Quill 1 Preview

Lightweight FIM autocomplete model for Python, built on Qwen2.5-Coder-0.5B.

Quill is a 0.5B parameter code-completion model designed for Fill-in-the-Middle (FIM) tasks in IDEs and editors. It runs comfortably on consumer hardware and is one of the few publicly available 0.5B models trained specifically for autocomplete.

Part of the Minsore family · minsore.com

---

✨ Highlights

• 🧠 FIM-native — understands <fim_prefix>, <fim_suffix>, <fim_middle>
• ⚡ Real-time — designed for live autocomplete in editors
• 📦 Tiny — 0.5B params, ~400 MB in Q4_K_M
• 🎯 Purpose-built — trained for autocomplete, not chat
• 🆓 Apache 2.0 — same license as base model

---

📊 Benchmarks

Evaluated against 0.5B–1B code models. All runs used temperature=0.0, raw FIM mode (--chat-template none).

Benchmark	Quill 1 Preview	Arche-Codium 0.5B	Granite 4.0 1B	Qwen2.5-Coder 0.5B
HumanEval@50	54.0%	96.0%	58.0%	52.0%
HumanEval-Infilling (EditSim)	0.423	0.053	0.041	0.045
HumanEval-Infilling (Exact Match)	16%	0%	0%	0%
NextBench v0.2 (autocomplete)	45.0%	5.0%	25.0%	41.7%
Delulu FIM (EditSim)	0.318	0.087	0.041	0.035

📌 Key insight: On FIM-specific benchmarks, Quill outperforms competitors by 8–10×. On pure code generation (HumanEval), it matches the base model — training added FIM capability without hurting generation.

---

🚀 Quick Start

llama.cpp

llama-server -m quill-1-preview.Q4_K_M.gguf \
  --port 8080 \
  -ngl 99 \
  --chat-template none

Requirements: any GPU with ≥2 GB VRAM (full offload), or partial CPU offload as fallback.

FIM request

curl http://localhost:8080/infill \
  -H "Content-Type: application/json" \
  -d '{
    "input_prefix": "def add(a, b):\n    ",
    "input_suffix": "\n\nprint(add(2, 3))",
    "n_predict": 64,
    "temperature": 0.0
  }'

Response:

{
  "content": "return a + b",
  "stop": true
}

Python

import requests

def fim_complete(prefix, suffix, max_tokens=64):
    r = requests.post("http://localhost:8080/infill", json={
        "input_prefix": prefix,
        "input_suffix": suffix,
        "n_predict": max_tokens,
        "temperature": 0.0,
        "stop": ["<fim_end>", "<|endoftext|>"],
    })
    return r.json()["content"]

print(fim_complete(
    "def fib(n):\n    if n <= 1:\n        return n\n    return ",
    "\n\nprint(fib(10))"
))
# → "fib(n - 1) + fib(n - 2)"

---

⚙️ Recommended Settings

Parameter	Value	Notes
n_predict	128–256	Works best on short completions
temperature	0.0	Deterministic; for creative variation use 0.2
repeat_penalty	1.1	Prevents number-looping
stop	`["", "<	endoftext
--chat-template	none	Required. Quill is not a chat model

---

⚠️ Limitations

• Autocomplete only — does not work as a chat assistant
• Occasionally appends tests after the function (inherited from mixed training data)
• Number looping — may repeat large integers on some prompts (use repeat_penalty=1.1)
• 1024 token context — long files are truncated
• Python-only — trained on Python FIM data
• Hallucinations on unfamiliar APIs (EditSim 0.318 on Delulu FIM benchmark)

---

🧬 Training Details

Base model	Qwen2.5-Coder-0.5B-Instruct
Method	QLoRA (r=16, α=16)
Data	Mixed FIM + Python instruction data
Epochs	1
Context	1024 tokens
FIM tokens	<fim_prefix>, <fim_suffix>, <fim_middle>, <fim_pad>

---

📁 Files

File	Size	Description
quill-1-preview.Q4_K_M.gguf	~400 MB	Ready to use (recommended)
quill-1-preview-f16.gguf	~1 GB	Full precision GGUF
lora/	~35 MB	LoRA adapter for fine-tuning

---

🗺️ Roadmap

• Quill 2 — fix number looping, raise pattern_hit to 70%+
• Quill 2 — multi-language support (JS/TS/Rust)
• Pepper — general-purpose code assistant (1.7B)
• Symphony — flagship code model (3B)

---

📜 License

Apache 2.0 — same as the base Qwen2.5-Coder-0.5B model.

---

🙏 Credits

• Base model: Qwen2.5-Coder-0.5B-Instruct by Alibaba Cloud
• Training framework: Unsloth
• Inference: llama.cpp

---

📬 Contact

Minsore — Ukrainian AI lab building open language models.

• 🌐 minsore.com
• 🤗 huggingface.co/Minsore
• 💬 Built by @Sollamon

---

⭐ If Quill is useful, star the repo and share your results.