build-small-hackathon/HF-Knight-Qwen2.5-1.5B-Instruct-GGUF

The HF Knight — Qwen2.5-1.5B-Instruct, fine-tuned

A small narrator model for The Adventures of the HF Knight, a medieval text RPG that teaches open-source / Hugging Face concepts. The player is a knight in the Thousand Token Wood; each trial is told as a short medieval story that poses a question. When the player answers, the model calls a validate_answer tool to submit it, then narrates the verdict in character.

Fine-tuned from Qwen/Qwen2.5-1.5B-Instruct so the whole game runs on a laptop / a single small GPU.

🏰 Built for the Hugging Face Build Small Hackathon

What the fine-tune actually teaches

The base model can already narrate and already has these facts. What it does not do on its own is what makes the game work:

1. Tool-call format (the essential one) — the un-tuned base answers conversationally and never emits <tool_call>{"name": "validate_answer", ...}</tool_call>, so the game never advances. The fine-tune teaches it to call the tool in Qwen's trained format on every answer.
2. A disciplined persona — the base narrates, but verbosely and inconsistently. The fine-tune shapes a concise, in-character Herald who poses the question as a brief medieval story and addresses the player by their current rank (Squire → Grandmaster).

Training

• Method: QLoRA SFT (4-bit nf4 base) with TRL + PEFT.
• LoRA: r=8, alpha=8 (scale 1.0), dropout 0.05, on all attention + MLP projections.
• Loss: assistant_only_loss=True — loss is computed only on the assistant turns (narration + tool calls), not on the prompt, so the model learns to generate conditioned on the persona rather than memorize and recite it.
• Data: 169 multi-turn traces hand-authored with Claude (data/traces.jsonl) — one full dialogue per trace, covering 90 questions across 6 stages with multiple answer paths (correct / wrong / clarify). Authored by hand because the larger dev model could not produce reliable tool-call traces.
• Train ≠ game (by design): the 90 training questions and the 60 questions in the live game are disjoint — zero overlap. The model never sees a real game question during training, so the fine-tune enhances a general skill (narrate any trial, call the tool) rather than memorizing specific answers. 90/10 train/eval split, seed 42.
• Schedule: 3 epochs, lr 2e-4, effective batch 8, max_len 2048.
• Result: held-out eval_loss (assistant tokens only) 1.91 → 1.42 over 3 epochs, no overfit.

Files

• hf-knight-qwen2.5-1.5b-instruct-q8_0.gguf — q8_0 quant (~1.6 GB) for llama.cpp / llama-cpp-python. This is the served model; it is what the Gradio Space runs. Distributed as GGUF, so it loads on a laptop CPU or a single small GPU.

Usage (llama-cpp-python, as in the game)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="build-small-hackathon/HF-Knight-Qwen2.5-1.5B-Instruct-GGUF",
    filename="hf-knight-qwen2.5-1.5b-instruct-q8_0.gguf",
    n_gpu_layers=-1, n_ctx=8192,
)
out = llm.create_chat_completion(
    messages=[{"role": "system", "content": "You are the Herald-Mentor ..."},
              {"role": "user", "content": "I am ready. Let us begin."}],
    tools=[...],  # the validate_answer schema
)

Limitations

• Single-purpose: it is trained to be the narrator for The Adventures of the HF Knight, not a general assistant.
• English only.