Qwen-Unitopia-Style

LoRA fine-tunes of Qwen models on the UNItopia LPC mudlib, a German LPMud library (sources and documentation from ftp://unitopia.de). The goal is a model that writes LPC in the UNItopia house style: file headers, inherit lines, the room, item and monster APIs, and the German documentation format. The current release is built on Qwen3.8-27B.

Files

Everything is in qwen3.8-27b/:

File	Size	What
qwen3.8-27b-unitopia-q8_0.gguf	29.0 GB	The fine-tune. Q8_0, near-lossless, includes the multi-token-prediction block
qwen3.8-27b-unitopia-q4_k_m.gguf	16.8 GB	The fine-tune in plain Q4_K_M (no importance matrix), for smaller cards
qwen3.8-27b-base-q8_0.gguf	29.0 GB	The unmodified base model in the same Q8_0 conversion, for A/B comparison
adapter-pretrain/, adapter-sft/	435 MB each	PEFT LoRA adapters of the two phases (rank 16, alpha 32, 109M params). adapter-sft is what the merged GGUFs contain; load it on Qwen/Qwen3.8-27B with PeftModel.from_pretrained
metrics-*.jsonl, run-*.json		Training curves and resolved arguments
SHA256SUMS		Checksums of the three GGUFs

All GGUFs are text-only conversions with llama.cpp (convert_hf_to_gguf.py); the vision tower of the base checkpoint is not included. The Q8_0 files were converted directly from bf16 safetensors, the Q4_K_M with llama-quantize at default settings.

Also here: Qwen3.5-35B-A3B ("flash-lite")

qwen3.5-35b-a3b/ holds the same fine-tune applied to Qwen3.5-35B-A3B, a mixture-of-experts model with 3B active parameters: qwen3.5-35b-a3b-unitopia-q8_0.gguf (37.8 GB, Q8_0 with MTP, GGUF architecture qwen35moe), the two LoRA adapters and the training metrics. It answers at roughly the speed of a 3B model with the knowledge of a 35B one; final eval loss 1.14 against the 27B's 0.97, so the 27B writes better code and the 35B-A3B answers faster.

Also here: Gemma 4 31B

gemma-4-31b/ holds the same fine-tune on Google's Gemma 4 31B (google/gemma-4-31B-it, dense, Apache 2.0), trained on one H200 on 2026-10-02:

File	Size	What
gemma-4-31b-unitopia-q8_0.gguf	32.6 GB	The fine-tune, Q8_0
gemma-4-31b-unitopia-q4_k_m.gguf	18.7 GB	The fine-tune in plain Q4_K_M (no importance matrix)
gemma-4-31b-unitopia-bf16.gguf	61.4 GB	The fine-tune unquantized (bf16), the source for your own quantizations
gemma-4-31b-base-q8_0.gguf	32.6 GB	The unmodified base model in Q8_0, for A/B comparison
adapter-pretrain/, adapter-sft/	490 MB each	PEFT LoRA adapters (rank 16, alpha 32, 122M params) for google/gemma-4-31B-it
metrics-*.jsonl, run-*.json, SHA256SUMS		Training curves, resolved arguments, checksums of the four GGUFs

Same two phases and settings as the Qwen runs: phase 1 (causal LM on the mudlib) 139 min, eval loss 6.22 → 1.03; phase 2 (instruction pairs) 38 min, 1.10 → 1.02. The high phase-1 start is the instruction-tuned Gemma's poor fit to raw non-chat text (llama.cpp measures the same on the base model), and phase 1 removes it within a hundred steps. Training used the chat template with thinking off; Gemma 4 switches thinking on with <|think|> in the system prompt (enable_thinking), and the model works either way. The GGUFs are text-only (GGUF architecture gemma4) and carry Gemma's chat template.

How it was trained

Two LoRA phases with PyTorch and PEFT on one RTX PRO 6000 (96 GB):

Phase	Data	Optimizer steps	Time	Eval loss start → end
1 causal LM	mudlib sources and docs, 4411 windows of ≤2048 tokens, 3.97M target tokens, 2 epochs	1102	274 min	1.51 → 0.97
2 SFT	2815 instruction pairs derived from the sources (write this file, explain this help page, where is X), 2 epochs	703	78 min	1.02 → 0.97

LoRA on all attention, MLP and gated-delta-net projections, bf16 base, gradient checkpointing, AdamW lr 2e-4 with warmup and cosine decay. Eval loss is measured on a held-out 5% of the same data. For scale, the same recipe gave 1.38 on Qwen3-0.6B, 1.19 on Qwen3-4B-Instruct-2507 and 1.06 on Qwen3.5-9B.

Using it

Prompts in German work best, phrased like the training data:

Implementiere `/room/kirche/treppe5.c` für die UNItopia Mudlib (Die Treppe zum Kirchturm).
Dokumentation für die Hilfeseite `rm` (UNItopia Mudlib)?
Schreibe einen einfachen NPC für die UNItopia Mudlib: ein Bäcker, der Brot verkauft und auf 'hallo' antwortet.

Training used the chat template with thinking disabled (enable_thinking=False); the model still works with thinking on. Tool calling from the base model is intact. With llama.cpp:

llama-cli -m qwen3.8-27b-unitopia-q8_0.gguf -ngl 99 -cnv

What to expect

It writes idiomatic UNItopia LPC, reproduces the documentation format, and composes new objects from the mudlib's conventions (a seller inherits /i/money/verkaeufer, an NPC /i/monster/monster with monster::create()). It is a junior builder, not a senior one: when it does not remember a function it tends to invent a plausible name instead of saying so. Give it file access to the mudlib and a system prompt that tells it where the mudlib lives and to look functions up before using them, and review what it writes before anything reaches players.

Licences

The base models are Qwen3.8-27B and Qwen3.5-35B-A3B under their own licences and Gemma 4 31B under Apache 2.0. The training data is the UNItopia mudlib and documentation, which are licensed for non-commercial use only; treat these fine-tunes the same way. Base weights are redistributed here only as a conversion for comparison.