LissaBrat-Instruct-GGUF

GGUF quantizations of LissaBrat-Instruct.

Demo: Lissa Playground — compare quantizations side by side in the browser, no server.

Middle sibling of the Lissa family: 103M parameters, Llama architecture, 1024-token context, 16384 vocab, trained from scratch on MLP fandom prose and then supervised on roleplay dialogue.

Parameters: 103M · Context: 1024 tokens · Base weights: bf16, quantized with llama.cpp

Quantizations

Perplexity was measured on the same held-out 300-token English text for every file and for the bf16 original, with llama.cpp llama-perplexity. Delta is relative to bf16 (3.528).

file	size	PPL	delta	verdict
LissaBrat-Instruct-GGUF-Q8_0.gguf	124.7 MB	3.527	0.0%	reference
LissaBrat-Instruct-GGUF-Q5_K_S.gguf	98.1 MB	3.458	-2.0%	good
LissaBrat-Instruct-GGUF-Q4_K_M.gguf	90.6 MB	3.469	-1.7%	recommended
LissaBrat-Instruct-GGUF-Q6_K.gguf	106.1 MB	3.571	+1.2%	good
LissaBrat-Instruct-GGUF-Q5_K_M.gguf	98.2 MB	3.556	+0.8%	good
LissaBrat-Instruct-GGUF-Q4_K_S.gguf	90.6 MB	3.54	+0.3%	good
LissaBrat-Instruct-GGUF-Q3_K_L.gguf	84.8 MB	3.544	+0.5%	at your own risk
LissaBrat-Instruct-GGUF-Q3_K_M.gguf	83.7 MB	3.568	+1.1%	at your own risk
LissaBrat-Instruct-GGUF-IQ4_XS.gguf	88.8 MB	3.608	+2.3%	recommended (imatrix)
LissaBrat-Instruct-GGUF-Q3_K_S.gguf	82.5 MB	3.724	+5.6%	quality not guaranteed
LissaBrat-Instruct-GGUF-IQ3_XS.gguf	80.7 MB	3.781	+7.2%	quality not guaranteed
LissaBrat-Instruct-GGUF-IQ3_M.gguf	83.6 MB	3.875	+9.8%	quality not guaranteed
LissaBrat-Instruct-GGUF-Q2_K.gguf	77.3 MB	3.844	+9.0%	quality not guaranteed
LissaBrat-Instruct-GGUF-IQ2_XS.gguf	72.1 MB	3.862	+9.5%	quality not guaranteed
LissaBrat-Instruct-GGUF-IQ2_M.gguf	74.8 MB	3.89	+10.3%	quality not guaranteed

Q4_K_M is the practical pick (90 MB, -1.7% vs bf16 — within noise). IQ4_XS gives 88.8 MB at +2.3%.

Nothing below Q4 is guaranteed. Q3 and IQ3 keep the topic and the format but lose voice and coherence. Q2 and IQ2 files are published for size experiments only — check the numbers above before using them in a product. Types that came out broken (Q2_0, TQ2_0, Q1_0 — perplexity thousands of times above bf16) are deliberately not uploaded.

IQ-quants were built with an importance matrix collected on the supervised fine-tuning corpus (400 samples, 100 chunks, llama-imatrix).

Prompt format (required)

The model was supervised on exactly this layout. Tags are concatenated without spaces, and a non-empty system message is required:

<system>{persona}<user>{message}<assistant>{reply}<eos>

Special tokens: 0 <pad>, 1 <eos>, 2 <system>, 3 <user>, 4 <assistant>.

BOS warning. The model was trained without a BOS token, and its bos_token is <eos>. Feeding a BOS degrades output badly — you get markup tables instead of dialogue. The published files already carry tokenizer.ggml.add_bos_token = false, so llama.cpp, Ollama and LM Studio behave correctly out of the box. If you re-quantize yourself, keep that flag.

Running

llama-simple -m LissaBrat-Instruct-GGUF-Q6_K.gguf -n 120 \
  "<system>You are Twilight Sparkle, a studious purple unicorn who loves books.<user>Twilight, I found a strange book in the library.<assistant>"

Sampling: temp 0.8, top_p 0.9, repetition_penalty 1.1.

Ollama:

FROM ./LissaBrat-Instruct-GGUF-Q4_K_M.gguf

Some chat wrappers inject their own template instead of the raw format. For exact reproduction use the completion path, not the chat path.

Intended use

laptops and small servers; longer and more coherent text than Nano; character agents; experiments with small-model persona consistency; a base for further fine-tuning.

License

Apache 2.0, same as the base model.