LissaNano-Instruct-GGUF

GGUF quantizations of LissaNano-Instruct.

Demo: Lissa Playground — compare quantizations side by side in the browser, no server.

The smallest My Little Pony language model: 32M parameters, Llama architecture, 1024-token context, 16384 vocab, trained from scratch on MLP fandom prose and then supervised on roleplay dialogue.

Parameters: 32M · Context: 1024 tokens · Base weights: bf16, quantized with llama.cpp

Quantizations

Perplexity was measured on the same held-out 300-token English text for every file and for the bf16 original, with llama.cpp llama-perplexity. Delta is relative to bf16 (2.945).

file	size	PPL	delta	verdict
LissaNano-Instruct-GGUF-Q6_K.gguf	25.6 MB	2.926	-0.6%	recommended
LissaNano-Instruct-GGUF-Q8_0.gguf	33.0 MB	2.941	-0.1%	reference
LissaNano-Instruct-GGUF-IQ4_XS.gguf	19.1 MB	2.971	+0.9%	recommended (imatrix)
LissaNano-Instruct-GGUF-Q5_K_M.gguf	23.0 MB	3.015	+2.4%	good
LissaNano-Instruct-GGUF-Q5_K_S.gguf	22.6 MB	3.027	+2.8%	good
LissaNano-Instruct-GGUF-Q4_K_M.gguf	20.6 MB	3.121	+6.0%	lower bound of safe
LissaNano-Instruct-GGUF-Q4_K_S.gguf	19.9 MB	3.21	+9.0%	at your own risk
LissaNano-Instruct-GGUF-Q3_K_L.gguf	18.9 MB	3.205	+8.8%	at your own risk
LissaNano-Instruct-GGUF-Q3_K_M.gguf	17.9 MB	3.323	+12.8%	at your own risk
LissaNano-Instruct-GGUF-IQ3_M.gguf	17.2 MB	3.296	+11.9%	at your own risk (imatrix)
LissaNano-Instruct-GGUF-Q3_K_S.gguf	16.8 MB	3.354	+13.9%	quality not guaranteed
LissaNano-Instruct-GGUF-IQ3_XS.gguf	16.3 MB	3.626	+23.1%	quality not guaranteed
LissaNano-Instruct-GGUF-Q2_K.gguf	15.4 MB	4.25	+44.3%	quality not guaranteed
LissaNano-Instruct-GGUF-IQ2_M.gguf	13.6 MB	4.054	+37.7%	quality not guaranteed
LissaNano-Instruct-GGUF-IQ2_XS.gguf	12.7 MB	5.326	+80.9%	quality not guaranteed

Q6_K is the sweet spot: smaller than Q8_0 and marginally better on held-out perplexity. For tight edge budgets take IQ4_XS (19 MB, +0.9%).

Nothing below Q4 is guaranteed. Q3 and IQ3 keep the topic and the format but lose voice and coherence. Q2 and IQ2 files are published for size experiments only — check the numbers above before using them in a product. Types that came out broken (Q2_0, TQ2_0, Q1_0 — perplexity thousands of times above bf16) are deliberately not uploaded.

IQ-quants were built with an importance matrix collected on the supervised fine-tuning corpus (400 samples, 100 chunks, llama-imatrix).

Prompt format (required)

The model was supervised on exactly this layout. Tags are concatenated without spaces, and a non-empty system message is required:

<system>{persona}<user>{message}<assistant>{reply}<eos>

Special tokens: 0 <pad>, 1 <eos>, 2 <system>, 3 <user>, 4 <assistant>.

BOS warning. The model was trained without a BOS token, and its bos_token is <eos>. Feeding a BOS degrades output badly — you get markup tables instead of dialogue. The published files already carry tokenizer.ggml.add_bos_token = false, so llama.cpp, Ollama and LM Studio behave correctly out of the box. If you re-quantize yourself, keep that flag.

Running

llama-simple -m LissaNano-Instruct-GGUF-Q6_K.gguf -n 120 \
  "<system>You are Twilight Sparkle, a studious purple unicorn who loves books.<user>Twilight, I found a strange book in the library.<assistant>"

Sampling: temp 0.8, top_p 0.9, repetition_penalty 1.1.

Ollama:

FROM ./LissaNano-Instruct-GGUF-Q4_K_M.gguf

Some chat wrappers inject their own template instead of the raw format. For exact reproduction use the completion path, not the chat path.

Intended use

phones, single-board computers, embedded and offline use; character agents; experiments with small-model persona consistency; a base for further fine-tuning.

License

Apache 2.0, same as the base model.