---

LissaBrat-Instruct

1. Introduction

103 million parameters of pure pony language, still only 206 MB on disk. This is the supervised checkpoint, tuned for character-driven conversation.

• Trained from scratch. No distilled weights and no borrowed backbone: a Llama architecture with a 16,384-token vocabulary, pretrained on a fandom corpus and then supervised on dialogue.
• Built for character work. The supervised stage uses roleplay and persona data, so the model spends its capacity on voice, character and scene.
• Dialogue format is required. The model expects the exact <system> / <user> / <assistant> layout described in section 6.
• Trained by one person on one consumer GPU. No cluster, no budget, no institutional support.

2. Model Overview

field	value
Type	Causal Language Model
Training stage	Pretraining from scratch, then supervised fine-tuning
Architecture	Llama
Parameters	102,760,704 (103M)
Layers	8
Hidden size	1,024
Attention heads	32 (4 key/value heads)
Head dimension	32
Intermediate size	2,730
Context length	1,024 tokens
Vocabulary	16,384 (byte-level BPE)
Tied embeddings	yes
Precision	bfloat16
License	Apache 2.0

3. The Lissa family

model	parameters	weights	best for
LissaNano-Base	32M	64 MB	continued pretraining, research
LissaNano-Instruct	32M	64 MB	phones, single-board computers, embedded use
LissaBrat-Base	103M	206 MB	continued pretraining, research
LissaBrat-Instruct	103M	206 MB	laptops, longer and more coherent text
Lissa 1.5B Instruct	1.45B	2.9 GB	a full roleplay partner
Lissa 1.5B Base	1.45B	2.9 GB	pretraining, research

GGUF quantizations: LissaBrat-Instruct-GGUF — 15 files from Q8_0 down to IQ2, each with measured perplexity against this bf16 checkpoint.

Demo: Lissa Playground — compare quantizations side by side in the browser, no server.

4. Requirements

pip install torch transformers

No custom kernels and no exotic dependencies: this is a plain Llama checkpoint and loads with any recent transformers.

5. Quick start

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CastIronMind/LissaBrat-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16)
model.eval()

SYS, USR, AST, EOS = 2, 3, 4, 1

system = "You are Twilight Sparkle, a studious purple unicorn who loves books."
user = "Twilight, I found a strange book in the library."

ids = [SYS] + tok.encode(system, add_special_tokens=False) \
    + [USR] + tok.encode(user, add_special_tokens=False) + [AST]
x = torch.tensor([ids], dtype=torch.long)

out = model.generate(x, max_new_tokens=160, do_sample=True,
                     temperature=0.8, top_p=0.9, repetition_penalty=1.1,
                     pad_token_id=0, eos_token_id=EOS)
print(tok.decode(out[0][len(ids):], skip_special_tokens=True))

6. Prompt format

The model was supervised on exactly this layout:

<system>{persona}<user>{message}<assistant>{reply}<eos>

Tags are concatenated without spaces. A non-empty system message is required: every supervised example carried one.

Special tokens. 0 <pad>, 1 <eos>, 2 <system>, 3 <user>, 4 <assistant>.

Sampling. Temperature 0.8, top_p 0.9, repetition penalty 1.1.

7. Data

Pretraining. Fandom prose and forums from the My Little Pony community, about 1.5 billion unique tokens, two epochs.

Supervised fine-tuning. Roleplay and persona dialogues with the <system> / <user> / <assistant> layout, two epochs.

8. Evaluation

This checkpoint is tuned for dialogue, so held-out perplexity is reported on the base model card, where it is measured on the same frozen set for every model in the family: see LissaBrat-Base.

What the supervised stage targets: character voice, in-persona dialogue, and stable formatting across turns.

9. Hardware and cost

item	value
Training hardware	one consumer GPU
Authors	one person
Weights on disk	206 MB (bfloat16)
License	Apache 2.0

10. Intended use

• roleplay bots and character agents in chat applications
• embedded and offline deployments: games, visual novels, toys
• experiments with character consistency in small models
• further fine-tuning on a specific character; the model retrains in minutes

11. License

Apache License 2.0. Use it, fine-tune it, quantize it, embed it, and publish your own versions.