ettin-markup-17m

Reads how a roleplay message is marked up, on two axes:

• narration: asterisk (*She smiles.*), bare (She smiles.), or unknown
• dialogue: quoted ("Hi."), bare (Hi. next to asterisk narration), or unknown

unknown means there is nothing to read, or the message mixes both styles. Treat it as "leave this alone".

It exists so a chat app can keep a character's markup consistent without rewriting messages that were already fine.

Why does this even exist? Wouldn't a simple substring match be enough?

Consider: Narration *Thought* "Dialogue." This adds a lot of variants that can't be done accurately with algorithm-based heuristics.

Base	jhu-clsp/ettin-encoder-17m (ModernBERT)
Parameters	16,865,545
Head	one 9-way softmax over narration × dialogue, read as two marginals
Input	up to 512 tokens
Files	safetensors (fp32, 67 MB) · gguf/markup-17m-q8_0.gguf (19.7 MB) · onnx/model.onnx (fp32, 67.6 MB)
Speed	0.7 ms median per message (q8_0, llama.cpp, 4 CPU threads)

Use

The model was trained on lightly cleaned text: code blocks, bold runs and *** dividers blanked out, bullet stars turned into dashes. Do the same.

import re
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

REPO = "chartreuse-verte/ettin-markup-17m"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForSequenceClassification.from_pretrained(REPO).eval()

NARRATION = ["asterisk", "bare", "unknown"]
DIALOGUE = ["quoted", "bare", "unknown"]
PROTECTED = re.compile(r"```.*?```|\*{2,}[^\n]*?\*{2,}|_{2,}[^\n]*?_{2,}|[*_]{3,}", re.DOTALL)
BULLET = re.compile(r"[ \t]*\*(?=[ \t])")

def shape(text):
    lines = PROTECTED.sub(" ", text).split("\n")
    for i, line in enumerate(lines):
        m = BULLET.match(line)
        if m and "*" not in line[m.end():]:
            lines[i] = line[:m.end() - 1] + "-" + line[m.end():]
    return "\n".join(lines)[:4000]

def read(text):
    ids = torch.tensor([tok(shape(text))["input_ids"][:512]])   # the first 512 ids, as in training
    with torch.no_grad():
        logits = model(input_ids=ids, attention_mask=torch.ones_like(ids)).logits
    grid = logits.softmax(-1)[0].reshape(3, 3)
    return NARRATION[grid.sum(1).argmax()], DIALOGUE[grid.sum(0).argmax()]

print(read('*She sets the cup down.* "You came back."'))    # ('asterisk', 'quoted')
print(read("*She sets the cup down.* You came back, huh?"))  # ('asterisk', 'bare')
print(read('She sets the cup down. "You came back."'))       # ('bare', 'quoted')

Sum the grid's rows and columns, then take the argmax. Don't read both labels off the top cell.

The GGUF is the same model for llama.cpp:

from llama_cpp import Llama, LLAMA_POOLING_TYPE_RANK

llm = Llama("gguf/markup-17m-q8_0.gguf", embedding=True, pooling_type=LLAMA_POOLING_TYPE_RANK,
            n_ctx=512, verbose=False)
logits = llm.embed(shape('*She sets the cup down.* "You came back."'))[:9]   # the same 9 cells

embed returns a hidden-size vector; only its first 9 values are the cells.

The ONNX file takes input_ids and attention_mask (batch and length dynamic) and returns logits.

How well it works

Held-out validation, 5,967 messages, split by conversation:

	accuracy	macro-F1
narration	0.976	0.973
dialogue	0.989	0.978

As a rewrite gate, on 778 held-out chat windows (a new message and the three before it), with every rewrite judged by an LLM: 4 harmful rewrites (0.5%), against 39 (5.0%) for the regex heuristic it was built to replace. That is behind a rule that skips structured messages (lists, headings, transcripts, nested emphasis). Without the rule: 23 (3.0%).

Hand-written probes: 68/73 narration, 72/78 dialogue.

Limits

• English roleplay only.
• Weakest case: bare narration next to an asterisked beat or thought, with no quote marks. *Lena wipes the counter.* We're closed, Lena sighed. reads as asterisk narration with bare speech.
• It reads markup, not meaning. It can't tell an italic thought from an asterisked action.
• No human labels. Training labels come from a parser, plus an LLM on the hard cases. The rewrite judge is an LLM too.

Training

43,273 messages (and 5,967 for validation) from roleplay logs, character cards and real chat messages, plus two synthetic sets: swapped quote glyphs and boundary cases. 5 epochs, lr 4.5e-5, batch 32, cosine schedule, class weights capped at 5×, best checkpoint by validation macro-F1. The data is real conversation text, so it isn't published.

License

MIT, same as the base model.

Sibling of ettin-povtense-17m-v2.