VertexAGI/prism-roleplay-2-small

Prism Roleplay 2 Small

Better multi-turn roleplay than 1.5, plus optional private inner thoughts and emotional-state tracking. Qwen3-8B, LoRA-tuned.

---

What changed from Prism Roleplay 1.5 Small to 2

Prism Roleplay 1.5 Small was strong on single scenes but weak in long chats. Prism Roleplay 2 Small was built to fix that and to add two new abilities.

	1.5 Small	2 Small
Multi-turn chats	Not specifically targeted	Trained on 1,575 multi-turn chats, including short-history chats so replies don't shrink to match a terse history
Training signal	Loss on every token (mlx-lm default)	Loss on the final reply only for chat rows, so the model learns to answer, not to copy earlier turns
Private inner thoughts	No	Optional inner_thoughts tool call (first person, ~60 words, written before the reply)
Emotional state	No	Optional track_state tool call (six 0-100 values before/after, plus a reason)
Memory of earlier sessions	Not targeted	Trained on replies that use remembered details
LoRA	rank 8, 2,048-token sequences	rank 16, 3,072-token sequences
Training data	Forum transcripts + synthetic	Synthetic + public-domain chats + character-card chats (details and credits below)

Results vs Prism Roleplay 1.5 Small

76 hand-written prompts, blind, each pair judged in both orders, and a win only counts when both orders agree from both of two judges (otherwise it's a tie). Counts are Prism Roleplay 2 wins / 1.5 wins / ties.

Prompt set	Standard judging	Length-neutral judging*
All 76	43 / 2 / 31	39 / 2 / 34
Single scenes (24)	10 / 0 / 14	10 / 0 / 14
Memory (16)	3 / 2 / 11	3 / 2 / 10
Multi-turn (12)	7 / 0 / 5	7 / 0 / 5
Multi-turn, short history (12)	12 / 0 / 0	9 / 0 / 3
Multi-turn, long history (12)	11 / 0 / 1	10 / 0 / 2
Replies of similar length only (22)	9 / 0 / 13	10 / 0 / 11

*The length-neutral pass tells the judges not to reward length. Prism Roleplay 2 writes longer replies (median 126 words vs 80), so this pass checks the win isn't just length.

The biggest gain is multi-turn. Memory is a modest gain only.

vs the untuned Qwen3-8B base

52 prompts (single scenes, memory, multi-turn), one judge (nemotron-3-ultra), both orders must agree. Prism Roleplay 2 wins / base wins / ties: 32 / 6 / 14. Multi-turn: 11 / 0 / 1. Format check (asterisks for actions, quotes for speech): 40/40 passed.

Inner thoughts and state tools

Judged against the base on 12 scenes x 2 samples per tool (both judges, both orders; the judged outputs came from the unmerged adapter, the validity and length rows from the released build):

	Prism Roleplay 2	Qwen3-8B base
Thoughts judged better / worse / tie	5 / 0 / 19
State tracking judged better / worse / tie	1 / 0 / 23
Valid thoughts calls (this release's 6-bit build; 24/24 with the unmerged adapter)	23/24	24/24
Valid state calls (this release's build; 24/24 with the adapter)	23/24	24/24
Median thought length	64 words	47 words
Thoughts containing *stage directions*	0/23	0/24
State: mean total change per step (lower = calmer)	65.6	29.1

Honest read: the thoughts are clean and usable, and slightly better than the base. State tracking is valid but over-reactive: it changes almost every value on almost every turn, so treat its numbers as rough.

Limitations

• Evaluation is small and automated. 76 and 52 hand-written prompts, judged by two language models (nemotron-3-ultra and muse-glimmer-30b). The two judges agreed on only 49 of 76 pairs in the main comparison. No human evaluation. One of the judges (Muse Glimmer) also wrote part of the training thoughts, which may favour this model slightly.
• Replies are longer than 1.5's (126 vs 80 words median). Some people won't want that.
• Memory improved only a little (3 wins, 2 losses, 11 ties vs 1.5).
• State tracking over-reacts, as above.
• Not a safety-tuned model. The data was filtered to remove sexual content, but the Qwen3 base can still produce anything. Don't treat it as a moderation layer.
• It is an 8B model fine-tuned with LoRA; very long sessions can still drift.

Formats available

Available formats

Everything below lives in this one repository: pick the format that matches your runtime.

Format	File / folder	Precision	Size	Works with
MLX	model-00001-of-00002.safetensors (+1 more, repo root)	4-bit (group size 64)	5.38 GB	mlx-lm / MLX on Apple Silicon
GGUF	prism_roleplay_2_small_Q4_K_M.gguf	Q4_K_M (4-bit)	5.03 GB	llama.cpp, Ollama, LM Studio
GGUF	prism_roleplay_2_small_Q8_0.gguf	Q8_0 (8-bit)	8.71 GB	llama.cpp, Ollama, LM Studio

The MLX build is mixed precision: the 16 LoRA-tuned layers are 6-bit, every other layer is the original 4-bit (per-layer bits are in config.json). The GGUF files are made from the exactly merged fp16 model.

Usage: MLX

from mlx_lm import load, generate

model, tokenizer = load("VertexAGI/prism-roleplay-2-small")
system = ("You are Mara, a lighthouse keeper. The user plays a stranger who arrives in a storm. "
          "Stay fully in character as Mara and write only Mara's next turn: speech in double quotes where natural, actions in *single asterisks*.")
messages = [{"role": "system", "content": system}, {"role": "user", "content": "*I knock on the door, soaked through.* Please, I need shelter."}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=text, max_tokens=400))

Usage: GGUF (llama.cpp)

Two builds are included, made directly from the fp16-merged model (the LoRA is merged exactly, not re-quantized): Q8_0 (near-lossless) and Q4_K_M (smaller). Use the model's default chat template. Do not pass -rea off / --reasoning off or enable_thinking=False: the model was trained without the empty think block those options insert, and with it the tool calls become unreliable (in our check, track_state calls dropped to about half). Add --jinja for the tools:

llama-cli -hf VertexAGI/prism-roleplay-2-small -m prism_roleplay_2_small_Q4_K_M.gguf -st -sys "You are Mara, a lighthouse keeper. Stay fully in character as Mara and write only Mara's next turn: speech in double quotes where natural, actions in *single asterisks*."
llama-server -m prism_roleplay_2_small_Q4_K_M.gguf --jinja

Tool check on the GGUF builds (llama-server with --jinja, default template, 12 scenes per tool, temperature 0.7, top-p 0.95; valid calls out of 12):

Build	inner_thoughts valid	track_state valid
Q8_0	12/12	12/12
Q4_K_M	12/12 (twice)	8/12 and 12/12 in two runs

Q8_0 is the safer choice if you rely on the tools; Q4_K_M is fine for plain roleplay.

System prompt

Every evaluation on this card used this short system prompt, with the scene described in the user's first message (the chat Space uses it too):

You are the character in the scene the user describes. Stay fully in character and write your next reply.

The model was also trained with longer per-character system prompts (a character section and a scenario section), which work as well.

Optional tools

Pass the two tools to the chat template and the model will call one before its reply. The call is private (never show it to the user); answer the tool result with "ok" and the model then writes the reply.

tools = [
 {"type": "function", "function": {"name": "inner_thoughts", "description": "Privately think as the character before replying: what they notice, feel and decide. Never shown to the user.",
   "parameters": {"type": "object", "properties": {"thought": {"type": "string", "description": "First-person inner monologue in the character's own voice."}}, "required": ["thought"]}}},
]
text = tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True, tokenize=False)

Add a line to the system prompt that asks for the call, as in our evaluations: "Before every reply, call the inner_thoughts tool with the character's private first-person thoughts, then reply in character." (For state tracking: "Before every reply, call the track_state tool with the character's emotional state before and after this reply (0-100 per dimension) and a one-sentence cause, then reply in character.") Without such a line it normally just replies. track_state takes previous and current (each with trust, anger, fear, joy, sadness, affection, 0-100) and a one-sentence reason.

Sampling used in all evaluations: temperature 0.7, top-p 0.95.

Training

• Base: mlx-community/Qwen3-8B-4bit (Qwen3-8B).
• Method: LoRA, rank 16, scale 20, 16 layers, learning rate 2e-5, 1 epoch (4,304 steps, batch 1), sequence length up to 3,072.
• Loss: for chat rows, only the final assistant reply; for tool rows, the whole sequence.
• Held-out loss (80 rows): base 3.285 -> with adapter 2.093 -> released merged model 2.098.
• Release build: the 16 LoRA layers are merged at 6-bit and every other layer stays at the original 4-bit. A plain 4-bit merge lost much of the tuning (held-out loss 2.334), 6-bit matched the adapter.

Training data and credits

4,304 training rows: 2,729 single-turn and tool rows from the sources below, 900 multi-turn chats cut at random lengths, and 675 "terse-history" chats (earlier assistant turns shortened, final reply untouched). Rows were filtered for sexual content, bracketed/OOC markup and over-long text.

Part	Source	Written by	Licence / terms
Single-scene roleplay (ours)	Prism Roleplay 2 core set	NVIDIA Nemotron 3 Super, Ultra and Nano	Ours; the teacher models' own terms apply
Multi-turn chats (900 + 675 rows)	Our own scenarios	MiniMax M3.1 Flash (nearly all) and StepFun Step 5 Preview	Built with MiniMax M3.1-Flash. See the MiniMax Community License
Shortened history turns	Rewrites of the chats above	MiniMax M3.1 Flash, Step 5 Preview	as above
Memory replies	Situations from jihyoung/ConversationChronicles	MiniMax M3.1 Flash (mostly), GLM 5.3, Step 5 Preview	CC BY 4.0, Jang et al., Conversation Chronicles
Emotional-state annotations	Our scenes	Step 5 Preview, GLM 5.3	Ours
Inner thoughts	Original drafts by Kimi K3, GLM 5.3, DeepSeek V4.1 Flash; every thought then rewritten to ~35-70 words	Meta Muse Glimmer 30B (Apache-2.0) and Step 5 Preview	Ours
Public-domain character chats	practical-dreamer/RPGPT_PublicDomain-ShareGPT	LLM-written by the dataset author, about public-domain characters	MIT tag
Character-card chats (about 6%, ~250 rows)	SFW rows of beyoru/Aesir-Character-CoT-roleplay	See below	See below

The part we are least comfortable with: the character-card rows

About 250 of the 4,304 training rows (about 6%) come from beyoru/Aesir-Character-CoT-roleplay. Its card says the original prompts and replies are Claude Opus traces from Gryphe's Aesir character-card data (packaged by PJMixers-Dev, a copy that states no licence), with the reasoning written by DeepSeek V4 Pro and reviewed by Claude Sonnet. Its own licence note reads "Inherits from source dataset. Verify before commercial use", although it carries an apache-2.0 tag. We could not establish the rights to the upstream character cards or to the Claude-written text. We only used rows that passed our SFW filter. If that matters to you, treat this model accordingly. A future version may replace those rows with data from permissive sources.

Other notes

• Every teacher-model call for the synthetic parts went through NVIDIA's hosted API (build.nvidia.com).
• Skipped on purpose: the scraped roleplay-forum corpus used by 1.5 (unclear rights, explicit content) and the Persona-Chat sets.
• Evaluation judges: nvidia/nemotron-3-ultra-550b-a55b and meta/muse-glimmer-30b.

License

Apache 2.0, inherited from the Qwen3 base model. The training data carries the separate terms listed above.