VertexAGI/prism-caption-2-5-micro

Prism Caption 2.5 Micro

Prism Caption 2.5 Micro is a chat-titling model — given the first user message of a conversation, it generates a short, specific, correctly-formatted title (4-6 words, title case, naming the actual subject). Fine-tuned via LoRA on LiquidAI's LFM2-350M, the same dataset and recipe as Prism Caption 2 Micro, just swapped down to a smaller base to see how far the task tolerates shrinking the model.

Part of the Prism family of small, single-purpose models.

Why LFM2-350M

Chat titling is a narrow, low-complexity task — the question this model answers is whether a 700M-parameter base is actually necessary for it, or whether a purpose-built LoRA fine-tune closes most of the gap on a much smaller one. LFM2-350M (354.5M parameters) is the next tier down in the same LFM2 family used for Caption 2 Micro, so the comparison isolates parameter count rather than architecture:

Model	Params	MMLU	IFEval
LFM2-350M	354M	43.43	65.12
LFM2-700M	742M	49.9	72.23

LFM2-350M trails LFM2-700M on general benchmarks, as expected for a model less than half the size. The question this repo answers is how much of that gap survives task-specific fine-tuning on chat titling specifically — see Evaluation below.

Model Details

Base model	LiquidAI/LFM2-350M
Fine-tuning base checkpoint	mlx-community/LFM2-350M-4bit
Architecture	LFM2 -- hybrid short-convolution / attention (16 blocks total, 6 full-attention, 10 short-conv)
Fine-tuning method	LoRA (rank 8, scale 20.0, 16 layers)
Fine-tuning framework	MLX / mlx-lm, on Apple Silicon
License	LFM Open License v1.0

Training Data

Fine-tuned on the same chat-titling dataset as Prism Caption 2 Micro, extended from 10,000 to 13,000 examples (11,700 train / 1,300 validation) -- 1,207 unique topics, 7,984 unique generated titles. Distilled across a mix of four teacher models, cycled to spread load across free-tier rate limits:

Teacher	Examples	Share
openai/gpt-oss-20b (NIM)	8,339	64.1%
nvidia/nemotron-3.5-lightning-30b-a3b (NIM)	3,201	24.6%
poolside/laguna-s-2.1:free (OpenRouter)	1,027	7.9%
openai/gpt-oss-120b (NIM)	433	3.3%

The 3,000-example extension came entirely from openai/gpt-oss-20b, despite the generator cycling between it and openai/gpt-oss-120b every 120 seconds -- gpt-oss-120b's calls need far more token headroom to clear its own internal reasoning before answering (400 vs. 24 max tokens), so fewer of its calls land inside the retry budget per cycle window.

Training Procedure

• Method: LoRA fine-tuning (rank 8, scale 20.0, dropout 0.0, 16 layers -- full model depth)
• Optimizer: Adam, learning rate 1e-5
• Sequence length: 256 tokens
• Training steps: 6,000 iterations, validation every 200 steps
• Best validation loss: 0.200 at iteration 4,400 (down from 7.146 at initialization) -- used as the final checkpoint rather than iteration 6,000 (0.209), which had drifted slightly above the best point
• Throughput: ~2.06 it/sec, ~900 tokens/sec, peak memory ~1.0GB -- notably lighter and faster than Caption 2 Micro's already-light 700M run

Evaluation

Compared against base LFM2-350M and Prism Caption 2 Micro (the 700M sibling fine-tuned on the same recipe, before this dataset's 3,000-example extension) on the same 275-topic held-out set used for Caption 2 Micro's own eval -- zero overlap against the training bank, identical inputs across all three systems:

System	Issues	Relevant	3-6 words	Avg length
Base LFM2-350M	236/275	233/275	87/275	10.5w
Prism Caption 2.5 Micro	0/275	273/275	241/275	5.3w
Prism Caption 2 Micro (700M)	0/275	271/275	262/275	5.0w

"Issues" = formatting problems (too long/short/terse, leaked preamble, trailing punctuation, multiline). "Relevant" = title shares a non-stopword with the source topic.

The fine-tune has the same effect on the 350M base as it did on the 700M one: zero formatting issues, versus 236/275 for the untuned base. More notably, Caption 2.5 Micro is statistically on par with the 700M Caption 2 Micro on this task -- marginally higher relevance (273 vs. 271), marginally looser adherence to the 3-6 word spec (241 vs. 262, still averaging a reasonable 5.3 words) -- at less than half the parameter count. For a narrow, single-purpose task like chat titling, the larger base's extra capacity mostly goes unused once both are fine-tuned on the same data.

Formats available

This repo includes both:

Format	File	Notes
MLX (4-bit)	model.safetensors + config	For Apple Silicon via mlx-lm
GGUF (Q4_K_M)	prism_caption_2_5_micro_Q4_K_M.gguf	For llama.cpp and compatible runtimes (LM Studio, Ollama, etc.)

Usage -- MLX

from mlx_lm import load, generate

model, tokenizer = load("VertexAGI/prism-caption-2-5-micro")

messages = [{"role": "system", "content": (
    "You name chat conversations. Given the user's first message, reply with ONLY a short, "
    "specific chat title (4-6 words, title case, no quotes, no punctuation at the end, no "
    "preamble). The title MUST name the main subject of the message -- do not over-abbreviate "
    "into something vague. Nothing else -- just the title."
)}, {"role": "user", "content": "Any advice on how to fix a leaking kitchen faucet?"}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

response = generate(model, tokenizer, prompt=text, max_tokens=24)
print(response)

Usage -- GGUF (llama.cpp)

llama-cli -hf VertexAGI/prism-caption-2-5-micro -m prism_caption_2_5_micro_Q4_K_M.gguf \
  -sys "You name chat conversations. Given the user's first message, reply with ONLY a short, specific chat title (4-6 words, title case, no quotes, no punctuation at the end, no preamble). The title MUST name the main subject of the message -- do not over-abbreviate into something vague. Nothing else -- just the title." \
  -p "Any advice on how to fix a leaking kitchen faucet?"

Limitations

Trained on a synthetic dataset distilled from a shifting mix of teacher models rather than a single consistent one; some stylistic inconsistency between examples from different teachers may be present. Titling behavior is only validated on English, conversational, everyday-topic inputs -- highly technical or non-English inputs are untested. At 354M parameters, this is the smallest base in the Prism Caption line; the evaluation above uses the same 275-topic set as Caption 2 Micro rather than a larger held-out sample.

License

LFM Open License v1.0, inherited from the LFM2-350M base model. Free for research/non-commercial use and for commercial use under $10M annual revenue.