llav — the open decision model (v0.4 text · v0.5-preview image & audio)

llav turns a fuzzy if in software into a typed call: give it a state (ticket, passage, image, audio clip) and a yes/no question, get back a calibrated probability. It speaks the System One protocol (drop-in for Jev-style clients) and is served at https://llav.app (free tier: 10,000 decisions / day / key).

This card describes the models and publishes the benchmark numbers. Weights are not in this repository yet: the text judge weights (llav-0.4-round0, llav-0.4-rounds) and the image/audio adapters open with the v0.5 release. Code, training and evaluation scripts are open now at https://github.com/llachat/llav.

Models

Model	Base	What it judges
llav-0.4 text judge	Qwen3-Reranker-4B (LoRA, weight-averaged)	evidence / relevance / yes-no questions about text
llav image judge (0.5-preview)	Qwen3-VL-Reranker-8B (LoRA)	yes/no questions about an image (one or two photos)
llav audio judge (0.5-preview)	Qwen2.5-Omni-3B thinker (LoRA, bf16)	emotion / intent / sound questions about a clip

Numbers (measured by us, same candidates, same day; details and caveats at https://llav.app/benchmarks)

Benchmark	llav	Jev 1.14	Gemini Decision 1
Text judge, MuSiQue held-out q50–200 (AUC)	0.970 single 4B · 0.977 five-model ensemble	0.969	0.966
Calibration, round 0 (Brier / ECE, lower is better)	0.0095 / 0.004	0.0113 / 0.015	0.0118 / 0.031
Image judge, blind set (DocVQA / ChartQA / MMStar, 3,120 items, AUC)	0.830	0.984	0.962
Audio judge, blind set (CREMA-D emotion, 1,200 clips, AUC)	0.697	0.455	0.714

Image and audio numbers are blind sets (tasks llav never saw in training). On sets drawn from llav's own training sources it scores 0.950 (image) and 0.952 (audio).

Use

curl https://api.llav.app/v1/systemone -H "Authorization: Bearer $LLAV_KEY" -H "Content-Type: application/json" \
  -d '{"state":"Customer: the parcel never arrived and I want my money back.","questions":{"refund":{"type":"noul","instructions":"Is the customer asking for a refund?"}}}'

Self-hosting (MLX on Apple silicon, or one NVIDIA GPU) is documented in the GitHub repo; weights follow with v0.5.

License

Apache-2.0 for code and the released weights. Benchmarks reuse public datasets under their own licenses.