jev-vlm-4b — a calibrated visual decision model (best calibration)

A Jev-style System One model for images: it answers typed questions (choice / noul / score) about a state that may contain document pages, receipts, charts, or annotated screenshots — in a single prefill per state, no token generation — returning calibrated probability distributions.

Built on Kev's pointer-head architecture, extended to vision on Qwen/Qwen3.5-4B-Base. This is the best-calibrated variant (transformers, trust_remote_code). For a portable single GGUF (llama.cpp / LM Studio) that carries the same calibration via distillation, see the sibling sidman007/jev-vlm-4b-lite-gguf.

Open weights under Apache-2.0 — free for commercial and non-commercial use. (Training code and data are not part of this release.)

Quick start (transformers, trust_remote_code)

import torch
from transformers import AutoModel, AutoProcessor

model = AutoModel.from_pretrained("sidman007/jev-vlm-4b", trust_remote_code=True,
                                  dtype=torch.bfloat16, device_map="cuda")
processor = AutoProcessor.from_pretrained("sidman007/jev-vlm-4b")

answers = model.decide(
    processor,
    images=["invoice.png"],                       # paths or PIL.Image; omit for text-only states
    state="See the attached invoice.",
    questions={
        "total":   {"type": "choice", "instructions": "Which value is the total?",
                    "criteria": {"a": "240.00", "b": "12.00", "none": "Not shown"}},
        "overdue": {"type": "noul",   "instructions": "Is this invoice past due as of 2026-01-01?"},
        "quality": {"type": "score",  "instructions": "How complete is this invoice?",
                    "criteria": ["missing key fields", "partial", "complete"]},
    })
# answers["total"]  -> {"choice": "a", "confidence": 0.98, "probabilities": {...}}
# answers["overdue"]-> {"probability": 0.07}
# answers["quality"]-> {"score": 1.83, "probabilities": {...}}

Every question is answered in one forward pass over the state — 20 questions about one document cost one image prefill plus 20 short text rows, not 20 generations. A low confidence / flat distribution is the model abstaining, not guessing.

Alternative loading (kev-style): base checkpoint + adapter/ (PEFT LoRA) + head.pt, driven by the repo's jevvlm.pointer.vision.VisionDecisionModel.

Benchmarks

benchmark	metric	score
In-domain locked test (n=4,987, held-out; calibrated)	acc / ECE / cov@5%-err	0.9519 / 0.013 / 1.00
confident-error rate		0.0036
AI2D diagram-MCQA leaderboard (n=3,000)*	accuracy	0.835

Probabilities are post-hoc temperature-calibrated (T≈0.54) on a dev split; out-of-distribution inputs produce coverage collapse (abstention) rather than confident errors. *AI2D measured on the distilled GGUF sibling (constrained letter readout); the pointer model is the better-calibrated teacher it was distilled from. It's a comparable data point, not a like-for-like leaderboard submission — the model is document/chart-specialized.