Karti-Small-VL-4B

Status: v0 baseline pinned and measured. Training not yet started. This page is published from the scaffolding stage on purpose, and it will be updated as runs happen — including the ones that fail.

Karti-Small-VL-4B is Lumbridge's small vision + tool-calling model: a 4B model meant to run locally and do the two things a household agent actually needs together — look at something, and then call a tool about it. Read a camera frame and check whether the job that should have cleared it ran. Look at a receipt and file it. See a screen and act on it.

It is a sibling to Karti-Small-RSI-3B, which is text-only, and it starts from a different base for a specific measured reason.

Why this base

Base: Qwen/Qwen3.5-4B · 4.66B · Apache-2.0 · ungated · pinned at 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a

Selection ran in two steps: a gate that removes most candidates, then a decision among the ones left. An earlier revision of this page stopped at the gate and named Qwen/Qwen3-VL-4B-Instruct. The gate was right; it just does not finish the argument.

The gate — can the base's own chat template render a tool call?

The question asked of every candidate, before any training was planned:

Can this model's own chat template render a tool call?

That question came from an expensive lesson on the 3B program: its base advertises tool support but its template renders message.content only, so an assistant turn carrying structured tool_calls renders as an empty turn. The model could not see the call it had just made, and evaluation was measuring the template rather than the model.

So we checked, rather than assumed. Most small vision-language models fail:

candidate	advertises tools	renders tool_calls
Qwen3.5-4B	yes	✅ yes
Qwen3-VL 2B / 4B / 8B	yes	✅ yes
MiniCPM-V-4.5	yes	✅ yes
Ovis2-4B	yes	✅ yes
Qwen2.5-VL 3B / 7B	no	❌ no
Phi-4-multimodal	yes	❌ no
InternVL3 / InternVL3.5	no	❌ no
SmolVLM2 · GLM-4.1V · Kimi-VL	no	❌ no

Both finalists render image tokens, tool definitions, the assistant's own <tool_call>, and a wrapped <tool_response> natively. That means training and serving can share one apply_chat_template call against the same pinned revision, so train/serve token parity holds by construction instead of being verified after the fact. An entire class of bug stops being reachable.

The gate is necessary and not sufficient. It left more than one base standing.

The decision — which passing base do we already know how to serve?

Qwen3.5-4B is Qwen3_5ForConditionalGeneration: the same architecture as the 27B that is already our resident local vision model. Qwen3-VL-4B-Instruct is Qwen3VLForConditionalGeneration, a second architecture that would have to be carried in parallel — its own serving path, its own quantization work, its own set of runtime surprises, for one model.

The two are closer than the names suggest, and further apart where it counts:

	Qwen3.5-4B	Qwen3-VL-4B-Instruct
architecture	Qwen3_5ForConditionalGeneration	Qwen3VLForConditionalGeneration
shares arch with our 27B	yes	no
parameters	4,659,865,088	4,437,815,808
vision tower	depth 24 · hidden 1024 · out 2560 · patch 16	identical
layers	32	36
attention heads	16	32
KV heads	4	8
FFN	9,216	9,728
vocabulary	248,320	151,936
context	262,144	262,144

The vision towers are byte-for-byte the same configuration, so this is not a trade of sight for convenience — the thing that sees is the same thing. What differs is the language model wrapped around it: half the KV heads, which is half the KV cache at a given context length, on a model whose whole purpose is to run locally and cheaply alongside other resident services.

The larger vocabulary is a real cost in embedding parameters and the reason this base is the larger of the two despite having fewer layers. We are taking it knowingly.

What the swap costs

Two differences we carry deliberately, recorded here so they are not rediscovered later as bugs:

1. The call format is not JSON. Qwen3.5 renders a call as <function=name><parameter=key>value</parameter></function> inside <tool_call> tags, where Qwen3-VL renders {"name": …, "arguments": {…}}. Parity still holds by construction — we pass structured tool_calls and let the template do the rendering, never hand-formatting either shape. But the planned control run cannot claim the 3B corpus is reaching the model unchanged: the same rows now serialize differently, and the control measures base plus serialization together, not the base alone.

2. Thinking is on by default. The template opens a <think> block and the generation prompt ends inside one. On a sibling deployment this silently returned empty content under a small max_tokens budget — the reasoning block consumed the whole allowance and the response came back finished-on-length with nothing in it. Every run and every serving config here pins enable_thinking: false, and an empty-content check under a tight token budget is a required canary before any result from this model is believed.

Checkpoint identity

Runs and releases are different objects here, and are named differently.

Every training run:

Karti-Small-VL-4B-2026.08.29-r007-a1b2c3d
└──── family ────┘ └── date ──┘ └run┘ └digest┘

• date — the calendar day the run started. Recorded, never derived from a week number, with no weekday requirement.
• rNNN — a globally monotonic run counter that never resets. Five runs in one day are r007–r011. The scheme assumes no cadence.
• digest — 7 hex of the run manifest digest, covering the corpus, the base revision and the hyperparameters. Two runs differing in any of those cannot collide; identical inputs reproduce the same id.

A release is a checkpoint the promotion gate accepted, and only then does it get a short name — Karti-Small-VL-4B-v1. Versions are immutable and are never reassigned, so from v1 onward a version number means something passed a gate rather than merely finished training.

v0 is the exception that makes the rule usable: it is the reference baseline — the pinned base, measured on our own hardware, before any training. It is what every later version has to beat, so it is numbered rather than left floating.

This replaces a week-anchored scheme that could not express iterating more than once a week without inventing a date.

Versions

Karti-Small-VL-4B-v0 — reference baseline

The starting line, pinned and measured so later versions have something exact to be compared against.

weights	Qwen/Qwen3.5-4B @ 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
precision	BF16
parameters	4,659,865,088
training applied	none — this is the baseline
serving alias	karti-vl · karti-vl-4b-base

Measured on a DGX Spark (GB10, 121.7 GB unified), vLLM 0.27.1, BF16, max_model_len 32768, KV cache FP8, temperature 0, single stream:

check	result
generation, 128 tokens	18.24 tok/s
generation, 512 tokens	17.74 tok/s
vision — shapes and colours off a synthetic image	correct
tool call via qwen3_xml parser	parsed — cron(job="driveway-clear")
arithmetic canary 19 × 23	437
empty-content canary at max_tokens: 80	non-empty, finish_reason: stop

Vision baseline

A curated vision slice, mined from public benchmarks (MMStar, RealWorldQA, V*Bench) and frozen with per-item provenance:

vision_probe accuracy	0.727 ± 0.030 (n = 220)
contamination canary	clean

Selection is model-independent — benchmark category label plus a seeded balanced sample, never observed difficulty. Curating on what a model gets wrong produces a suite whose score can only go up, which measures the curation and calls it progress. The slice scores into its own lane and is never averaged into anything else, because selected items cannot be compared to a leaderboard.

This is the untrained base. It is what v1 has to beat; it is not a claim that the model is good at vision, only that we can now tell whether training moved it.

A private task scored on our own cameras also exists. It refuses to run against any target that is not a local service, the frames never leave our hardware, and its score stays private.

The throughput number is the interesting one: at BF16 this 4B runs at approximately the same single-stream rate as the 27B NVFP4 brain on the same box. That is a memory-bandwidth result, not a capability one, and it is the argument for quantizing the deployment lane rather than shipping BF16.

The weights are mirrored here and byte-identical to the pinned Qwen revision above — same two shards, same SHA-256 — so the repository is self-contained and this table is its measurement. See Weights to load or serve it.

What it is being trained to do

The tool surface it inherits from the 3B program, now with images in the context:

tool	with vision
phone	read device-broker state
cron	inspect what a scheduled job actually left behind
browser	take a page snapshot and read it
route	send a request to the right tier, or to an explicit stop
tera	bounded workflow actions
reply	deliver the answer to the requesting channel

Every trajectory ends by delivering through reply. Nothing in the callable set executes a confirmed action — consequential steps are proposals, and the confirmation boundary sits outside the model.

Plan

1. Control run — the existing 3B text corpus on this base, with the serialization caveat above stated in the result rather than hidden by it.
2. Vision rows — images paired with tool-calling targets. Vision tower frozen, LoRA on the language model.
3. Topology — decide on evidence whether this model subsumes the text-only 3B or is routed to by it.

The measurement lanes are separate on purpose: canonical BF16 for quality, and deployment quantization measured on its own afterward, never mixed into a quality score.

Weights

v0 is published here and runnable. It is the BF16 base mirrored at the pinned revision, so the repository is self-contained rather than a set of instructions pointing somewhere else.

from transformers import AutoProcessor, AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained(
    "KartiOS/Karti-Small-VL-4B", revision="v0", dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("KartiOS/Karti-Small-VL-4B", revision="v0")

Serving it, including vision and the tool-call parser this template needs:

vllm serve KartiOS/Karti-Small-VL-4B --revision v0 \
  --max-model-len 32768 --kv-cache-dtype fp8 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --default-chat-template-kwargs '{"enable_thinking": false}'

Pin revision="v0". Each version gets its own immutable tag, and main moves — so an unpinned load will silently change under you when v1 lands.

Trained checkpoints are a different matter. Publishing one is a separate decision with its own privacy review: a vision model's training data is a different privacy object from synthetic text, and the scoping of what may enter the corpus happens before a checkpoint absorbs it, not after.

Follow the program

lumbridgecorp.com/models