cerebellum-08b-scrollvocab-sft (step 424)

QDoRA adapter for a small Android UI action model. It is the execution layer of a two-tier design: a frontier model plans, this model does the actual touching. That split is why it is 0.8B — the execution layer runs constantly, so it is sized to be cheap to run rather than to be clever.

Where it runs today: on a computer — a dev machine or CI server — driving an emulator or an attached device over adb. It is not an edge model yet. Running it on the phone itself is the eventual goal and the reason for the size budget, but the latency and packaging work to get there has not been done.

Trained on a single RTX 3060 12GB.

What it does

Given a screenshot with Set-of-Mark labels and an accessibility tree, it emits a single action per step in a compact grammar. Labels are single-token symbols, so an action is a couple of tokens rather than a sentence.

• tap: T <label>
• scroll: S <dir> <label> (decomposed grammar: direction and target are separate tokens, which is what this run introduced)
• plus type/back/home/wait and terminal finish/infeasible signals

Prompts are full-episode and append-only: the episode's history accumulates rather than sliding, so the model sees what it already did.

Checkpoint

This is SFT step 424 of run som_08b_r128_scrollvocab_20260831, taken before the visual-element-pointer experiments branched off it. It is kept as a reset point — the checkpoint later work returns to when a branch does not pan out.

• rank 128, alpha 256, DoRA, nf4 base
• to merge: load nf4, dequantize, then merge_and_unload (merging against the quantized weights silently degrades grounding)

Evaluation, and how to read it

Held-out AndroidControl shards (0 and 1), exact-match:

metric	value
tap, exact match	73.5%
tap, after human review of misses	81.6% floor

Tap is the metric that matters here; rare-class gains that cost tap are not gains, so it is always reported separately.

The gap between those two rows is the important part. Exact match undercounts: of 36 confident "misses" reviewed by hand, only 9 were genuinely wrong. The rest were label-space artifacts — duplicate targets, a parent and child that both cover the tap, or a popup whose row was labelled with the widget underneath it. So 81.6% is a floor on real tap accuracy, not a tuned number.

A separate 51% figure for this line of work refers to a held-out 100-task AndroidWorld run and measures whole-task completion, which is a much harder thing than per-step accuracy. Do not compare the two directly.

Limitations

• The grammar can only tap element centres. Opaque custom-drawn views (a calendar grid or a drawing canvas exposed as one big View) hold many targets in one node, and the model will tap confidently and wrongly. Affects ~0.3% of held-out taps. Intended fix is to escalate those screens to the frontier model rather than to guess.
• Rows that the accessibility tree marks non-actionable cannot be targeted at all, regardless of what is visible on screen.
• This is an adapter, not a standalone model; it needs the base and the same prompt format it was trained on.