bukva-0.8b

Anything in, one letter out. Bukva (Russian буква, "letter") is a 0.8B decision model: you give it a situation (text and/or a screenshot), a question and a list of options, and it picks one — in a single forward pass, by reading the next-token logits of the option letters A, B, C, …. No generation, no output parsing, no retries. About 36 ms per decision on an RTX 3060 (≈150 ms with a 1280×800 screenshot).

It is a set of LoRA adapters on Qwen3.5-0.8B-Base (the base model is frozen; the vision tower is untouched). Four decision types:

type	what you get
choice	one of up to 26 options + calibrated probabilities
noul	yes / no with a probability
score	an ordinal option (e.g. 1–5 stars) + the expected value
multi	every option that applies

What it is good at: routing intents, moderation (EN/RU toxicity, spam), topic / sentiment / rating, "does this passage answer the question", reading UI screenshots, and — in the agent builds — driving an app or a website step by step ("what do I tap next?", "is the goal reached?").

Which build?

All builds share the same code; pick one with build=.

build	use it for
v8-classifier	the best pure classifier (text) — classic tasks 85.1
mix-v8v7	universal: classifier + app agent (phone test 39/40)
mix-v5v6	classifier + web-browsing agent (browser 5/8 text)
v7-agent-app / v6-agent-web	agent specialists (raw, before merging)
v5-classifier, v4-vision	earlier stages, kept for comparison

mix-* builds are exact linear merges of LoRA deltas (no extra training): specialise, then merge — the agent skill stays and the classifier forgets much less.

Benchmarks

Text version: BENCHMARKS.md. Every test is held out — no test item was trained on:

• Phone agent — an unseen synthetic test app (switches, files, delete-with-confirm, profile), 40 multi-step episodes in EN and RU, screenshot only. Success = the goal state is reached and the model itself says it is done.
• Browser — a real Chrome on live Wikipedia (EN/RU) and python.org, 8 navigation tasks; the page is given as text or as a screenshot.
• Classic tasks — 100 items each: intent routing (clinc151, 16 options), SMS spam, Russian news topics, toxicity EN and RU (textdetox), review stars (Yelp), QNLI.
• Probes — small hand-written question sets (47 EN, 21 RU, 11 game situations) that are in no dataset. They are small: one question is 2–5 points.
• UI screenshots — 484 questions about synthetic UI screenshots (toggles, sliders, dialogs, tabs…).
• Vision tests — mean over 11 public sets: AI2D, BLINK, CountBench, CV-Bench, GameQA, MMBench EN/RU, MMStar, ruCLEVR, RealWorldQA, ScreenSpot (multiple-choice form, 400 items each).

Quick start

pip install torch transformers safetensors huggingface_hub pillow
# Qwen3.5 linear-attention layers run much faster with flash-linear-attention (optional):
pip install flash-linear-attention

from huggingface_hub import hf_hub_download
import importlib.util, sys
spec = importlib.util.spec_from_file_location("bukva", hf_hub_download("annelo/bukva-0.8b", "bukva.py"))
bukva = importlib.util.module_from_spec(spec); spec.loader.exec_module(bukva)

m = bukva.Bukva.from_pretrained("annelo/bukva-0.8b", build="v8-classifier")   # or "mix-v8v7"

m.decide("hey, i can't log into my account since this morning",
         "Which team should handle this?", ["billing", "tech support", "sales", "spam"])
# {'answer': 'tech support', 'index': 1, 'probs': [...], 'confidence': ...}

m.decide("Это просто ужас, а не сервис!", "Есть ли в тексте оскорбление?", ["да", "нет"], type="noul")
m.decide("Great coffee but we waited 40 minutes.", "How many stars?", ["1", "2", "3", "4", "5"], type="score")
# -> also "expected": ~3.1
m.decide("", "Is Wi-Fi turned on?", ["yes", "no"], image="settings.png", type="noul")

The prompt the model was trained on (the code builds it for you):

<state text>

Question: <question>
Options:
A. <option 1>
B. <option 2>
Answer:

The answer is read from the logits of A, B, … right after Answer:. Probabilities are temperature- calibrated (T is stored with each build). A useful rule: act on its own when max(probs) ≥ 0.8 (on the classic tasks it is right 93–100% of the time there), otherwise escalate.

More than 26 options: split them into groups of ≤ 20, pick a winner per group, then pick among winners.

Agent loop (app or web)

m = bukva.Bukva.from_pretrained("annelo/bukva-0.8b", build="mix-v8v7")
log = []
for step in range(12):
    shot = take_screenshot()                          # your code
    state = "Screenshot of the app.\nActions taken so far:\n" + "\n".join(log)
    done = m.decide(state, "Task: turn off Bluetooth.\nIs the task finished?", ["yes", "no"], image=shot, type="noul")
    if done["answer"] == "yes":
        break
    buttons = list_clickable_labels()                 # your code: what can be tapped right now
    a = m.decide(state, "Task: turn off Bluetooth. What is the next click?", buttons, image=shot)
    result = tap(a["answer"])                         # your code
    log.append("step %d: %s => %s" % (step + 1, a["answer"], result))

How it was trained

• Base: Qwen3.5-0.8B-Base, frozen. LoRA r=16 (α=32) on all attention / linear-attention / MLP projections of the language model (150 matrices, 10.2M parameters). The vision tower is frozen; image features were pre-computed once and cached.
• Readout: the tied embedding rows of the letters — no new head. Options are reshuffled every epoch so that position carries no signal. Temperature fitted on validation afterwards.
• Data: ~140 task types, EN and RU — NLI, QA, sentiment, topics, intents (MASSIVE, banking77, HWU64, SNIPS, ATIS, Bitext), spam (email and SMS), toxicity, ratings, reasoning, games (chess, tic-tac-toe), synthetic UI / game / chart screenshots, A-OKVQA, ScienceQA, GQA-ru, ScreenQA, ChartQA, and synthetic multi-step app and website trajectories (shortest-path expert, pop-ups, dead buttons, "already done" states). Every benchmark above was de-duplicated out of the training data.
• Stages: v4 vision → v5 classic-task gaps → v6/v7 agents → v8 classifier; each stage ~20–40 minutes on one RTX 5090. Before every run an independent reviewer model audited the data generator from scratch (labels, leaks, shortcuts).

Limitations and honest notes

• It is not a chat model and does not generate text. It chooses among the options you give it.
• EN and RU only were trained and tested.
• The app-agent builds recognise "done" much better than the web builds. On real websites the app agent tends to fall back to "go to the main page" when lost (browser 3/8 vs 5/8).
• Small test sets: the browser test has 8 tasks and the probes 11–47 questions, so treat 1–2 point differences as noise.
• The v5–v7 replay data contained, word for word, the question wording of two benchmarks: clinc151 routing and Yelp stars. Test items were never included. v8 rewords those questions.
• SMS spam test contamination in v8 / mix-v8v7 (found by our own audit, fixed for later builds): one of the three public spam corpora used for v8 contains UCI SMS messages with punctuation deleted, which slipped past the de-duplication. 3 of the 300 held-out SMS test rows (1 of the 100 scored items) were in v8's training data. Re-scored without that item, v8's spam score moves from 89.7 to 89.5, so the effect is negligible. Builds after v8 use a stricter letters-and-digits filter.
• The phone test app uses pill-style switches, a standard widget that also appears among the v7 training styles. Its labels, wording and history format differ from the training ones.

Files

bukva.py                          loader + decide() (no PEFT needed; merges the LoRA at load time)
builds/<build>/adapter.safetensors
builds/<build>/bukva.json         rank, alpha, temperature, multi-label calibration
BENCHMARKS.md, bench.png, header.png

License

See LICENSE.md. In short: bukva.py is Apache-2.0. The adapters are for research and non-commercial use, because some training datasets are non-commercial (e.g. ScienceQA, CC BY-NC-SA 4.0). The base model is Apache-2.0.