daintreehq/canopy-reader-68m

Superseded (2026-10-26). Use daintreehq/canopy-reader-17m: it was retrained for a summary-only card with line scores, and in blind judging on 153 real screens its card beats the previous 17M's (49 better, 16 worse); a 68M trained the same way was no better than the new 17M (14 better, 23 worse), at four times the size. This repo stays as it was for anything that already uses it.

Canopy reader 68M

A small encoder that reads one AI coding agent's terminal screen (Claude Code, Codex, Gemini CLI, Copilot, Cursor, Aider and a dozen more) and tells you, in one forward pass, what state the agent is in, whether it needs you, and which lines on the screen to show: a headline, up to three summary lines, the question and its options, the action it wants approval for, and the request it is working on. Every piece of text on the card is a line copied from the screen, never generated, so the card can't invent anything.

It is the model behind Canopy in Daintree, which watches every agent you have running and surfaces the ones that need you. It runs on the user's own machine, on the CPU: 70 MB, about 130 ms a screen on one thread.

Best: the best line picks on real screens (headline, summary and options), for about five times the work of Light; category is as good as Light's.

Pick a size

Both share the same heads and the same input, so they are drop-in replacements for each other. INT8, CPU, one thread, one screen at a time on an Apple M4 Max; real screens are never trained on.

Model	Download	Per screen	Real gold category	Real agent screens	Headline line	Card F1
canopy-reader-17m	17 MB	25 ms	58/60	89/93	75.8%	0.65
canopy-reader-68m	70 MB	130 ms	57/60	89/93	81.7%	0.71	this model

On x86 (an Intel Xeon Platinum 8358 cloud VM at 2.6 GHz, INT8): 17M 71 ms on one thread, 42 ms on two · 68M 463 ms on one thread, 248 ms on two. A current laptop CPU clocks higher and should land between that and the M4 Max figure. Two threads roughly halve the time.

Headline line and card F1 are scored on the 153 real screens against careful line labels made blind, independently of the training labels. Two careful labellers agree with each other at about 84% on the headline and 0.78 card F1, so that is roughly the ceiling.

Quick start (Python)

pip install onnxruntime tokenizers numpy huggingface_hub

import sys
from huggingface_hub import snapshot_download

folder = snapshot_download("daintreehq/canopy-reader-68m")
sys.path.insert(0, folder)
from canopy_reader import CanopyReader   # canopy_reader.py, shipped in this repo

reader = CanopyReader(folder)            # model.int8.onnx, one CPU thread
lines = [
    "› Add a --verbose flag to the CLI and log each request",
    "• Added --verbose to src/cli.py and request logging in src/http.py.",
    "• All 14 tests pass; the two files are uncommitted.",
    "  Worked for 1m 12s • 10:30",
    "› Ask Codex to do anything",
]
state = {"agent_program": "codex"}
reading = reader.read(state, lines)
print(reader.card(reading, lines))

This model's output for that screen:

{
 "category": "finished",
 "needs_attention": true,
 "score": 55,
 "headline": "Added --verbose to src/cli.py and request logging in src/http.py.",
 "summary": [
  "All 14 tests pass; the two files are uncommitted."
 ],
 "question": null,
 "options": [],
 "action": null,
 "task": "Add a --verbose flag to the CLI and log each request",
 "tests": "passing",
 "changes": "uncommitted",
 "progress": 100
}

reader.read() returns every head (line indices and classes); reader.card() turns them into text: bullets, tree and box glyphs, spinners and timers stripped, and a sentence the agent wrapped over two lines joined back together. A Node/TypeScript port with the same behaviour is in node/reader.ts (onnxruntime-node + @huggingface/transformers for the tokenizer); both match the reference implementation on every field of 300 test screens.

Input

• lines: the visible screen (plus a little scrollback if you have it), one string per line, newest last. Undo terminal wrapping of a logical line, strip the input box and footer chrome if you can, redact secrets. Blank lines are fine (they are skipped). The model was trained on screens prepared by Daintree's prepareScreen (with some lines dropped and wraps left unjoined, so it copes with rough screens), and prepared screens read best.
• state: {"agent_program": "<cli>"} and, when you have them, terminal_title and daintree_observed: {"agent_state": "working|waiting|idle|completed", "waiting_reason": ..., "seconds_in_state": n} (the host's own reading of the terminal's activity; the screen wins when they disagree).

Encoding (done for you by canopy_reader.py): [CLS] <state as compact JSON> [SEP] then [i] <line>\n for each non-blank line, filled from the bottom of the screen up until 1024 tokens, then [SEP]. The state is cut to 192 tokens and each line to 256. Every line head reads the first token of its [i] prefix. 1024 tokens measured as good as 2,048 on real screens.

Outputs

ONNX inputs: input_ids, attention_mask, line_pos, line_mask, q_mask, s_mask (batch-first; line_pos is each line's first-token position, q_mask/s_mask mark the bottom 40 / 60 rows with more than three characters). Outputs, in order:

Output	Shape	Meaning	Decode
category	(B, 7)	approval, question, finished, error, working, running, idle	softmax, argmax
needs_attention	(B,)	does this screen need the user	gate below
blocked_on_user	(B,)	is the agent paused on the user	sigmoid
notable	(B,)	is there something worth reading (including on a busy screen)	sigmoid
tests	(B, 4)	passing, failing, not_run, unknown	argmax
changes	(B, 5)	uncommitted, committed, pushed, none, unknown	argmax
risk	(B, 3)	none, caution, unknown (for approvals)	argmax
instruction	(B, 5)	the user's standing instruction: none, asked, ongoing, done, irreversible	argmax
progress	(B, 12)	none, then 0-100 in steps of 10	argmax
failure_repeats	(B, 4)	0, 1, 2, 3+	argmax
score	(B, 11)	attention score on Canopy's anchors 98, 94, 92, 86, 75, 70, 65, 55, 40, 20, 5	argmax
question_line	(B, M+1)	the line that asks (approval or question)	argmax over lines + last column "none"
status_line	(B, M+1)	the headline: the one line that best says what is going on now	as above
action_line	(B, M+1)	the command or file an approval is about	as above
task_line	(B, M+1)	the user's request, when it is on screen	as above
summary_lines	(B, M)	per line: is it one of the 0-3 summary lines	sigmoid; rank, take the first if ≥ 0.5, more while ≥ 0.6, at most 3
option_lines	(B, M)	per line: is it an option of the live menu	sigmoid ≥ 0.5, bottom rows only
key_lines	(B, M)	per line: does a reader need it (a recall-heavy extract)	logit > 0

The attention gate is g = max(sigmoid(needs_attention), P(approval) + P(question) + P(finished) + P(error)), mapped so that 0.4 reads as 0.5: g < 0.4 ? 0.5·g/0.4 : 0.5 + 0.5·(g−0.4)/(1−0.4). At ≥ 0.5 the agent needs the user. All of this is in reader.json, which the readers load.

The headline is picked to be the most important line to look at right now: the question or approval being asked, the agent's own report of what it did or found, the step it is on, or the error that stopped it. The agent's own words beat tool output and Bash noise; the newest beats history. Summary lines add the result, the reason, or what failed and why, never chrome, spinners, diffs or menu options.

How well it works

INT8, as shipped. INT8 scored the same as FP32 within noise for every size measured in development (17M, 32M and 68M).

Real screens (never trained on): 60 hand-checked gold screens from real sessions and 93 researched screens across 17 agent CLIs.

Gold screens, category (question and finished counted together, as Canopy shows them)	57/60
Gold screens, attention gate right	57/60
Researched agent screens, category	89/93
Researched agent screens, attention gate right	92/93
Headline line = the blind reference label's	81.7%
Options exactly right (screens with a menu)	87.0%
Task line right	98.7%
Whole card (headline + summary lines) F1	0.714
Tests / changes field right (gold)	52/60, 51/60

Held-out synthetic test (1,142 screens in agent × situation cells never seen in training): category 99.7%, gate missed 3 screens that needed the user, question line 100.0%, headline line 95.6%, options exact 99.6%, first summary line among the labelled ones 78.3%, tests 98.3%, changes 93.4%, score within one anchor 98.9%.

Training

• Base: jhu-clsp/ettin-encoder-68m (MIT), the whole encoder fine-tuned with all heads together: AdamW, encoder lr 6e-5, heads 1e-3, 3 epochs, length-bucketed batches, bf16.
• Data: about 16,000 synthetic terminal screens across 17 agent CLIs and 23 situations (approvals, questions and menus, trust and startup dialogs, finishes, interrupts, errors and limits, working, idle, running servers), generated from researched and live-captured interface fragments with the agent and situation drawn in code, run through Daintree's own screen preparation, and checked mechanically; plus a targeted top-up of the hard situations (trust dialogs, question menus against approvals, interrupts, quiet finishes, errors), and about 1,000 real screens with their private paths and names scrubbed.
• Labels: the screen generator's own labels for state, attention, question and headline lines and the card fields; summary and task lines picked by a second model pass and then checked by a third, which a blind judge preferred and which agrees better with independent gold on real screens. Synthetic screens and labels were produced with OpenAI models.
• Augmentation: screens cut to their bottom few lines, lines dropped, wraps left unjoined and other agents' glyphs swapped in, so it holds up on screens that look different from the training set.
• Split: whole agent × situation cells are held out, never single rows, so the test measures new combinations; the real screens are test only.

Limits

• Text only, English, one screen at a time; it does not remember earlier readings (keep that in the host).
• The card is extractive: if the screen doesn't say it, the card won't either.
• Agent CLIs change their screens often. New layouts mostly still read correctly (the training covers older, current and plausible future versions), but check the category on a new CLI before relying on it.
• Category on real screens is very good; the line picks are good but a matter of taste (see the ceiling above): the model is a little terser than a human labeller.

Licence

MIT, as the base model.