daintreehq/canopy-reader-17m

Canopy reader 17M

One small encoder (17M parameters, 17 MB) that reads an AI coding agent's terminal screen (Claude Code, Codex, Gemini CLI, Copilot, Cursor, Aider and a dozen more) and tells you, in one forward pass and about 21 ms on one CPU thread of an Apple M4 Max (about 0.2 s measured on an Intel Core 5 210H laptop):

• what state the agent is in: approval, question, finished, error, working, running or idle, and whether it needs you;
• the two lines on the screen most worth reading right now, copied from the screen, so the card doesn't invent text (judged faithful to the screen on 152 of 153 real screens);
• for an approval or question, the question line, its menu options and the command it asks about;
• the card's facts: tests, changes, progress, an attention score.

It is the model behind Canopy in Daintree, which watches every agent you run and surfaces the ones that need you. It runs on the user's own machine; no screen text leaves it.

The card

A Canopy row shows the terminal's own title, then at most two lines from the screen:

1. Line one: the best line on the screen (status_line, a one-of-N pick over the lines plus "none"). The agent's own words win: what it found, did, is asking or why it stopped, a clear error, a one-line result, or an interruption or gate it is held at. Raw commands, tool output, logs, chrome, menus and the user's own prompt are scored 0 in training, so they are rarely picked. On a fresh session, or a screen of nothing but commands and output, it picks none.
2. Line two (optional): the next-best line, shown only when its score reads at least 0.7 (about a "2 out of 3"); two lines of one paragraph are shown in screen order, and a sentence the agent wrapped is joined.

Every line on the screen gets a 0-3 score (summary_lines, trained on score/3): most lines are 0.

Quick start (Python)

pip install onnxruntime tokenizers numpy huggingface_hub

import sys
from huggingface_hub import snapshot_download

folder = snapshot_download("daintreehq/canopy-reader-17m")
sys.path.insert(0, folder)
from canopy_reader import CanopyReader   # canopy_reader.py, shipped in this repo

reader = CanopyReader(folder)            # model.int8.onnx, one CPU thread
lines = [
    "› Add a --verbose flag to the CLI and log each request",
    "• Added --verbose to src/cli.py and request logging in src/http.py.",
    "• All 14 tests pass; the two files are uncommitted.",
    "  Worked for 1m 12s • 10:30",
    "› Ask Codex to do anything",
]
reading = reader.read({"agent_program": "codex"}, lines)
print(reader.card(reading, lines))

This model's output for that screen:

{
 "category": "finished",
 "needs_attention": true,
 "score": 55,
 "headline": null,
 "summary": [
  "Added --verbose to src/cli.py and request logging in src/http.py.",
  "All 14 tests pass; the two files are uncommitted."
 ],
 "question": null,
 "options": [],
 "action": null,
 "task": "Add a --verbose flag to the CLI and log each request",
 "tests": "passing",
 "changes": "uncommitted",
 "progress": 100
}

A Node/TypeScript port with the same behaviour is in node/reader.ts (onnxruntime-node + @huggingface/transformers for the tokenizer). Both match the reference on every field and on the card text of 300 test screens.

How well it works

Real screens, never trained on: 60 hand-checked gold screens from real sessions and 93 researched screens from 17 agent CLIs.

Category, gold screens	58/60
Category, researched agent screens	87/93
Needs-you gate, gold / agent screens	57/60, 89/93
Card against the previous version (headline + summary), blind judge on all 153	better 49, worse 16, same 88
Card faithful to the screen (blind judge)	152/153

The judge (gpt-6.1-sol, cards shown in random order) was asked which card tells a developer glancing at it more about what is going on, and whether each card claims anything the screen doesn't show.

A generative alternative was measured and not shipped: a 31M T5 rewriting the reader's picked lines tied this card on usefulness (51 better, 50 worse, 52 same) but was faithful on only 143/153 screens; T5s summarising the raw screen (16M to 77M) were faithful on about 10%.

Input, outputs and decoding

Input is the visible screen as prepared lines (wrapping undone, chrome stripped; Daintree's prepareScreen) and a small state object (agent_program, terminal_title, Daintree's observed agent state). Encoding: [CLS] <state as compact JSON, cut to 192 tokens> [SEP] then [i] <line> for each non-blank line, filled from the bottom of the screen up to 1024 tokens, then [SEP]. canopy_reader.py and node/reader.ts do this; config.json (also reader.json) holds every number the runtime needs.

Outputs (ONNX, batch-first): category (7), needs_attention, blocked_on_user, notable, the classes tests (passing, failing, not_run, unknown), changes, risk, instruction, progress, failure_repeats, score; the picks status_line (card line one), question_line, action_line, task_line (each over the lines plus "none"); and per line summary_lines (the 0-3 score), option_lines, key_lines.

The needs-you gate: g = max(sigmoid(needs_attention), P(approval)+P(question)+P(finished)+P(error)), mapped so 0.4 reads 0.5; at >= 0.5 the agent needs the user.

Training

• Base: jhu-clsp/ettin-encoder-17m (MIT), fine-tuned from the previous release (which had learned from about 1,000 real terminal screens, scrubbed) with its classification heads held to their previous answers by distillation, while the card heads learned new labels.
• Data: about 17,700 synthetic terminal screens across 17 agent CLIs and 23 situations, generated from researched and live-captured interface fragments and prepared by Daintree's own screen code. Synthetic screens and labels were produced with OpenAI models.
• Card labels: every line of every screen scored 0-3 by gpt-6-luna against a written rubric (the agent's own words and results high; commands, tool output, chrome, menus and the user's prompt 0), revised once after reading real-screen failures.
• Quantisation: INT8 weight-only (int8 weights with per-channel scales, cast to float when the model loads), so the download is small but the arithmetic is float and it matches FP32: checked on an Apple M4 Max and an Intel Core 5 210H. The previous release's dynamic INT8 read wrongly on that Intel CPU (no VNNI); this one does not.

Limits

• Extractive: if the screen doesn't say it, the card won't either. One screen at a time, no memory between readings.
• English text only. New CLI layouts mostly read correctly; check the category on a new CLI before relying on it.
• Line choice is partly taste: against the previous card the judge preferred this one on 49 screens, the previous one on 16, and rated 88 the same.
• An interruption printed under tool output is not always picked as line one (the category and needs-you gate still are right).

Licence

MIT, as the base model.