php-ai/zeiger2-3.5-0.8b

Zeiger2 3.5-0.8B

Experimental. One training run (single seed). The successor candidate to php-ai/zeiger-0.6b; read the limits below before replacing it.

Zeiger — German for the hand that points. Give it a web page and an instruction, and it points at the element the instruction means — or says none of these — in a single forward pass. It answers yes/no and ordinal questions the same way. Nothing is generated: every option gets a calibrated probability.

Zeiger2 moves to the Qwen3.5-0.8B-Base hybrid backbone (Gated DeltaNet + full attention) and adds three structures that were selected in controlled experiments: non-causal full-attention layers, a memory head, and edit-pair training with two-order answering.

Results

Both models answer exactly the same questions on the same machine (AMD gfx1151), through the same code.

Lab suite — 2,000 held-out questions on websites and sources that appear nowhere in training:

	Zeiger2 3.5-0.8B	Zeiger 0.6B (round 17)
short questions, top-1 (600)	0.870	0.870
whole pages, top-1 (200)	0.825	0.765
general decisions, top-1 (600)	0.798	0.810
structured questions, top-1 (300)	0.790	0.590
logic, top-1 (300)	0.803	0.263 †
score (0.4 short + 0.4 long + 0.2 general)	0.838	0.816
"none of these", F1 (short / whole pages)	0.812 / 0.870	0.886 / 0.720
calibration error, ECE (short / pages / general)	0.041 / 0.061 / 0.077	0.047 / 0.079 / 0.039

† Zeiger 0.6B was never trained on the logic set's question family.

Decision Index 0.3 (multimodalart/jev-decision-index) — out of domain, neither model trained on it; a stratified sample of 3,000 requests (7,679 questions over all 36 benchmarks), scored with the kit's own scorer:

	Zeiger2 3.5-0.8B	Zeiger 0.6B
index (skill over 5 areas, 0 = chance)	17.6	14.1
knowledge & reasoning	8.3	8.6
language understanding	16.9	6.2
retrieval & classification	19.8	19.8
tools & automation	33.2	31.2
arts & human taste	10.6	5.8
calibration error (ECE)	0.158	0.178

The gain is uneven: Zeiger2 is better on 18 of 37 benchmarks and worse on the rest (FinEntity +67 skill points, Humicroedit +21, CRUXEval +19; GPQA −20, WinoGrande −17, CLINC150 −9).

Use it

hf download php-ai/zeiger2-3.5-0.8b --local-dir zeiger2
pip install -r zeiger2/requirements.txt

import sys; sys.path.insert(0, "zeiger2")
from zeiger2 import Engine

engine = Engine("zeiger2")            # picks the GPU when there is one

answers = engine.decide(
    {"instruction": 'Click the "Sign in" button', "page": {"title": "Shop", "url": "https://shop.test/"}},
    {"q": {"type": "choice",
           "instructions": "Which page element does the instruction refer to?",
           "criteria": {"e0": 'link "Home" href=/',
                        "e1": 'button "Sign in" in header',
                        "e2": 'input "Search"',
                        "none": "none of the listed elements fits this step"}}})

answers["q"]["choice"]         # 'e1'
answers["q"]["confidence"]     # 0.99997
answers["q"]["probabilities"]  # every option

decide() takes the same state and questions as Zeiger 0.6B and returns the same fields (choice, noul, score). The Zeiger engine serves this model too — Engine(path) picks the Zeiger2 engine from config.json, and its HTTP server and Docker images work unchanged:

git clone https://github.com/PurHur/zeiger && cd zeiger && pip install -r requirements.txt
python serve.py --model ../zeiger2          # POST /decide

zeiger2.py and the model code in zlab/ in this repository work standalone, without the engine.

On NVIDIA GPUs install flash-linear-attention and causal-conv1d for the fast DeltaNet kernels; without them transformers falls back to reference kernels (correct, much slower). Engine(..., two_order=False) halves the latency at a small cost in accuracy.

How it works

Qwen3.5-0.8B-Base with its language-model head removed is used as an encoder. The page becomes one sequence — state, question, then every option followed by a marker token — and the hidden state at each marker is that option's summary. Pages beyond 2k tokens are encoded in chunks that share a prefix.

Changes against Zeiger 0.6B, each tested against a control on the same data and budget before being combined:

• Non-causal full attention: the 6 full-attention layers of the hybrid backbone attend in both directions, so an option's summary sees the options after it.
• Memory head: 4 pooled memory tokens per chunk, cross-attended by every option marker (8 heads), on top of the 2-layer set transformer across options.
• Edit-pair training: minimally edited twins of examples whose answer flips, so the model learns which detail decides.
• Two-order answering: every question is answered with the options in the shown and the reversed order, and the probabilities are averaged per option.

Training: one pass over 60k examples (2,500 steps, ~94M tokens, 32k-token window) from the base weights, on one A100-80GB in 3.3 hours. Losses: cross-entropy, spherical score, decision reward and Brier.

Limits

• Speed: on the test machine it takes 234 ms per question against 69 ms for Zeiger 0.6B — about half of that is two-order answering, the rest DeltaNet reference kernels on AMD.
• It says "none of these" less reliably than Zeiger 0.6B on short questions (F1 0.812 vs 0.886) and is less well calibrated on general decisions (ECE 0.077 vs 0.039).
• One training seed; the training curve was still rising at the end (lab score 0.745 → 0.785 → 0.818 → 0.825 → 0.838 at 10/25/50/75/100 %).
• It reads text descriptions of elements, not pixels. It answers questions; it does not plan, click, or produce text. English and German.

Licence and credits

Weights: CC-BY-SA-4.0 — share adaptations of the weights under the same licence. The code in zlab/ and zeiger2.py is Apache-2.0.

Built on Qwen/Qwen3.5-0.8B-Base (Apache-2.0).

Trained on public datasets, credited to their authors under their licences: SNLI, STS-B, ARC (CC-BY-SA-4.0); BoolQ, DBpedia, MultiNLI (CC-BY-SA-3.0); Mind2Web, MT-Mind2Web, MASSIVE, Banking77, SMS-Spam (CC-BY-4.0); CLINC150 (CC-BY-3.0); GoEmotions (Apache-2.0); CommonsenseQA, HellaSwag (MIT); civil_comments (CC0); plus synthetic structured-reasoning and logic data generated for this project.