ProCreations/betterwright-encoder-350m

BetterWright Encoder 350M

A task-conditioned accessibility-tree relevance encoder for reducing browser agent context while preserving action targets and evidence. It is trained from LiquidAI/LFM2.5-Encoder-350M in bfloat16.

All 355,011,331 parameters are trained; this checkpoint does not use LoRA, adapters, or a frozen backbone. The model has three outputs: chunk relevance, token/node relevance, and a confidence signal for deterministic fallback. BetterWright keeps its original snapshot whenever the confidence gate is not met.

The training corpus contains 500,000 deduplicated structural examples grounded in 5,256 independently generated and audited tasks from 141 real-site domains. Domains—not rows—define the split: 110 train, 8 validation, and 23 held-out test domains. The corpus is primarily BetterWright accessibility trees, with a smaller share of alternate accessibility serializations for robustness.

Input

[BETTERWRIGHT_TASK]
<the current browser task or observation query>
[ACCESSIBILITY_SUBTREE]
<BetterWright aria tree chunk>

The released checkpoint is progressively trained at 8K, 16K, 32K, and 64K sequence lengths. Normal inference should still use hierarchical chunks for lower latency; 64K is intended for unusually large dumps and packed tabs.

Quick start

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "ProCreations/betterwright-encoder-350m"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).eval()
batch = tokenizer(
    "[BETTERWRIGHT_TASK]\nOpen pricing\n[ACCESSIBILITY_SUBTREE]\n"
    '- link "Pricing" [ref=e12]',
    return_tensors="pt",
)
with torch.inference_mode():
    output = model(**batch)
relevance = torch.sigmoid(output.logits)
confidence = torch.sigmoid(output.uncertainty_logits)
token_relevance = torch.sigmoid(output.token_logits)

Production pruning should use the validated thresholds in relevance_config.json, deterministic must-retain rules, and exact full-tree fallback. A raw relevance score alone is not a safe deletion decision.

Training

• 500,000 unique pairs: 77.7% BetterWright trees, 13.1% alternate ARIA trees, and 9.1% compact accessibility outlines.
• 391,777 train, 25,891 validation, and 82,332 untouched-test pairs, split by domain before structural expansion.
• Full-parameter BF16 AdamW training with global batch 128, a cosine schedule, and a class weight derived from the exact training distribution.
• A balanced positive/negative packed-tree curriculum at 8K, 16K, 32K, and 64K tokens, with exact required-node span supervision.

Evaluation policy

Fallback thresholds are selected on validation domains. The production policy uses eight ranked 1,800-character structural regions, preserves referenced nodes, bounded child context, ancestry, and deterministic must-retain states, and returns the exact original tree on low confidence, errors, insufficient benefit, or an over-limit result.

Domain split	Tasks	Target/evidence recall	Perfect-task recall	Pruning coverage	Mean token savings when active	Token-weighted savings
Validation (8 domains)	288	100.000%	100.000%	12.50%	33.38%	3.42%
Held-out test (23 domains)	877	99.584%	99.544%	22.46%	44.09%	9.69%
Post-training reserve (17 additional domains)	161	100.000%	100.000%	33.54%	16.78%	2.42%

The exact reports and frozen policy are distributed as eval.json, reserve_eval.json, and relevance_config.json. The 99.5% target is evaluated on whole tasks and required accessibility refs, not inferred from training loss. A nine-site live navigational replay retained and clicked 6/9 current targets; two large pages exceeded the runtime ceiling after fallback and one site did not expose the requested label. This limitation is intentionally not hidden by substituting easier sites.

Safety and limitations

This is a relevance model, not an autonomous browser agent. It must not be the only path for security decisions. Snapshot text is untrusted data. Consumers should keep deterministic must-retain rules and fall back to the full tree on model errors, timeouts, low confidence, or insufficient retained context. Savings are substantial when pruning activates, but coverage is deliberately conservative and very large pages can still fall back above the caller's size ceiling. This checkpoint is therefore an alpha integration, not a claim that every page becomes smaller.