MBHS v2 — the answer model of Find in Page AI (Qwen3.5-0.8B, fine-tuned on dataset v3)

The answer model of Find in Page AI, a browser extension that answers questions about the page you are reading, locally on WebGPU. This is Qwen3.5-0.8B with a LoRA (rank 16, all 186 linear layers) merged in. It is trained for the extension's answer format on dataset v3: 3,946 replies and 6,144 one-word-check examples on pages in 15 languages.

The ONNX files match onnx-community/Qwen3.5-0.8B-Text-ONNX:

• onnx/model_q4f16.onnx — WebGPU with shader-f16;
• onnx/model_q4.onnx — fp32 scales, for GPUs without shader-f16 and for the CPU.

The LoRA is in lora/ in mlx-lm's format.

v2 replaces MBHS (pilot B-360, 731 examples). It is as useful and is confidently wrong less often: 74 against 91 times on validation v3, 4 against 10 on live sites.

Prompt and reply

The same as MBHS: a system prompt and one user turn, with thinking off (enable_thinking: false), and no worked examples.

system: You answer questions about a web page using only the numbered fragments of it that the user gives you. Reply in this format: Quote [n]: … / Answer: … [n] / NOT_FOUND …
user:   Fragments:

        [1] (Section › Subsection) text…

        [2] …

        Question: …
        Answer in Russian.

• The reply: Quote [n]: <a sentence of fragment n, verbatim>\nAnswer: <one or two sentences, ending with [n]>, or NOT_FOUND when no fragment answers.
• Stopping: the model ends its turn with <|im_end|>, and generation_config.json stops on it as well as on <|endoftext|>.
• The one-word check: the model also does the extension's check ("Does the sentence answer the question?" → yes / no). For this model the check removes about as many right answers as wrong ones, so the extension can skip it.

Results

All models run on the same extension code and the same fragments from the extension's search. "Right" means shown as the answer, with the right fact, in the question's language. "Confidently wrong" means shown as the answer, but wrong.

	old bench: right / confidently wrong	validation v3: right / confidently wrong	live sites: useful / wrong shown
Base model, extension's prompt with examples	68 / 26	106 / 170	17 / 13
MBHS (v1)	85 / 15	190 / 91	48 / 10
Mac pilot	84 / 12	162 / 87	42 / 1
MBHS v2	92 / 11	191 / 74	48 / 4

• Old bench: 158 questions, 134 of them with an answer: test pages, Wikipedia, university and shop pages, and 37 questions in seven languages. q4 on the CPU, one-word check on.
• Validation v3: 480 replies on whole pages that are not in training, in 15 languages, 360 of them with an answer. q4 on the CPU, one-word check on.
• Live sites: 90 questions to 12 pages. Politecnico di Torino, Wikipedia, BBC, Guardian, BBC Sport, EPFL. Each extension build answered in Chrome on WebGPU with q4f16, and the answers were graded blind. "Useful" means a right answer or an honest "the page does not say".

Limitations:

• About a quarter of the answers in the live-site check have broken grammar or distorted words.
• On university and shop pages it still answers fewer questions than MBHS v1.
• French, Spanish and German are answered right about half the time.

Training

• Data: dataset v3 (finetune/v3/merged):
  • pages in 15 languages, about half of them English;
  • questions with about 22% that the fragments do not answer;
  • reference replies written from the fragments the extension shows, checked with its grounding checks;
  • plus one-word-check examples.
• LoRA (mlx-lm on a MacBook Air M3, 4 h 23 min):
  • rank 16, scale 20, dropout 0.05;
  • lr 1e-5 with cosine decay;
  • batch 1 with 4-step accumulation;
  • loss on the reply only;
  • 1.5 epochs, 15,135 steps; this is the snapshot at step 15,000.
• How the snapshot was chosen: screening in MLX, then the old bench, validation v3 and live sites in ONNX.
• Quantization: the linear layers are quantized again byte for byte like the official files (MatMulNBits, 4 bits, blocks of 32). Everything else in the ONNX graph is the official export.

Code and the full report: finetune/v3/ (TRAIN-REPORT.md) in the extension's repository, branch v3-train.