medgaze-webgpu: fixation-head steering for SmolVLM-256M in the browser

ONNX files for the in-browser fixation-head demo of the GazeHeads project. The model is HuggingFaceTB/SmolVLM-256M-Instruct (Apache 2.0), re-exported so a web page can steer its fixation heads and read their attention. It is a small generalist model used to demonstrate the mechanism. It is not one of the medical models evaluated in the paper and must not be used for clinical purposes.

• vision_q8.onnx (110 MB): SigLIP encoder + pixel-shuffle connector. pixel_values [N,3,512,512] to image_features [N,64,576]
• decoder_q8.onnx (152 MB): 30-layer decoder + LM head with the steering input and the attention output (below)
• embed_tokens.bin (29 MB): token embeddings: 49280 fp32 row scales, then int8 rows [49280,576]
• tokenizer.json (3.5 MB): the base model's tokenizer
• demo_config.json (16 KB): discovered heads, special token ids, tokenizer test vectors

Weights are 8-bit block quantised (MatMulNBits, block 32), which runs on the onnxruntime-web WebGPU and WASM execution providers.

Decoder interface

Inputs: inputs_embeds [1,S,576], past_kv [30,2,3,P,64] (keys and values of every layer, P may be 0), head_gain [30,9], key_sign [P+S].

Outputs: logits [1,49280] (last position), present_kv [30,2,3,P+S,64], attn_last [30,9,P+S] (post-softmax attention of the last query row, every head).

Head (l,h) gets head_gain[l,h] * key_sign[k] added to its attention logit on key k. With key_sign = +1 on the image tokens inside a region, -1 on the other image tokens and 0 on text, and head_gain = g on the fixation heads and 0 elsewhere, this is the paper's steering bias. Use it as the paper does: prefill all but the last prompt token with zero gain, then feed the last prompt token and every generated token with the gain on.

Checks

• Export: the fp32 graphs give the same greedy tokens as the PyTorch model steered with the medgaze.steering hooks, and the same fixation-head attention (r = 1.0000), on 3 SLAKE images, unsteered and steered.
• 8-bit: fixation-head attention map r = 0.996 to 0.999 against PyTorch; greedy answers identical for the first 5 to 24 tokens (complete answer identical in 5 of 12 runs). 4-bit was tried and rejected (answers change within 1 to 3 tokens, r as low as 0.50).

Fixation heads were found with the paper's discovery protocol on 200 VQA-RAD montages of 6 images. Built by scripts/118_webgpu_export.py in the GazeHeads repository.