falcon-perception-webgpu — FPQ4 (q8 + q4-lm) weights

Custom mixed-precision quants of tiiuae/Falcon-Perception (0.6B open-vocabulary detection VLM) in the FPQ4 wire format of the falcon-perception-webgpu-demo browser runtime — hand-written WGSL kernels, no ONNX Runtime for the language stack, GPU-resident decode loop with speculative decoding for video.

falcon-q8-lm4.bin (724 MB, single file, everything the engine needs):

• all 113 layer matrices int8 (group 128) — box quality demands it: any single q4 group's noise flips coordinate bins (q4 box-set IoU vs fp32 is only 0.43; q8 reaches hidden cosine 0.9993);
• lm_head int4 — bit-exact argmax vs q8, −34 MB and −half the lm-read traffic per decode token;
• coord/size decode heads f16, Fourier coord/size encoders f32 — 0/14 bin mismatches vs the ONNX reference;
• embeddings f16.

Numbers on an Apple M4 (10-core GPU, ~120 GB/s) in Chrome:

	this runtime	original Space (fp32 ONNX/WASM)
full detection query	1.2 s	7.8–8.7 s
decode	91–95 tok/s	~8.7 tok/s
download	724 MB	2.4 GB
video keyframes	2.2–2.5 Hz (≤3 objects) + 30 FPS correlation tracker	—

Quality vs the fp32 original on identical pixels: 8/8 boxes matched, mean IoU 0.957. The runtime itself is verified bit-exact per layer (75/75) against a numpy dequantized-weights reference, decode token-for-token.

FPQ4 container

'FPQ4' magic + u32 manifest length + JSON manifest + 256-byte-aligned blobs, absolute offsets — one fetch, sliced straight into GPU buffers:

const head = new DataView(await file.slice(0, 8).arrayBuffer());
const mlen = head.getUint32(4, true);
const manifest = JSON.parse(new TextDecoder().decode(
    await file.slice(8, 8 + mlen).arrayBuffer()));
// per tensor t: file.slice(t.packed.offset, t.packed.offset + t.packed.length) → writeBuffer

q8 matrix ≈ 1.02 bytes/param (packed + f16 scales + i8 zeros, group 128); q4 ≈ 0.52 bytes/param. Only the mask heads (anyup/segm_head) stay in ONNX on the demo's segmentation path; detection runs 100% on WGSL.

Try it: https://huggingface.co/spaces/borkiss/falcon-perception-webgpu-demo (needs WebGPU with shader-f16 + subgroups — Chrome/Edge 125+).