Qwen3-4B-ONNX-WebGPU

Qwen/Qwen3-4B converted to ONNX for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference, with an EAGLE-3 draft head for speculative decoding.

Changes from the original

• Converted to an ONNX decoder in ONNX Runtime GenAI's layout (input_ids, attention_mask, past_key_values.* → logits, present.*) by Quick Inference's own exporter.
• Linear layers are quantised to 4 bits (round to nearest, block size 32). Embeddings, activations and the KV cache are float16.
• The output layer is 4-bit too (the token embeddings are tied to it, so one 4-bit table serves both).
• The rotary embedding is applied inside GroupQueryAttention, and the per-head q/k RMS norms are folded into it as well (inputs 14 and 15 with qk_norm_epsilon, standard in ONNX Runtime 1.30).
• An extra output, eagle_taps: the residual stream entering three layers (2, N/2 and N-3, concatenated), for the EAGLE-3 draft head in draft/.

These are modified files: their outputs are close to, but not identical to, the original model's.

Draft head (draft/)

AngelSlim/Qwen3-4B_eagle3 (EAGLE-3) converted to ONNX: it guesses the next tokens from eagle_taps, and the model checks every guess, so the output is exactly the model's own. Its matrices are quantised to 4 bits (round to nearest, block size 32); its own KV cache is static; it returns its guess as a token id. Licensed under its own terms: draft/License_AngelSlim_model_and_dataset.txt.

Licence

The model is licensed under the same terms as the original model: see LICENSE.