Instructions to use AMINOXDEV/webgpu-chat-graphs with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use AMINOXDEV/webgpu-chat-graphs with Transformers.js:
// โ ๏ธ Unknown pipeline tag
WebGPU Chat graphs
ONNX graph files (no weights) used by WebGPU Chat (source).
Each file is the original model graph with one change: every com.microsoft.MatMulNBits node gets
accuracy_level = 4. Nothing else is modified (same nodes, same inputs and outputs, same external-data
references). With it, ONNX Runtime's WebGPU backend may use its int8 DP4A matmul path when processing
prompts (4 or more tokens at once) on GPUs that expose subgroups. Token-by-token generation is unchanged.
The weights are not here: they are loaded from the original repositories, at the revisions below.
| File | Source graph | Weights |
|---|---|---|
qwen3.5-4b-opt/decoder_model_merged_q4f16.onnx |
onnx-community/Qwen3.5-4B-ONNX-OPT@bb07226 | same repo and revision |
gemma-4-12b-qat/decoder_model_merged_q4f16.onnx |
james58899/gemma-4-12B-it-qat-ONNX@60555e6 (QAT Q4_0 weights, fused GroupQueryAttention) | same repo and revision |
gemma-4-12b/decoder_model_merged_q4f16.onnx |
AMINOXDEV/gemma-4-12B-it-ONNX@90e0a27 | same repo and revision |
Licences follow the original models: Qwen3.5 (Apache 2.0) and Gemma 4 (Apache 2.0).
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support