DeepSeek-R1-Distill-Qwen-7B-ONNX-WebGPU

deepseek-ai/DeepSeek-R1-Distill-Qwen-7B converted to ONNX for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference.

Changes from the original

Converted to ONNX with ONNX Runtime GenAI's model builder (-p int4 -e webgpu).
The weights of the linear layers are quantised to 4 bits (k-quant, block size 32); the rest are float16.
Activations and the KV cache are float16.

These are modified files: their outputs are close to, but not identical to, the original model's.

Licence

Licensed under the same terms as the original model: see LICENSE, LICENSE-Qwen2.5. Qwen2.5 is licensed under the Apache License 2.0; see LICENSE-Qwen2.5.

Files

LICENSE
LICENSE-Qwen2.5
chat_template.jinja
genai_config.json
model.onnx
model.onnx.data
tokenizer.json
tokenizer_config.json

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for danyalmalik/DeepSeek-R1-Distill-Qwen-7B-ONNX-WebGPU

Base model

deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

Quantized

(204)

this model