Embla1-4bit, WebGPU step graph

The ONNX graph that runs prefill for Embla 1 on WebGPU (live demo), a browser engine for sevren-ai/Embla1-4bit. One step of the model (a prefill chunk of up to 64 tokens, or one decode token) with the KV / engram-conv cache as explicit inputs and outputs, exported from the PyTorch reference and rewritten for onnxruntime-web's WebGPU execution provider.

  • embla_step.onnx + embla_step.onnx.data (0.48 GB): the non-expert weights in MLX's 4-bit format, as MatMulNBits with an exact offset term for the affine bias.
  • The routed experts (3.9 GB) are graph inputs, not part of this repo. The engine streams them from sevren-ai/Embla1-4bit/model.safetensors. The engram rows are an input too (looked up in that repo's engram_table.safetensors).

Build: sh tools/build_graph.sh in the GitHub repo (export + four graph rewrites, documented there).

Model by sevren.ai, MIT licensed. This is a derived format of their weights, under the same license.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for rasgaard/Embla1-4bit-webgpu

Quantized
(1)
this model