Laya for WebGPU in the browser (int8 ONNX, fp32 embeddings)

Unofficial copy of convaiinnovations/laya (Apache-2.0, Copyright ConvAI Innovations) for onnxruntime-web. Used by https://archevel.github.io/laya/.

Made from harshpreet931/cut-laya-onnx (revision 192138dfb32a0b2f73425ba70345946d415bfde1: 8-bit weight-only MatMulNBits, split encoder/head for laya-ts) with one change: the token-embedding table in encoder.onnx is widened from fp16 to fp32 (lossless). With the fp16 table, onnxruntime-web's WebGPU Gather requires the shader-f16 feature, which Chrome does not expose on e.g. Linux with NVIDIA GPUs. Script: tools/embeddings_to_fp32.py in https://github.com/archevel/laya.

Outputs differ slightly from the PyTorch checkpoint because of the 8-bit weights.

Licensed under the Apache License, Version 2.0, like the original.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for archevel/laya-web

Base model

convaiinnovations/laya

Quantized

(35)

this model