@ucalyptus on Hugging Face: "PREM-1B-CHAT QUANTIZED INTO Q4 THEN SERVED IN WEBGPU DEMO OG model…"

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

ucalyptus

posted an update May 20

Post

1517

PREM-1B-CHAT QUANTIZED INTO Q4
THEN SERVED IN WEBGPU DEMO

OG model premai-io/prem-1B-chat
Q4 model ucalyptus/prem-1B-chat-onnx-q4
WEBGPU demo ucalyptus/prem-1B-chat-webgpu

May 20

Outstanding issues :

Fix Q4 demo
https://huggingface.co/spaces/ucalyptus/prem-1B-chat-webgpu/discussions/1#664b621d8742922b9e4f3de8
Also work on fp16 (see what onnxruntime-web has to say about this)

In this post

ucalyptus Sayantan Das