Ubuntu
add processing notice
ce130fe
|
raw
history blame
1.23 kB
metadata
inference: false

Vezora/Dolphin-llama-Instruct-8b AWQ

** PROCESSING .... ETA 30mins **

About AWQ

AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference with equivalent or better quality compared to the most commonly used GPTQ settings.

AWQ models are currently supported on Linux and Windows, with NVidia GPUs only. macOS users: please use GGUF models instead.

It is supported by: