Llama 3.1 GPTQ, AWQ, and BNB Quants
Collection
Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗
•
9 items
•
Updated
•
56
Daniël de Kok
danieldk
AI & ML interests
Fast, low-level ML code, model implementation
Recent Activity
updated
a model
about 1 month ago
danieldk/Llama-3.1-8B-w4a16-int-24
updated
a model
about 1 month ago
danieldk/Phi-3.5-MoE-instruct-AWQ-INT4
updated
a model
about 1 month ago
danieldk/Qwen2.5-1.5B-Instruct-w8a8-int-dynamic-weight
Organizations
