Llama 3.1 GPTQ, AWQ, and BNB Quants - a hugging-quants Collection

hugging-quants 's Collections

Gemma2 AWQ Quants

Llama 3.2 3B & 1B GGUF Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

updated Sep 26

Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • Updated Sep 13 • 84.4k • 37
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4

Text Generation • Updated Sep 16 • 1.09k • 5
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 17.3k • 16
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 249k • 89
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit

Text Generation • Updated Sep 3 • 19.9k • 26
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 52.4k • 17
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 406k • 60
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4

Text Generation • Updated Aug 8 • 1.46k • 7
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 22.2k • 20