Llama-3.1 Quantization - a neuralmagic Collection

neuralmagic 's Collections

Vision Language Models Quantization

FP8 LLMs for vLLM

Llama-3.2 Quantization

Llama-3.1 Quantization

INT8 LLMs for vLLM

INT4 LLMs for vLLM

Sparse Foundational Llama 2 Models

Compression Papers

DeepSparse Sparse LLMs

Sparse Finetuning MPT

Compressed LLMs from the Community

Llama-3.1 Quantization

updated 8 days ago

Neural Magic quantized Llama-3.1 models

neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8

Text Generation • Updated Oct 9 • 53.8k • 32
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8

Text Generation • Updated Oct 9 • 108k • 34
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w4a16

Text Generation • Updated Oct 10 • 21k • 25
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16

Text Generation • Updated 29 days ago • 10.7k • 21
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16

Text Generation • Updated 29 days ago • 6.05k • 9
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16

Text Generation • Updated Oct 10 • 2.47k • 12
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic

Text Generation • Updated Oct 19 • 214 • 14
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • Updated Oct 10 • 567 • 2
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8-dynamic

Text Generation • Updated Oct 19 • 1.65k • 4
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8

Text Generation • Updated Oct 10 • 6.25k • 13
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8-dynamic

Text Generation • Updated Oct 19 • 2.4k • 5
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8

Text Generation • Updated Oct 9 • 10.2k • 31
neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a8

Text Generation • Updated 29 days ago • 8.6k • 12
neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 1.55k • 3
neuralmagic/Meta-Llama-3.1-8B-FP8

Text Generation • Updated Oct 9 • 3.61k • 5
neuralmagic/Meta-Llama-3.1-70B-FP8

Text Generation • Updated Oct 9 • 722
neuralmagic/Meta-Llama-3.1-8B-quantized.w8a16

Text Generation • Updated Oct 9 • 456 • 1
neuralmagic/Meta-Llama-3.1-8B-quantized.w8a8

Text Generation • Updated 29 days ago • 617 • 1
neuralmagic/Meta-Llama-3.1-405B-FP8

Text Generation • Updated Oct 9 • 54
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 512 • 2