Llama-3.2 Quantization - a neuralmagic Collection

neuralmagic 's Collections

Sparse-Llama-3.1-2of4

Vision Language Models Quantization

FP8 LLMs for vLLM

Llama-3.2 Quantization

Llama-3.1 Quantization

INT8 LLMs for vLLM

INT4 LLMs for vLLM

Sparse Foundational Llama 2 Models

Compression Papers

DeepSparse Sparse LLMs

Sparse Finetuning MPT

Compressed LLMs from the Community

Llama-3.2 Quantization

updated Sep 26

Llama 3.2 models quantized by Neural Magic

neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 156k • 15
neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 21.4k • 6
neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 1.15k • 2
neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 1.3k • 2
neuralmagic/Llama-3.2-1B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16 • 4.67k • 5
neuralmagic/Llama-3.2-3B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16 • 8.97k • 1
neuralmagic/Llama-3.2-1B-Instruct-FP8

Text Generation • Updated Oct 16 • 263k • 2
neuralmagic/Llama-3.2-3B-Instruct-FP8

Text Generation • Updated Oct 16 • 16.8k • 2
neuralmagic/Llama-3.2-1B-FP8

Updated Oct 9 • 234