FP8 LLMs for vLLM Collection Accurate FP8 quantized models by Neural Magic, ready for use with vLLM! • 44 items • Updated Oct 17, 2024 • 68
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic Text Generation • Updated Oct 19, 2024 • 191 • 14