Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
neuralmagic
's Collections
FP8 LLMs for vLLM
Llama-3.2 Quantization
Llama-3.1 Quantization
INT8 LLMs for vLLM
INT4 LLMs for vLLM
Sparse Foundational Llama 2 Models
Compression Papers
DeepSparse Sparse LLMs
Sparse Finetuning MPT
Compressed LLMs from the Community
FP8 LLMs for vLLM
updated
5 days ago
Accurate FP8 quantized models by Neural Magic, ready for use with vLLM!
Upvote
53
+43
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8
Text Generation
•
Updated
Aug 22
•
1.34k
•
28
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8
Text Generation
•
Updated
Aug 23
•
13.5k
•
27
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8
Text Generation
•
Updated
Aug 23
•
55.4k
•
28
neuralmagic/Phi-3-medium-128k-instruct-FP8
Text Generation
•
Updated
Aug 12
•
34.6k
•
5
neuralmagic/Mistral-Nemo-Instruct-2407-FP8
Text Generation
•
Updated
Jul 19
•
2.47k
•
13
neuralmagic/Meta-Llama-3-8B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
12.7k
•
17
neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic
Text Generation
•
Updated
Aug 22
•
260
•
13
neuralmagic/Meta-Llama-3-70B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
1.97k
•
10
neuralmagic/Mixtral-8x7B-Instruct-v0.1-FP8
Text Generation
•
Updated
Jul 18
•
1.32k
•
2
neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV
Text Generation
•
Updated
Jun 19
•
18.8k
•
6
neuralmagic/Meta-Llama-3-70B-Instruct-FP8-KV
Text Generation
•
Updated
Jun 26
•
199
•
2
neuralmagic/Mixtral-8x22B-Instruct-v0.1-FP8
Text Generation
•
Updated
Aug 12
•
320
neuralmagic/Qwen2-72B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
1.07k
•
9
neuralmagic/Qwen2-7B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
523
•
1
neuralmagic/Qwen2-1.5B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
92
neuralmagic/Qwen2-0.5B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
273
•
2
neuralmagic/Mistral-7B-Instruct-v0.3-FP8
Text Generation
•
Updated
Jul 18
•
601
•
2
neuralmagic/Llama-2-7b-chat-hf-FP8
Text Generation
•
Updated
Jul 18
•
266
neuralmagic/Phi-3-mini-128k-instruct-FP8
Text Generation
•
Updated
Aug 12
•
226
neuralmagic/gemma-2-9b-it-FP8
Text Generation
•
Updated
Jul 18
•
1.29k
•
5
neuralmagic/Qwen2-57B-A14B-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
354
•
1
neuralmagic/DeepSeek-Coder-V2-Lite-Instruct-FP8
Text Generation
•
Updated
Jul 18
•
2.37k
•
4
neuralmagic/DeepSeek-Coder-V2-Lite-Base-FP8
Text Generation
•
Updated
Jul 18
•
115
neuralmagic/DeepSeek-Coder-V2-Base-FP8
Text Generation
•
Updated
Jul 22
•
13
neuralmagic/DeepSeek-Coder-V2-Instruct-FP8
Text Generation
•
Updated
Jul 22
•
3.63k
•
6
neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8-dynamic
Text Generation
•
Updated
Aug 23
•
5.2k
•
5
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8-dynamic
Text Generation
•
Updated
Aug 23
•
1.73k
•
2
neuralmagic/Meta-Llama-3.1-8B-FP8
Text Generation
•
Updated
Aug 13
•
1.3k
•
5
neuralmagic/Meta-Llama-3.1-70B-FP8
Text Generation
•
Updated
Aug 13
•
249
neuralmagic/starcoder2-15b-FP8
Text Generation
•
Updated
Aug 1
•
61
neuralmagic/starcoder2-3b-FP8
Text Generation
•
Updated
Aug 1
•
27
neuralmagic/starcoder2-7b-FP8
Text Generation
•
Updated
Aug 1
•
8
neuralmagic/Meta-Llama-3.1-405B-FP8
Text Generation
•
Updated
Aug 13
•
7
neuralmagic/gemma-2-2b-it-FP8
Updated
Aug 13
•
428
•
1
neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic
Text Generation
•
Updated
6 days ago
•
382
•
1
neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic
Text Generation
•
Updated
6 days ago
•
189
•
1
neuralmagic/Llama-3.2-3B-Instruct-FP8
Text Generation
•
Updated
6 days ago
•
135
neuralmagic/Llama-3.2-1B-Instruct-FP8
Text Generation
•
Updated
5 days ago
•
135
neuralmagic/Llama-3.2-1B-FP8
Updated
6 days ago
•
38
neuralmagic/Phi-3.5-mini-instruct-FP8-KV
Text Generation
•
Updated
about 16 hours ago
•
106
Upvote
53
+49
Share collection
View history
Collection guide
Browse collections