Compressed LLMs for nm-vllm - a neuralmagic Collection

neuralmagic 's Collections

Sparse Foundational Llama 2 Models

Compressed LLMs for nm-vllm

DeepSparse Sparse LLMs

Compression Papers

Sparse Finetuning MPT

Compressed LLMs for nm-vllm

updated Mar 28

LLMs compressed using SparseGPT and GPTQ for optimized inference with nm-vllm https://github.com/neuralmagic/nm-vllm

neuralmagic/OpenHermes-2.5-Mistral-7B-pruned50

Text Generation • Updated Mar 5 • 1.34k • 1
neuralmagic/OpenHermes-2.5-Mistral-7B-pruned2.4

Text Generation • Updated Mar 5 • 1.25k
neuralmagic/OpenHermes-2.5-Mistral-7B-marlin

Text Generation • Updated Mar 6 • 831 • 1
neuralmagic/phi-2-pruned50

Text Generation • Updated Mar 5 • 26
neuralmagic/phi-2-super-marlin

Text Generation • Updated Mar 2 • 71 • 1
neuralmagic/Nous-Hermes-2-Yi-34B-pruned50

Text Generation • Updated Mar 5 • 14
neuralmagic/Nous-Hermes-2-Yi-34B-pruned2.4

Text Generation • Updated Mar 5 • 8
neuralmagic/Nous-Hermes-2-Yi-34B-marlin

Text Generation • Updated Mar 6 • 28 • 5
neuralmagic/TinyLlama-1.1B-Chat-v1.0-marlin

Text Generation • Updated Mar 6 • 1.39k • 1
neuralmagic/Llama-2-7b-pruned70-retrained-ultrachat

Text Generation • Updated 12 days ago • 41
neuralmagic/Llama-2-7b-pruned50-retrained-ultrachat

Text Generation • Updated 12 days ago • 131
neuralmagic/llama2.c-stories110M-pruned50

Text Generation • Updated Mar 5 • 459
neuralmagic/zephyr-7b-beta-marlin

Text Generation • Updated Mar 6 • 2.11k • 1
neuralmagic/Nous-Hermes-2-SOLAR-10.7B-pruned2.4

Text Generation • Updated Mar 5 • 16
neuralmagic/llama2.c-stories110M-pruned2.4

Text Generation • Updated Mar 5 • 29
neuralmagic/TinyLlama-1.1B-Chat-v1.0-pruned2.4

Text Generation • Updated Mar 5 • 571