squeezeailab/squeezellm: SqueezeLLM: Dense-and-Sparse Quantization
Category: llm
Language: Python
Topics: efficient-inference, large-language-models, llama, llm, localllm, model-compression, natural-language-processing, post-training-quantization, quantization, small-models, text-generation, transformer
Homepage: https://arxiv.org/abs/2306.07629v2