auto-gptq (pypi): An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
Keywords: gptq|quantization|large-language-models|pytorch|transformers|deep-learning|inference|llms|nlp|transformer