GitHub repository: turboderp-org/exllamav3
Primary language: Python
Description: An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs