GitHub repository: turboderp-org/exllamav2
Primary language: Python
Description: A fast inference library for running LLMs locally on modern consumer-class GPUs