GitHub repository: p-e-w/exllamav2
Description: A fast inference library for running LLMs locally on modern consumer-class GPUs