GitHub repository: kp-forks/PowerInfer
Primary language: C++
Description: High-speed Large Language Model Serving on PCs with Consumer-grade GPUs