sjtu-ipads/powerinfer: High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
Category: llm
Language: C
Topics: falcon, large-language-models, llama, llm, llm-inference, local-inference