GitHub repository: Anirudhaagrawal/sarathi-serve
Primary language: Python
Description: A low-latency & high-throughput serving engine for LLMs