GitHub repository: AlibabaPAI/llumnix
Primary language: Python
Description: Efficient and easy multi-instance LLM serving