Repository: mit-han-lab/omniserve
Description: [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
Languages: C++,Cuda,C,Python,Shell
Stars: 514
License: Apache License 2.0