GitHub repository: ykeming/mosec
Description: A high-performance serving framework for ML models, offers dynamic batching and multi-stage pipeline to fully exploit your compute machine