GitHub repository: IBM/onnx-mlir-serving
Primary language: C++
Created: 2022-08-26T06:16:34Z
Description: ONNX Serving is a project written with C++ to serve onnx-mlir compiled models with GRPC and other protocols.Benefiting from C++ implementation, ONNX Serving has very low latency overhead and high throughput. ONNX Servring provides dynamic batch aggregation and workers pool to fully utilize AI accelerators on the machine.