serving_agent (pypi): A middleware for model serving to speedup online inference.
Keywords: serving_agent