Building Machine Learning Inference Through Knative Serverless...- Shivay Lamba & Rishit Dagli
Channel: CNCF [Cloud Native Computing Foundation]
Date: 2022-11-01

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io​. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Lightning Talk: Building Machine Learning Inference Through Knative Serverless Framework - Shivay Lamba, Meilisearch & Rishit Dagli, University of Toronto Student

Deep learning is gaining tremendous momentum in certain academic and industry circles. Inference—the capability to retrieve information from real-world data based on pre-trained models—is at the core of deep learning applications. Deep learning inference can be used to classify images when they arrive in object storage, whether it's hosted on a public cloud, such as Amazon S3 or Azure Blob, or on-premises using an interface such as Ceph RADOS Gateway (RGW). The conventional workflow for this use case is that when an image is updated, it triggers an event, and the object storage publishes the event to its subscribers. The subscribers then download the image and send it to an inference service. Such an event-triggering mechanism can be found in AWS's Simple Queue Service. In this talk we are going to show how to leverage KNative serverless framework to create serverless solution for deep learning inference to move away from using on-prem storage.