Making the Most Out of Your Hardware Accelerators in a Kubernetes Clu... Rishit Dagli & Shivay Lamba
Channel: CNCF [Cloud Native Computing Foundation]
Date: 20/04/2023

Making the Most Out of Your Hardware Accelerators in a Kubernetes Cluster - Rishit Dagli, University of Toronto & Shivay Lamba

Working in the field of MLOps, making the most out of your hardware accelerators: GPUs, and TPUs, and optimizing their use is a very important aspect. Kubeflow, a popular open-source project makes working with MLOps easier but we have observed in our experiments that there are very few elements to handle using multiple GPUs and TPUs with Kubeflow. In this talk we share experiences and tips on how you can make use of the open-source projects TensorFlow and Kubernetes to accelerate your machine learning workloads. In this talk, we particularly show how each of the following components in using accelerators efficiently works in such a setting: managing computer instances differently than your hardware accelerators, understanding which Kubernetes Pods should be run on a GPU/ TPU and which ones on traditional compute, scaling, sharing hardware accelerators (time-sharing) and the link between a compute and hardware accelerator. We show the audience how they could optimize all these aspects and efficiently use hardware accelerators in their Kubernetes clusters.