Kubernetes can run on NVIDIA GPUs, allowing the container orchestration platform to leverage GPU acceleration. The NVIDIA device plugin enables GPU support in Kubernetes, so developers can schedule GPU resources to build and deploy applications on multi-cloud clusters.
Kubernetes has become increasingly important for developing and scaling machine learning and deep learning algorithms. If you are not a trained data scientist, containers can help simplify management and deployment of models. Containers allow you to package a model, making it more easily transferable. You don’t have to build a model from scratch every time, which can be complex and time consuming.
GPUs cannot be virtualized and allow developers to simultaneously process large data sets across heterogeneous environments, including cloud deployments and distributed networks. They can accelerate the development of data-heavy systems such as conversational AIs.
NVIDIA DGX systems support container orchestration for multiple open-source container runtimes, such as containerd, CRI-O and Docker. GPU metrics can be monitored via a monitoring stack, integrating NVIDIA DCGM with Prometheus and Grafana. You can specify attributes such as memory requirements and GPU type.
Container orchestration on NVIDIA GPUs is supported by a number of toolkits, which are continuously being developed. With the NVIDIA Container Toolkit, you can:
Build, deploy, orchestrate and monitor GPU-accelerated Docker containers
Automate container configuration using the container runtime library
Utilize Jetson edge devices running the same CUDA-X stack — for example, images pulled from NVIDIA GPU Cloud can be optimized for JetPack.
NVIDIA also offers a transfer learning toolkit that distributes pre-trained models for AI operations such as conversational AI and computer vision using Docker containers. Transfer learning allows you to transfer an existing neural network capability to a new model. Developers can use the NVIDIA GPU Cloud registry to access existing models packaged in containers.