Deploying Nvidia GPU enabled Tanzu Kubernetes Clusters
Tags: vmware, kubernetes, aws, gpu
Published: 2022-01-10 07:34:27

---
title: Deploying Nvidia GPU enabled Tanzu Kubernetes Clusters
published: true
date: 2021-12-09 23:01:04 UTC
tags: VMware,Kubernetes,AWS,GPU
canonical_url: https://veducate.co.uk/deploying-gpu-enabled-tanzu-kubernetes-clusters/#utm_source=rss&utm_medium=rss&utm_campaign=deploying-gpu-enabled-tanzu-kubernetes-clusters
---

In this blog post I’m going to detail how deploy and configure a Nvidia GPU enabled Tanzu Kubernetes Grid cluster in AWS. The method will be similar for Azure, for vSphere there are a number of additional steps to prepare the system. I’m going to essentially follow the official documentation, then run some of the Nvidia tests. Like always, it’s good to get a visual reference and such for these kinds of deployments.

###### Pre-Reqs

- Nvidia today only [support Ubuntu deployed images](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/platform-support.html#linux-distributions) in relation to a TKG deployment
- For this blog I’ve already deployed my TKG Management cluster in AWS

###### Deploy a GPU enabled workload cluster

It’s simple, just deploy a workload cluster that for the compute plane nodes (workers) that uses a GPU enabled instance.

You can create a [new cluster YAML file](https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.4/vmware-tanzu-kubernetes-grid-14/GUID-tanzu-config-reference.html#amazon-ec2-12) from scratch, or clone one of your existing located in:

```
~/.config/tanzu/tkg/clusterconfigs
```

Below are the four main values you will need to change. As mentioned above, you need a GPU enabled instance, and for the OS to be Ubuntu. The OS version will default if not set to 20.04.

```
CONTROL_PLANE_MACHINE_TYPE: t3.large
NODE_MACHINE_TYPE: g4dn.xlarge
OS_ARCH: amd64
OS_NAME: ubuntu
OS_VERSION: "20.04
```

The rest of the file you configure as you would for any workload cluster deployment.

[![TKG with GPU workload cluster file](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/TKG-with-GPU-workload-cluster-file.png?resize=604%2C987&ssl=1)](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/TKG-with-GPU-workload-cluster-file.png?ssl=1)

Create the cluster.

```
tanzu cluster create {name} -f {cluster.yaml}
```

You can retrieve the kubeadmin file to login by running.

```
tanzu cluster kubeconfig get {cluster_name} --admin
```

[![tanzu cluster create - kubeconfig get](https://i2.wp.com/veducate.co.uk/wp-content/uploads/2021/12/tanzu-cluster-create-kubeconfig-get.png?resize=604%2C196&ssl=1)](https://i2.wp.com/veducate.co.uk/wp-content/uploads/2021/12/tanzu-cluster-create-kubeconfig-get.png?ssl=1)

###### Deploying the Nvidia Kubernetes Operator

- Change the kubectl context to your newly deployed cluster.

Deploying the Nvidia operator couldn’t be easier, you can either download the files from the Cluster API for AWS github repo, or directly install them.

- [GPU cluster policy resource definition](https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-aws/de4fd54e6f988ca7fd3f94bce46867ba0523e23b/test/e2e/data/infrastructure-aws/gpu/clusterpolicy-crd.yaml)
- [GPU operator components](https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-aws/de4fd54e6f988ca7fd3f94bce46867ba0523e23b/test/e2e/data/infrastructure-aws/gpu/gpu-operator-components.yaml)

```
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-aws/de4fd54e6f988ca7fd3f94bce46867ba0523e23b/test/e2e/data/infrastructure-aws/gpu/clusterpolicy-crd.yaml

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-aws/de4fd54e6f988ca7fd3f94bce46867ba0523e23b/test/e2e/data/infrastructure-aws/gpu/gpu-operator-components.yaml
```

[![Install Nvidia Kubernetes Operator](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/Install-Nvidia-Kubernetes-Operator.png?resize=604%2C169&ssl=1)](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/Install-Nvidia-Kubernetes-Operator.png?ssl=1)

###### Validate the installation

Validate the operator pods in the default namespace, and then “nvidia” pods in the namespace “gpu-operator-resources”.

```
kubectl get pods

kubectl get pods -n gpu-operator-resources
```

[![validate nvidia operator installation - kubectl get pods -n gpu-operator-resources](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/validate-nvidia-operator-installation-kubectl-get-pods-n-gpu-operator-resources.png?resize=604%2C180&ssl=1)](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/validate-nvidia-operator-installation-kubectl-get-pods-n-gpu-operator-resources.png?ssl=1)

If you scale out your cluster with additional nodes, the Nvidia operator will ensure the additional pods run on the new nods.

###### Running the Sample Applications

From here to further validate, I am running the [sample applications](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/getting-started.html#running-sample-gpu-applications) from the Nvidia documentation.

So rather than copy the exact configs here, I’m just showing the outputs.

- CUDA VectorAdd

[![kubectl create cuda-vectoradd](https://i1.wp.com/veducate.co.uk/wp-content/uploads/2021/12/kubectl-create-cuda-vectoradd.png?resize=604%2C351&ssl=1)](https://i1.wp.com/veducate.co.uk/wp-content/uploads/2021/12/kubectl-create-cuda-vectoradd.png?ssl=1)

- CUDA load generator

[![kubectl create CUDA load generator FP16 Matrix multiply](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/kubectl-create-CUDA-load-generator-FP16-Matrix-multiply.png?resize=604%2C1106&ssl=1)](https://i0.wp.com/veducate.co.uk/wp-content/uploads/2021/12/kubectl-create-CUDA-load-generator-FP16-Matrix-multiply.png?ssl=1)

If you want to look at further examples, Nvidia have some fantastic Deep Learning examples in this repository.

- [https://github.com/NVIDIA/DeepLearningExamples](https://github.com/NVIDIA/DeepLearningExamples)

###### Wrap-up and Resources

Hopefully you can see that to use the GPU support with a Tanzu Kubernetes Grid cluster is quick and simple to setup and consume.

- Blog – [VMware Tanzu Kubernetes Grid Now Supports GPUs Across Clouds](https://tanzu.vmware.com/content/blog/tanzu-kubernetes-grid-supports-gpus-across-clouds)
- Documentation
  - [Deploy Tanzu Kubernetes Clusters to Amazon EC2](https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.4/vmware-tanzu-kubernetes-grid-14/GUID-tanzu-k8s-clusters-aws.html)
  - [Deploy a GPU-Enabled Cluster](https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.4/vmware-tanzu-kubernetes-grid-14/GUID-tanzu-k8s-clusters-aws.html#deploy-a-gpuenabled-cluster-6)

Regards

[Follow @Saintdle](https://twitter.com/saintdle)<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

[Dean Lewis](https://uk.linkedin.com/in/saintdle?trk=profile-badge)

The post [Deploying Nvidia GPU enabled Tanzu Kubernetes Clusters](https://veducate.co.uk/deploying-gpu-enabled-tanzu-kubernetes-clusters/) appeared first on [vEducate.co.uk](https://veducate.co.uk).