TABLE OF CONTENTS
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . 1
System Configuration . . . . . . . . . . . . . . . . . . . . . . 2
Introduction to Kubernetes . . . . . . . . . . . . . . . . 4
Kubernetes Cluster Deployment. . . . . . . . . . . 4

Supermicro AMD based WIO System

Scale-up With Kubernetes Cluster . . . . . . . . . 8Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9SUPERMICROSupermicro (Nasdaq: SMCI), the leading
innovator in high-performance, highefficiency server and storage technology isa premier provider of advanced server
Building Block Solutions® for Enterprise
Data Center, Cloud Computing, Artificial
Intelligence, and Edge Computing
Systems worldwide. Supermicro iscommitted to protecting the environment
through its “We Keep IT Green®” initiativeand provides customers with the mostenergy-efficient, environmentally-friendly
solutions available on the market.

Executive Summary
The Deep Learning (DL) benchmark results in the previous white paper1 clearlyshow that a DL workload in Docker containers performs the same as on aBareMetal system. Building an on-prem Kubernetes cluster with GPU workers andAI framework-specific Docker containers can help organizations run projects orproductions in a highly reliable and scalable platform. This white paper
introduces Supermicro AMD based WIO systems, AS-1114S-WTRT, as Kubernetes
Admin and master nodes. Along with AS-2023US-TR4, we build an NVIDIA GPU
capable Kubernetes cluster that uses Cloud-native CEPH storage as persistent
volumes and demonstrates how a DL workload can scale the Kubernetes cluster.

1White paper: SUPERMICRO® SYSTEM COMBINES AMD EPYC™PROCESSORS AND NVIDIA GPUS TO ACHIEVECONSISTENT DEEP LEARNING PERFORMANCE WITH LINEAR SCALING

System Configuration

AS-1114S-WTRT is one of the Supermicro AMD EPYC™ 7002 series based WIO series servers, offering a wide range of I/O options
to deliver truly optimized systems for specific requirements. For more detailed system information, please go HERE.
Customers can optimize the storage and networking alternatives to accelerate performance, find the perfect fit for theirapplications. In our case, it uses the VMWare host and Kubernetes master nodes. Figure 1 and Figure 2 provide an overview ofthe system:
Table 1 provides a sample configuration for AS-1114S-WTRTFigure 1SystemPart NumberFigure 2QTYPart Description

Infrastructure

AS-1114S-WTRT1

H12 SSW-NT,CSV116TS-R504WBPCPU

PSE-ROM7552

1AMD EPYC™ 7552 DP/UP 48C/96T 2.2G 192M 200W 4094

MemoryMEM-DR416L-HL01-ER32

8

16GB DDR4-3200 2Rx8 ECC REG DIMM

HDD/SSD (Storage)

HDS-X2A-XS7680TE70004

2[NR]Seagate Lange 7.68TB SAS 12Gb/s, 15mm, 2.5", 0.8DWPD SSD, HF

HDS-SMN1-MZ1LB3T8HMLA07

2Samsung PM983 3.84TB NVMe PCIe3x4 V4 M.2 22x110mm (1.3 DWPD)

AOC

MCX4121A-ACAT

2Standard Low-profile Mellanox 25GbE card with 2x SFP28 ports

Storage Controller

AOC-S3008L-L8i

1

AOC-S3008L-L8i Retail Pack

Cable

CBL-SAST-0593

1Internal Mini-SAS HD to Mini-SAS HD 60cm,30AWG,12Gb/s

Software License

SFT-DCMS-SINGLE

1Supermicro System Management Software Suite node license, HF, RoHS/REACH, PBF

HDD/SSD (OS)Table 1
Table 2 provides cluster system roles and components. For more information on solution reference architectures, which consists of different
level bundle configurations with various software vendors depending on the customers' workloads, refer to our team's solution pagehttps://verticalsolutions.supermicro.com for more information.

AI Workloads At Scale

March 2021

2SystemRole

QTY

SoftwareAS-1114S-WTRT WIO server

Virtual host with Kubernetes admin node

1

VMWare 6.5

AS-1114S-WTRT WIO server

Kubernetes Master nodes

3Kubernetes 1.19

AS-2023US-TR4 with 2xV100 32GB PCIe GPU Per
NodeKubernetes Worker nodes

3Kubernetes 1.19
Nvidia-docker

SSE-X3648T

Network Switches TOR

2

SMCI OSTable 2

Figure 3 shows the reference architecture.

Figure 3

AI Workloads At ScaleMarch 2021

3

Introduction to Kubernetes
Kubeflow is a machine learning toolkit that runs on Kubernetes by abstracting machine learning solutions and best practices
utilizing the features of Kubernetes. It makes deployments of machine learning workflows portable and scalable and runs in
distributed environments. Kubeflow's core and ecosystem critical user journeys (CUJs) provide software solutions for end-toend workflows to build, train, deploy, and/or develop a model and create and run a workflow pipeline. Details regarding
Kubernetes are at: https://www.kubeflow.org/docs/started/kubeflow-overview/
The AI/ML applications can be deployed in either a single node cluster or multi-node clusters, with each node loaded withmultiple GPUs. Single node clusters can be used to build proof of concept solutions and smaller-scale training tasks utilizing
multiple GPUs. The single-node clusters are usually used in development environments and for non-production use. Multinode clusters are used in production deployments as they provide high availability, significant improvement in speed ofexecution, and efficient scalability.

Kubernetes Cluster Deployment
Making GPU work with the Kubernetes cluster on-prem can be a complicated task. Therefore, NVIDIA has developed a tool
named DeepOps, to automate Kubernetes cluster provisioning, configuring, and scaling. Further information regarding
DeepOps can be found here.
Figure 4 outlines the process to prepare and deploy the NVIDIA GPU cluster on-prem.

Figure 4AI Workloads At ScaleMarch 20214The tool is based on two open-source projects: Ansible and Kubespray, which are very flexible and scalable. It can deploy theKubernetes cluster onto a single GPU node or hundreds of systems except for the cluster-admin node. Once DeepOps isinstalled on the admin node, modifying the inventory file according to the design. Figure 6 is the inventory file located under
the deepops/config directory. In addition, the k8s-cluster.yml in the deepops/playbooks provides cluster options.

Figure 5

To set up the Kubernetes cluster with GPU installed worker nodes, run the following command:ansible-playbook --limit k8s-cluster playbooks/k8s-cluster.yml -k -K 3

2

Link to the k8s-cluster.yml file https://drive.google.com/file/d/1EZLH4Ifp1IVtaJWhPOvcjvNyut7L5ady/view?usp=sharing

3

More instructions can be found in https://verticalsolutions.supermicro.com and HERE

AI Workloads At Scale

March 2021

5

After the installation process is completed, the Kubernetes cluster should be up and running. To validate the cluster, type thefollowing command on the admin node: kubectl get node. Figure 6 shows the output.

Figure 6Figure 7 shows the overall architecture discussed in this paper.

Figure 7

NVIDIA DeepOps is a handy open-source tool to deploy GPU Cloud-Native clusters. The tested environment consists of one VM
as a deployment host and two AS-2023US-TR4 as GPU nodes. You can find the detailed features and configurations here.

AI Workloads At Scale

March 2021

6

The tool also installs and configures a Cloud-native certified rook-ceph CEPH storage system as the cluster's persistentvolumes, which is a must for all Deep Learning workloads since the datasets and intermediate training results have to beshared among the GPU nodes. Figure 8 and Figure 9 show the dynamic CEPH StorageClass information for the Kubernetes
cluster.

Figure 8Figure 9AI Workloads At ScaleMarch 2021

7

Scale-up with Kubernetes Cluster
Not only does a Cloud-native platform can provide reliable, resilient, and distributed microservices, but it also can scale up ondemand. In the test environment, we have tested Deep Learning benchmark throughput with the number of POD replicas.
Figure 10 is the snippet of the Kubernetes YAML configuration of the Deep Learning benchmark application. Figure 11 depicts
the benchmark under different Deep learning frameworks.

Figure 10Figure 11

AI Workloads At ScaleMarch 20218

Conclusion
The foundation of Cloud-native technologies is Kubernetes can be complicated to deploy, maintain and upgrade. Adding GPU
into the mix can introduce another layer of complexity. However, with Supermicro AS-1114S-WTRT WIO systems, NVIDIA GPU
appliances, and NVIDIA DeepOps tool, it becomes more straightforward for an organization to build a private Cloud-nativeplatform along with MLOps applications. From a designing perspective, AS-1114S-WTRT as a flexible cloud node can meetcompute, storage, and virtualization requirements for the hardware infrastructure; From an operational aspect, DeepOps
makes the on-prem Cloud-native platform easier to deploy, maintain, and scale. This paper also demonstrated how theKubernetes PODs could speed up Deep Learning processes that can reduce time-to-market.
Referencehttps://docs.nvidia.com/datacenter/cloud-native/index.html
https://www.kubeflow.org/docs/started/kubeflow-overview/

AMD, the AMD Arrow logo, EPYC, and combinations thereof are trademarks of Advanced Micro Devices, Inc.

AI Workloads At ScaleMarch 2021

9