SOLUTION BRIEF

SYS-120U-TNR

2P / 1 GPU

SYS-220U-TNR

SYS-120GQ -TNRT

SYS-220GP-TNR

2P / 2 GPU

AS -2124GQ-NART/+

SYS-220GQ-TNAR/+

SYS-740GP-TNRT

AS -4124GS-TNR

2P / 4 GPU

2P / 8 GPU

Executive Summary
TABLE OF CONTENTS
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Artificial Intelligence Applicable Across Industries . . . . . . . . . . . . . . . . 2
Incorporating Artificial Intelligence into Enterprise Applications . . . . 2
Enterprise Challenges to Adopting AI . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Easing into AI, Leveraging Existing Virtual Infrastructure . . . . . . . . . . . 3
NVIDIA AI Enterprise Software Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
AI Software Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Enterprise Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Virtualized GPU Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Mixing CPU Cores and GPU Cores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Supermicro NVIDIA-Certified Systems, also VMware Certified . . . . . . . 7
Conclusion and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

SUPERMICROAs a global leader in high performance, high efficiency server
technology and innovation, we develop and provide end-to-end green
computing solutions to the data center, cloud computing, enterpriseIT, big data, HPC, and embedded markets. Our Building BlockSolutions® approach allows us to provide a broad range of SKUs, andenables us to build and deliver application-optimized solutions based
upon your requirements.

Enterprises are increasingly using AI in their IT workflows. AItechniques have proven to be very effective in identifying
patterns, finding defects in manufacturing, recognizing objects
and images, generating effective recommendations, and manyother functions. By incorporating these new AI techniques,
enterprises can significantly improve their operational efficiency,
grow their top lines, and optimize their bottom lines.
There are several challenges to incorporating these newtechnologies, including identifying the AI frameworks and tools,
optimized systems, and implementing IT management softwarefrom development to deployment. NVIDIA AI Enterprise, an endto-end cloud-native suite of AI and analytics software, addresses
these challenges. NVIDIA AI Enterprise brings AI to existing
VMware vSphere infrastructure that many enterprises use.
Developers and IT administrators can incorporate AI training andinference in small steps. With included AI and data science tools
and frameworks, enterprises can start with small investments
and grow that capability over time. In addition, enterprises candeploy NVIDIA AI Enterprise software optimized and certified by
NVIDIA to run on VMware vSphere with Supermicro NVIDIACertified Systems™. The servers are also VMware certified tosupport GPUs.

1© 2021 Copyright Super Micro Computer, Inc. All rights reserved

November 2021

1

As a leader in Green Computing, Supermicro is offering multiple choice of energy efficient server options, so customers can best
optimize on power, cooling, space, and cost to fit their infrastructure while bringing AI to their enterprise.

Artificial Intelligence Applicable Across Industries
Since the advancement of GPUs to accelerate deep neural network processing and the fast-paced evolution of the technology
in the last few years, deep learning artificial intelligence has evolved into multiple techniques used across every industry. Thetechniques include image and object recognition, text and speech understanding, finding correlations and anomalies in data.
Here are some examples in select industries:••••

In Finance, text and speech understanding can be applied to news and information to extract potential trading signals.
In Healthcare, image processing is applied to find diseases in MRI, X-Ray, or other medical imaging techniques.
In Retail, image processing can be used to build automated customer purchases in automated stores.
In Manufacturing, object recognition can be applied to identify components and to find manufacturing defects.

The applications are pervasive, and most industries are beginning to take advantage of these artificial intelligence capabilities.

Incorporating Artificial Intelligence into Enterprise Applications
Within an industry, a corporation or enterprise can apply artificial intelligence in the various functions within the organization,for example:•••

Marketing and sales departments use
collaborative filtering to make cross productselling to customers.
Human resources departments use text
processing to identify talents.
Manufacturing and services departments usecomputer vision techniques to identify
defects in parts and components.

Each department within the enterprise can takeadvantage of AI techniques to make jobs more
efficient and more cost effective, producing morereliable and accurate results.
Multiple departments can share the compute resources to maximize results and minimize infrastructure costs by leveraging
virtualized data centers and NVIDIA AI Enterprise.

2© 2021 Copyright Super Micro Computer, Inc. All rights reserved

November 2021

2

Enterprise Challenges to Adopting AI
For many enterprises, there could besignificant challenges to adopting artificialintelligence. Since the recent AI
advancements are mostly new and fastchanging, enterprises have to find thepeople and know-how with domainspecific understanding to successfully
integrateAI
intotheir
existing
applications.

Challenges

Solutions

Additional Assistance

PeopleData scientists

Best-in-class AI frameworks and tools
included in the NVIDIA AI Enterprise suite

Knowhow

AI and domain knowledge

NVIDIA AI Enterprise Support Services

ConsistentTools

Containers with consistentand tested tools & libraries

Regular updates & support

Performance

NVIDIA Ampere-based GPUsand networking solutions

Supermicro NVIDIA-Certified Systems

With NVIDIA AI Enterprise running on
NVIDIA AI Enterprise on
Supermicro NVIDIA-Certified Systems,
Risk
Supermicro NVIDIA-Certified Systems
existing VMware infrastructure
enterprises
canovercome
thesechallenges to start integrating AI intoScaling
VMware scaling and future
Supermicro NVIDIA-Certified Systems,
Deployment
Kubernetes support
networking
enterprise workflows. Supermicro NVIDIACertified Systems and NVIDIA AI Enterprisesoftware together minimize risks because the entire solution is pretested through NVIDIA’s rigorous certification process. Inaddition, the certified solutions assure performance and deployment scaling as the systems and software are designed toprovide high performance and can be scaled using server clusters.
Supermicro servers are designed for high performance and cluster scaling, with built-in PCI-Express Gen-4 I/O to support highbandwidth, low latency networking devices such as the NVIDIA ConnectX-6 and ConnectX-7, fast storage such as the latest Gen4NVMe disks, as well as the NVIDIA GPUs. As a result, data scientists and developers can get the most productivity using theSupermicro systems to run their AI development and deployment.

Easing into Artificial Intelligence, Leveraging Existing Virtual InfrastructureWith VMware vSphere, NVIDIA AI
Enterprise makes it easy to start AI
development in small steps forenterprises that have virtualized their ITinfrastructure. An enterprise can startwith a single server with a single GPU torun NVIDIA AI Enterprise. This AI
Enterprise system operates as a memberin the VMware vSphere environment.
vCenter Server manages the AI
Enterprise server as another member ofthe infrastructure, immediately enabling
AI workloads for the enterprise.
Depending on the size of the AI modeland batch data, an AI developer may
need a fraction of a GPU or full utilization of a GPU. This approach minimizes risk and maximizes the opportunity to create AIvalue. Over time, as the enterprise begins to realize the potential of AI enabled applications, the IT department in the enterprise
can add servers running NVIDIA AI Enterprise to scale the applications.

3© 2021 Copyright Super Micro Computer, Inc. All rights reserved

November 2021

3

NVIDIA AI Enterprise Software Suite
NVIDIA AI Enterprise is a software suite that runs in the VMware vSphere environment running on Supermicro NVIDIA-CertifiedSystems. The software suite includes AI tools and frameworks, cloud native deployment, and infrastructure optimization
software to enable rapid AI development and deployment in VMware infrastructures.

NVIDIA AI Enterprise is optimized, certified, and supported on VMware to achieve near bare-metal performance with
virtualization of AI workloads on NVIDIA-Certified Systems, of which Supermicro offers a wide myriad of optimal choices. By
making it a low risk and simple approach to integrate AI into the existing enterprise virtualization environment, NVIDIA AI
Enterprise enables an end-to-end software stack approach to start using AI in the enterprise. Enterprise developers caninitially run small trials until they feel comfortable expanding to more extensive deployment. At that point, the solution is very
scalable to deployment in multiple racks.
The design of NVIDIA AI Enterprise targets two personas:••

Data scientists and developers who use and incorporate AI, andIT administrators who need to ensure operational uptime of the systems

NVIDIA AI Enterprise provides the AI and data science tools and frameworks, which data scientists and developers need fortheir AI applications. By using Supermicro NVIDIA-Certified Systems, the IT administrators get assurance of reliable systems
with high performance. These Supermicro systems are also VMware certified to support GPUs. Using VMware, the ITadministrators are familiar with their existing infrastructure.

4© 2021 Copyright Super Micro Computer, Inc. All rights reserved

November 2021

4

AI Software Stack
The NVIDIA AI Enterprise software suite offers the best-in-class tools and frameworks that AI practitioners and data scientists
need, from data preparation (NVIDIA RAPIDS), training neural networks (TensorFlow and PyTorch), inference (NVIDIA TensorRT),
to scaling inference operations (NVIDIA Triton Inference Server.) The deployment can start on one or more GPUs within a server.
As the datasets grow, multi-node scaling can level up for many organizations.
Optimized for VMware vSphere environments, the suite delivers near bare-metal performance across multiple nodes,
streamlining the production of new AI applications and services.

Supermicro NVIDIA-Certified Systems

5© 2021 Copyright Super Micro Computer, Inc. All rights reserved

November 2021

5

Enterprise Support
Using the NVIDIA AI Enterprisesoftware suite, enterprisecustomers get enterprise-grade
support for the entire system,from AI software to thevirtualizationandsystem
hardware, including NVIDIA
data center GPUs and network
accelerators, all tested andoptimized in the Supermicrosystems. As a one-stop shop,
Supermicro offers the entire system, with VMware and NVIDIA AI Enterprise with support.

Virtualized GPU Options
NVIDIA AI Enterprise provides
multiple virtualized GPU options.
By picking the appropriate vGPU
profile, each CPU in the system
can be partitioned into virtualmachines accordingly.
For the NVIDIA A30 Tensor Core
GPUs and NVIDIA A100 Tensor
Core GPUs, there is the choice ofusing Multi-Instance GPU (MIG)
partitioning, which provides
hardware isolation for eachhardware partition relative toother partitions in a single A30 orA100 GPU.

NVIDIA A100
MIG Mode
EnabledNVIDIA A100
MIG Mode
Disabled

NVIDIA A30
MIG Mode
EnabledNVIDIA A30
MIG Mode
Disabled

NVIDIA A40(no MIG)

Max partitions

710 (A100/40GB)
20 (A100/80GB)

46(A30/24GB)

32 (*VDI)(A40/48GB)

Partition TypeSPACESLICED

TIME-SLICEDSPACESLICED

TIME-SLICED

TIME-SLICED

Partition Sizes

Different
sizes, as longas they add
up to 1 GPU

All the same
size per GPU

Different
sizes, as longas they add
up to 1 GPU

All the samesize per GPU

All the same
size per GPU

Largest vGPU

One A100

One A100

One A30

One A30

One A40

Computeresources

Dedicated

SharedDedicated

SharedSharedVirtualized 3D
Graphics Support

No

No

No

NoYes

NVIDIA® NVLink®
SupportNo

Yes (SXM4)No

No (PCIeonly)

No (PCIeonly)

Yes

No

No

HeterogeneousYes

No

Alternatively, the GPUs can be Profiles
partitioned as time-slices using
NVIDIA vGPU included in the NVIDIA AI Enterprise suite, which would allow users to invoke their use by sharing the GPUs withthe VMware hypervisor allocating the GPU slices for use as needed by users in their respective virtual machines. With the NVIDIA
A40 GPUs, customers can also use the GPUs for virtualized graphics applications. One popular use of the NVIDIA A40 is to deploy
them for virtual desktop applications during the day, where users can run high-end 3D graphics applications in the centralized
infrastructure. Then at night, while users no longer use the systems, the NVIDIA A40 GPU systems can be automatically6© 2021 Copyright Super Micro Computer, Inc. All rights reservedNovember 2021

6

redeployed to run AI workloads, such as AI training or inference. Customers have a wide range of choices to optimize the use ofGPUs.

Mixing CPU Cores and GPU Cores
NVIDIA AI Enterprise offers
CPUAI Use Case in a VM Examples
GPU Cores / GPU Memory
GPU TypeCoresflexibility in how to partition
GPU resources among the1 / 10 GPU to full GPU
A100 (40GB)
virtualmachines.
For
Data Scientist AI Development
1–8
1 / 20 GPU to full GPU
A100 (80GB)
example, some applications
AI Training / Inference using preA100 (40GB)
may need more CPU cores to
1–8
1 / 7 GPU (MIG) to full GPU
trainedmodel
A100 (80GB)do work, while GPU cores run
compute-intensive
Data Scientist AI Development
1–81 / 6 GPU to full GPU
A30
operations, such as matrix
AI / Inference Training using premultiplications for AI. The
1–8
1 / 4 GPU (MIG) to full GPU
A30
trained modeltable shows some examplesof the choices that can be
Mixed Workloads (Virtual Desktop
1–8
1 / 32 GPU to full GPU
A40
madefordifferentby Day, Compute at Night)
applications. The Supermicrosystems provide wide options of CPU cores that could be matched to the system with the maximum number of supported GPUs.

Supermicro NVIDIA-Certified Systems, also VMware Certified
Supermicro offers a wide range of systems that are NVIDIA-Certified to support PCI-E Gen 4 based NVIDIA A30, A40, and A100, as
well as the NVIDIA HGX-A100™ 4-GPU systems. Choosing from these optimized Supermicro systems, customers can select thesystems that provide the best performance while optimizing their energy usage and data center cooling.
Supermicro systems range from 1U to 4U support, supporting 1 to 8 GPU, depending on needs. NVIDIA network accelerators,e.g., NVIDIA® ConnectX®-7 SmartNIC are also supported. ConnectX-7 and NVIDIA® Blue-Field® DPUs (Data Processing Units)provide fast, low-latency network connectivity to the Supermicro systems. As a result, enterprises can select the mostappropriate systems with needed power, cooling, and GPU support.
Supermicro ServerRackHeightDepthMax
GPUGPU Type

Network
Acceleration

Disks

SYS-120U-TNR/+

1U29.1”

1

NVIDIA A100, A30, A40

CX6/7, BF2-DPU

12

SYS-220U-TNR/+2U28.2”

2

NVIDIA A100, A30, A40

CX6/7, BF2-DPU

24

SYS-120GQ-TNRT/+1U35.2”

4

NVIDIA A100, A30, A40

CX6/7, BF2-DPU

2

SYS-220GP-TNR/+2U30.18”

4

NVIDIA A100, A30, A40

CX6/7, BF2-DPU

10

AS -2124GQ-NART/+2U32.7”

4

NVIDIA HGX A100-4GPU

CX6/7, BF2-DPU

4

SYS-220GQ-TNAR/+

2U

32.7”

4

NVIDIA HGX A100-4GPU

CX6/7, BF2-CPU

4

SYS-740GP-TNRT

4U/Tower

26.5”

4

NVIDIA A100, A30, A40

CX6/7, BF2-DPU

8

AS-4124GS-TNR

4U29”

8

NVIDIA A100, A30, A40

CX6/7, BF2-DPU

16

Dual CPU
3rd Gen Intel® Xeon®
Scalable
3rd Gen Intel® Xeon®
Scalable
3rd Gen Intel® Xeon®
Scalable
3rd Gen Intel® Xeon®
Scalable
3rd Gen AMD EPYC™
3rd Gen Intel® Xeon®
Scalable
3rd Gen Intel® Xeon®
Scalable
3rd Gen AMD EPYC™

7© 2021 Copyright Super Micro Computer, Inc. All rights reserved

November 2021

7

Conclusion
Supermicro
NVIDIACertified Systems offer
flexible solutions tosupport
NVIDIAAIEnterprise in the VMware
environment, enabling
AI developments anddelivery to run small andlarge AI models for AI
training and inference.
Usingthehighest
performing NVIDIA A100,
A30, and A40 GPUs,
enterprise developers
minimize their valuable
time to run experiments
and development ontheirAImodels,delivering fast and cost effective AI features into new and existing products and services. In addition, by selecting theappropriate Supermicro systems that provide the best performance, I/O, power, and cooling, from the many choices
Supermicro offer, IT managers can deliver the best AI capable additions to their VMware infrastructure.
Supermicro offers these as integrated solutions, including systems, software, and support. Please call your Supermicro
representative for more information.

References
1.

NVIDIA AI Enterprise2.

NVIDIA-Certified Systems

3.

NVIDIA Virtual GPU Software Documentation

4.

Supermicro GPU Servers supporting the Ampere architecture

5.

Supermicro Ultra Servers© 2021 Copyright Super Micro Computer, Inc. All rights reserved. Supermicro, the Supermicro logo. Building Block Solutions. We Keep IT Green, SuperServer, Twin, BigTwin, TwinPro, TwinPro 2,SuperDoctor are trademarks and/or registered trademarks of Super Micro Computer, Inc. All other product names, logos, and brands are property of their respective owners.

8© 2021 Copyright Super Micro Computer, Inc. All rights reserved

November 2021

8