PRODUCT BRIEF

Executive Summary
TABLE OF CONTENTS

Artificial Intelligence(AI) affects many aspects of daily life, whether
we know it or not. This advancement of AI technologies, both in
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
hardware and software, continues to evolve, with new and more
Supermicro Product Lines for AI and HPC . . . . . . . . . . . . . . . . . . . . . . . . 2
enabling products being released constantly. Underlying andNVIDIA H100 Highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
supporting these new AI applications is the sophisticatedaccelerated compute infrastructure that powers systems, storage,Workload Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
and networking creating new user experiences in every part of theAdditional Information and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . 6market. Central to the infrastructure is the latest generation ofGPUs (the hardware at the core of many AI solutions) which
advance what is possible for AI implementations and high-performance computing applications. For example, with thearchitecture of the latest GPUs, multiple numeric formats are available for specific types of calculations that can be executed inparallel for a tremendous performance increase.
................................................................

The
7 NVIDIA H100 Tensor Cores GPUs are a significant step forward in the evolution of GPUs. With the performance of up to 30
times1 over the current NVIDIA A100 GPU, innovative AI applications that were previously not possible will now function© 2022 Copyright Super Micro Computer, Inc. All rights reserved

September 20221

perfectly, reducing AI training times significantly. In addition, HPC applications are anticipated to run up to 7X compared toprevious generations using the NVIDIA A100 GPUs.

Supermicro Product Lines for AI and HPC Based on NVIDIA H100
Supermicro has a long history of providing the most advanced systems for accelerated compute infrastructure and HPCapplications. In addition to incorporating the latest GPU technologies from NVIDIA, Supermicro offers GPUs in a wide range ofdesigns optimized for specific workloads. By incorporating GPUs from the edge units to the very powerful data center GPU
servers, applications can be placed closer to where data is generated, reducing latency and optimizing AI training andinferencing applications.
The Supermicro GPU Family of Servers, workstations, and edge units are designed and optimized to address engineering
professionals requiring the latest CPU and GPU technologies. These no-compromise systems are designed for the latest CPUsand GPUs, with options for liquid cooling where needed. While many Supermicro servers can house a single or dual GPU per
node, Supermicro’s NVIDIA-certified GPU servers are designed for up to ten GPUs. These servers require a detailed engineering
effort to ensure that the proper amount of cooling is designed into the system to handle the high-end thermal demands of GPUs.
Supermicro is offering the following servers, containing the NVIDIA H100 GPU and containing the latest processors from Intel orAMD.
•4U 10GPU systems - SYUS-420GP-TNR and SYS-420GP-TNR2 with dualprocessors. The 10 GPU server is ideal for AI Training, Large Scale Metaverse
implementations, and High-Performance Computing applications. The NVIDIA
H100 GPUs are connected to the CPUs via the latest generation PCI-E bus.

•4U Workstation – SYS-740GP-TNR – is ideal for professional workstations users
in the areas of Scientific Visualization, High-Performance Computing, Rendering and AI
Training, and Deep Learning

Comparison of Supermicro Systems initially available with the NVIDIA H100 GPU

Supermicro Server or WS
4U 10GPU4U 4GPU
Workstation - SYS-740GP-TNR

Max Memory
2 TB
2 TB
1 TB© 2022 Copyright Super Micro Computer, Inc. All rights reserved

Max Number H100 GPUs (PCI-E)
1044

September 20222NVIDIA H100 Highlights
The NVIDIA H100 is the latest GPU in a long line of optimized GPUs from NVIDIA. The NVIDIA H100 Tensor Core GPU, powered by
the Hopper architecture, delivers a leap forward in accelerating AI and HPC workloads. In addition, the H100 secures data
through its NVIDIA Confidential Computing capability. As a result, application developers can distribute and deploy theirproprietary AI models at scale on a shared or remote infrastructure. The NVIDIA H100 specifically uses these advancedtechnologies:•80 GB of HBM3 memory, including a 50 MB of L2 cache – more data can be acted on. To fully utilize that computeperformance, H100 contains HBM3 memory with a class-leading three terabytes per second (TB/sec) of memory
bandwidth, a 50 percent increase over the previous generation. The combination of this faster HBM memory and larger
cache provides the capacity to accelerate the most computationally intensive AI models.

•PCI-E-5.0 – Faster communication with the CPUs - H100 is the first GPU to support PCIe Gen5, providing the highestspeeds possible at 128GB/s (bi-directional). This fast communication enables optimal connectivity with the highest
performing CPUs and NVIDIA ConnectX-7 SmartNICs and BlueField-3 DPUs, allowing up to 400Gb/s Ethernet or NDR
400Gb/s InfiniBand networking acceleration for secure HPC and AI workloads.

•Up to 256 GPUs can be linked together with the NVIDIA NVLink Switch System. This technology accelerates everythingfrom exascale scale workloads with a dedicated Transformer Engine for trillion parameter language models to rightsized Multi-Instance GPU (MIG) partitions.

From NVIDIA, in describing the NVIDIA H100.
“An Order-of-Magnitude Leap for Accelerated Computing The NVIDIA H100 Tensor
Core GPU delivers unprecedented performance, scalability, and security for everyworkload. With NVIDIA® NVLink® Switch System, up to 256 H100 GPUs can beconnected to accelerate exascale workloads, while the dedicated Transformer Engine
supports trillion parameter language models. H100 uses breakthrough innovations inthe NVIDIA Hopper™ architecture to deliver industry-leading conversational AI,
speeding up large language models by 30X over the previous generation.”© 2022 Copyright Super Micro Computer, Inc. All rights reserved

September 2022

3The following table shows the raw performance of the NVIDIA H100 GPU:
Form Factor
FP64
FP64 Tensor CoreFP32
TF32 Tensor Core
BFLOAT16 Tensor Core
FP16 Tensor Core
FP8 Tensor Core
INT8 Tensor Core
GPU Memory
GPU Memory BandwidthDecoders
Max Thermal Design Power (TDP)
Multi-Instance GPUs
Form Factor
Interconnect
Server Options

H100 PCI-E
24 Teraflops
48 Teraflops
48 Teraflops
800 Teraflops *1,600 Teraflops *1,600 Teraflops *2,200 Teraflops *3,200 Teraflops *
80GB
2TB/s
7 NVDEC , 7 JPEG
350W
Up to 7 at 10GB/sec
PCI-E Dual-slot Air-Cooled
NVLINK: 600GB/s PCI-E 5.0 128GB/s
NVIDIA-Certifed with 1-8 GPUs

* Shown with sparsity. Specifications are one-half lower without sparsity.
From a performance view, the H100 is significantly faster than the NVIDIA A100 in several tasks andapplications. The following charts (courtesy of NVIDIA) show how much faster the NVIDIA H100 is,compared to the NVIDIA A100.

Figure 1 - AI Training on Various Workloads (H100 vs. A100) – Benchmarks courtesy of NVIDIA© 2022 Copyright Super Micro Computer, Inc. All rights reserved

September 20224Workload Matching
Several workloads will be optimized with different products from Supermicro. Below is a generalmatching of AI and HPC workloads that are optimized with the NVIDIA H100 GPU.
SupermicroServer/Workstation
4U 10GPU (PCI-E)
4U 4 GPU (PCI-E)
Workstation - SYS-740GPTNRAI Training
Large
ModelsMediumModelsMediumModelsAI Inferencing

3D Metaverse
Collaboration
Large Models

Large Models

Medium Models

Medium Models

Medium Models

HPC
Large
Simulations
MediumSimulations
MediumSimulations

Additional Informationwww.supermicro.com/en/accelerators/nvidia
https://www.supermicro.com/en/products/system/gpu/4u/sys-740gp-tnrt
https://developer.nvidia.com/blog/nvidia-hopper-architecture-in-depth/?ncid=so-link-236335-vt25#cid=gtcs22_so-link_enus

References
1 – According to NVIDIA estimates as of 9/1/2022, https://resources.nvidia.com/en-us-tensor-core/nvidia-h100-datasheet

SUPERMICROAs a global leader in high performance, high efficiency server
technology and innovation, we develop and provide end-toend green computing solutions to the data center, cloudcomputing, enterprise IT, big data, HPC, and embedded
markets. Our Building Block Solutions® approach allows us toprovide a broad range of SKUs, and enables us to build anddeliver application-optimized solutions based upon yourrequirements.

© 2022 Copyright Super Micro Computer, Inc. All rights reserved

September 20225