Accelerate Everything

Supermicro Large Scale AI Training

Large Language Models, Generative AI Training, Autonomous Driving, Robotics
Large-Scale AI training demands cutting-edge technologies to maximize parallel computing power of GPUs to handle billions if
not trillions of AI model parameters to be trained with massive datasets that are exponentially growing. Leverage NVIDIA’s HGX™
H100 SXM 8-GPU/4-GPU and the fastest NVLink™ & NVSwitch™ GPU-GPU interconnects with up to 900GB/s bandwidth, and fastest
1:1 networking to each GPU for node clustering, these systems are optimized to train large language models from scratch in theshortest amount of time. Completing the stack with all-flash NVMe for a faster AI data pipeline, we provide fully integrated racks
with liquid cooling options to ensure fast deployment and a smooth AI training experience.

Systems
AI Rack Solutions

HGX H100 Systems

Multi-Architecture Flexibility with Future-Proof Open-Standards-Based Design for
POD, and SuperPOD with Liquid CoolingMulti-Architecture Flexibility with Future-Proof Open-Standards-Based Design

Extra Large Workload:
Liquid Cooled AI Rack Solutions

Large Workload:
8U 8-GPU System• NVIDIA HGX H100 SXM 8-GPU• Up to 80 kW/Rack• NVIDIA HGX H100 SXM 8-GPU• 16 U.2 NVMe Drives• 8 PCIe 5.0 x16 networking slots

SYS-821GE-TNHR /AS -8125GS-TNHR

Medium Workload:
4U 4-GPU• NVIDIA HGX H100 SXM 4-GPU• 6 U.2 NVMe Drives• 8 PCIe 5.0 x16 networking slots

Petabyte Scale Storage
High Throughput and High-Capacity Storage for AI Data Pipeline

Recommended NVIDIA GPUs

Petabyte Scale NVMe Flash:

1U 24-Bay E1.S
SSG-121E-NES24R

1U 16-Bay E3.S
SSG-121E-NE316R /ASG-1115SNE316R

2U 24/32-Bay E3.S
SSG-221E-NE324R /ASG-2115S-NE332R

Petabyte Scale HDD:

4U 60/90-BayTop-Loading

HGX H100 SXM5 4-GPU or 8-GPU• H100 SXM5 board with 4-GPU or 8-GPU• NVLink & NVSwitch Fabric• PCIe 5.0• 700W per GPU• 80GB HBM3 per GPU

SSG-640SP-E1CR60 /SSG-640SP-E1CR90© 2023 Copyright Super Micro Computer, Inc. All rights reserved

SYS-421GU-TNXR

September 2023

Supermicro Large Scale AI Training

Accelerate Large Scale AI Training Workloads

Large Language Models, Generative AI Training, Autonomous Driving, Robotics
Opportunities and Challenges:• Pool of 10,000+ GPUs and GPU memory to fit large AI models to maximize parallel computing and minimize training time
• Training with massive amount of data with continuous growth of data size (e.g. over 1 trillion tokens)• Serve AI models (inference) to millions of concurrent users• High performance everything: GPUs, memory, storage, and network fabric
Key Technologies:• NVIDIA HGX H100 SXM 8-GPU/4-GPU with 900GB/s NVLink interconnect• Dedicated, lots of high performance, high bandwidth GPU memory - HBM3, HBM2e• 400GbE networking (Ethernet or InfiniBand), PCIe 5.0 storage for fast AI data pipe
• NVIDIA GPUDirect RDMA and Storage to keep feeding data to GPUs with minimum latency• Liquid cooling for GPUs and CPUs
Solution Stack:• DL Frameworks: TensorFlow, PyTorch• Transformers: BERT, GPT, Vision Transformer• NVIDIA AI Enterprise Frameworks (NVIDIA Nemo, Metropolis, Riva, Morpheus, Merlin)• NVIDIA Base Command (infrastructure software libraries, workload orchestration, cluster management)• High performance storage (NVMe) for training cache• Scale-out storage for raw data (data lake)
Use Cases:• Large Language Models (LLMs)• Autonomous Driving Training• Recommender Systems

GPU Acceleration for Complete Range of Workloads
Accelerate Everything

Accelerate Everything

Accelerate Everything

Accelerate Everything

Accelerate Everything

Accelerate Everything

Supermicro Enterprise AI Inference &Supermicro
Training
Supermicro Content Delivery & Virtualization
Visualization & Design
Supermicro AI Edge

Supermicro HPC/AI
Supermicro Large Scale AI Training

Generative AI Inference, AI-enabled Services/Applications, Chatbots,Simulation, Scientific Research, Genomic Sequencing, Drug
Discovery
Large Language Models, Generative AI Training, AutonomousEngineering
Driving, Robotics
Recommender
System, Business Automation

Real-Time Collaboration, 3D Design, Game Development

Content Delivery Networks (CDNs), Transcoding, Compression, Cloud Gaming/Streaming

Edge Video Transcoding, Edge Inference, Edge Training

Video delivery workloads continue to make up a significant portion of current Internet traffic today. As streaming service providers

increasinglyoffertransforming
content in 4K and even 8K, or cloud gaming in a higher refresh rate, GPU acceleration
withindustries,
media engines
isThe augmenting
rise of generative
AI has been recognized as the next frontier for various industries, from tech toIncreasedbanking andmedia.of
The
Acceleratingtime
discovery
for ifscientists, researchers, and engineers, more and more HPC workloads aremachine
fidelity
3Dracegraphics and AI-enabled applications by modern GPUs is accelerating industrial
digitization,
Acrossbusinesses
whose employees and customers engage at edge locations - in cities, factories, retail stores,
Large-Scale AI training demands cutting-edge technologies to maximize parallel computing powerof GPUs
to tohandlebillions
a must to enable multi-fold throughput performance for streaming pipelines while reducing the amount of data required withto adopt AI has begun as a source to breed innovation, significantly boost productivity, streamline operations, make data-driven
product development and design processes, manufacturing, and content creation with true-to-realitybetter
3D simulations
to achieve
hospitals, and many more - are increasingly investing in deploying AI at the edge. By processing data and utilizing AI and ML
algorithms
GPU accelerated
parallel computing to achieve faster results. Many of the world’s
fastest supercomputing
not trillions of AI model parameters to be trained with massive datasets that are exponentiallylearning
growing.
Leverageand
NVIDIA’s
HGX
visual fidelity, thanks to the latest technologies such as AV1 encoding and decoding.
decisions,and improve customer experience.
new heights of quality, infinite iterations at no opportunity costs, and faster time-to-market.
algorithms at the edge, businesses overcome bandwidth and latency limitations, enabling real-time analytics for timely decision
are now
taking advantage
of GPUs and the power of AI.
H100 SXM 8-GPU/4-GPU and the fastest NVlink® & NVSwitch® GPU-GPU interconnects with upclusters
to 900GB/sbandwidth,and fastest
Supermicro’s multi-node and multi-GPU systems, such as 2U 4-Node BigTwin system meet the stringent
requirements
of modern
Whether it is AI-assisted applications and business models, intelligent human-like chatbots for customer service, or AI to comaking,predictive
care and personalized services, and streamlined business operations.
1:1 networking to each GPU for node clustering, these systems are optimized to train large language
modelstypicallyfrom scratch
in thevideofully
delivery,each node supporting NVIDIA L4 GPU with the ability to feature plenty of PCIe Gen5 storage and networking speed
Build virtual
production
infrastructure at scale to accelerate industrial digitalization through Supermicro’s
integrated
pilot code
generation andcontent creation, enterprises can leverage open frameworks, libraries, pre-trained
AI models,and
HPC workloads
requiredata-intensive simulations and analytics with massive datasets and precision
requirements.
GPUsto drive
the Omniverse
demanding data pipeline for content delivery networks.
finetune them for unique use cases with their own dataset. As the enterprise adopts AI infrastructure,
Supermicro’svariety ofGPU
solutions,
includingthe4U/5U 8-10 GPU systems, an NVIDIA OVX™ reference architecture, optimized for
NVIDIA
Purpose-built, environment-optimized Supermicro AI Edge servers with various compact form factors deliver the performance
shortest amount of time. Completing the stack with all-flash NVMe for a faster AI data pipeline,
weas
providefully
integrated
such
NVIDIA’s
H100
provide racks
unprecedented double-precision performance, delivering 60 teraflops peroptimized
GPU, andSupermicro’s
systems provide open modular architecture, vendor flexibility, and easy deployment andEnterprise
upgrade paths
forUniversalrapidlywith
Scene Description (USD) connectors, and NVIDIA-certified rackmount servers and multi-GPU
needed for low-latency, open architecture with pre-integrated components, diverse hardware and software stack compatibility,with liquid cooling options to ensure fast deployment and a smooth AI training experience. highly flexible HPC platforms allow high GPU counts and CPU counts in a variety of dense form factors withrack technologies.
scale integration andevolving
workstations.
and privacy and security feature set required for complex edge deployments out of the box.
Systems
Recommended NVIDIA GPUs
liquid cooling.
Systems
Recommended NVIDIA GPUs.
BigTwin®

Systems

Systems

Recommended NVIDIA GPUs

HGX H100 Systems
Petabyte Scale Storage

AI Rack Solutions
Multi-Architecture Flexibility with Future-Proof Open-Standards-Based Design for
POD, and SuperPOD with Liquid Cooling

Designedfor Largest AI-fused HPC Clusters
High Throughput and High-Capacity Storage for AI Data
Pipeline

Extra Large Workload:
Liquid Cooled AI Rack Solutions

Petabyte Scale NVMe Flash:Large Workload:
4U/5U 4-GPU or 8U 8-GPU System• NVIDIA HGX H100 SXM 8-GPU• Up to 80 kW/Rack
1U 24-Bay E1.S
SSG-121E-NES24R• NVIDIA HGX H100 SXM 8-GPU or4-GPU•• 10 U.2 NVMe Drives
10 PCIeE3.S
5.0 x16 networkingslots E3.S1U• 16-Bay
2U 32-Bay
SSG-121E-NE316R /SSG-221E-NE332R /ASG-1115SNE316R
ASG-2115S-NE332R

Large Workload:
8U SuperBlade®

HGX H100 Systems
Multi-Architecture Flexibility with Future-Proof Open-Standards-Based Design

HighestDensity
Architecture for HPC, AI and Cloud Applications• 2 FHFL
H100
GPU Multi-Nodewith NVLinkBridge
Large Workload:• PCIe 5.0
6U SuperBlade®
• 400W per GPU• 2 NVIDIAH100• 94GB
HBM3perPCIeGPU

SBI-611E-5T2N

Medium Workload:
2U MGX System

H100 PCIe

4U/5U 8 or 10 GPU PCIe - Maximum Performance and Flexibility• FHFL• PCIe 5.0 x16• 350W• 80GB HBM2e• Up to 10 H100 PCIe
• 8 NVMe + 8 SATA drives• 4-5 PCIe 5.0 x16 networking slots• 2 U.2 NVMe Drives• 3 M.2 NVMe Drives• 2 E1.S Drives• 2x25GbE LOM(4x faster than PCIe)• PCIe 5.0• 400W per GPU• 94GB HBM3 per GPUL40• FHFL DW• PCIe 4.0 x16MediumWorkload:• 300W• 48GBGDDR62UHyper

16 DIMMs• FHFL
DW DDR5-4800
SYS-221GE-NR• PCIe 4.0 x16• 350W
2U Grace MGX System• 48GB GDDR6Modular Building Block Platform with Energy-efficient Grace CPU Superchip• 4 NVIDIA H100 PCIe, NVL, or
L40S 8 E1.S + 2 M.2 drives
960GB LPDDR5X• ARS-221GL-NR

SYS-421GU-TNXR /SYS-521GU-TNXR(Codenamed: RedStone-Next)© 2023 Copyright Super Micro Computer, Inc. All rights reserved

1U Grace Hopper MGX Systems© 2023 Copyright Super Micro Computer, Inc. All rights reserved

CPU+GPU Coherent Memory System for AI and HPC Applications• H100 SXM5 board with 4-GPU or 8-GPU• NVLink & NVSwitch FabricMedium Workload:• PCIe 5.0
1U Grace Hopper MGX System• 700W per GPU• 1 NVIDIA Grace Hopper SuperChip• 80GB HBM3 per GPU(ARM CPU and H100 with 96GB HBM3)• 8 E1.S + 2 M.2 drives(Codenamed: CG1)
• 480GB LPDDR5X
ARS-111GL-NHR• 200G HDR InfiniBand

August 2023

© 2023 Copyright Super Micro Computer, Inc. All rights reserved

August 2023

Compact Systems for the Intelligent Edge

Medium Workload:
Short-Depth Multi-GPU EdgeServer• 3 NVIDIA H100 PCIe
• 6 NVMe drives• 32 DIMMs DDR5-4800

Large Workload:
Compact System
SYS-221HE-FTNR• FHFL DW• PCIe 4.0 x16• 300W• 48GB GDDR6
SYS-521C-NR / AS -2015CS-TNR• 1U Compact Edge/5G Server• 2 NVIDIA L4 2 Internal Drive Bays
8 DIMMs DDR5-4800

SYS-111E-FWTR

Hyper-E
High Performance and Flexibility at the Edge

• 3 NVIDIA L40 PCIe
• 6 NVMe drives• 32 DIMMs DDR5-4800

4-GPU Rackmount/Full Tower

AI Workstations:
5U Full-Tower Workstation

L4CloudDC• Powerful expandable Server for theEdge• 3 NVIDIA L4• 8 DIMM slots DDR4-3200• 4 SATA Drives

August 2023

• Ultra-compact Fanless Edge Server
CPU (or ASIC) based Inference• Up to 64GB DDR5
• M.2 M/B/E-Key with Nano SIM CardSlot

SYS-221HE-FTNR / SYS-221HE-FTNRD

L4SYS-741GE-TNRT

Graphic Workstations:
5U Full-Tower Workstation© 2023 Copyright Super Micro Computer, Inc. All rights reserved• 4 NVIDIA RTX A6000 or
3 RTX 6000 ADA• AMD Ryzen™ Threadripper™ PRO• 8 DIMM Slots DDR4-3200

August 2023

• FHFL DW• PCIe 4.0 x16• 300W• 48GB GDDR6AS -5014A-TT© 2023 Copyright Super Micro Computer, Inc. All rights reserved

August 2023

© 2023 Copyright Super Micro Computer, Inc. All rights reserved© 2023 Copyright Super Micro Computer, Inc. Specifications subject to change without notice. All other brands and names are the property of
their respective owners. All logos, brand names, campaign statements and product images contained herein are copyrighted and may not be
reprinted and/or reproduced, in whole or in part, without express written permission by Supermicro Corporate Marketing.

SYS-E403-12P

Small Workload:
Embedded System

Recommended NVIDIA GPUsMedium Workload:
2U Hyper-E

Workstations(Codenamed: C2)
ARS-221GL-NR

Go to www.supermicro.com/ai or scan the QR code todownload the AI Workload Solution Brochure:
HGX H100 SXM5 4-GPU or 8-GPU• FHFL DW• PCIe 4.0 x16• 300W• 48GB GDDR6Extra Large Workload:
2U Hyper-E• 4 NVIDIA L40 PCIe
• Dual 4th Gen Intel® Xeon® Scalable• 16 DIMM slots DDR5-4800

Medium Workload:
2U Grace MGX System
SYS-421GE-TNRT / SYS-521GE-TNRT /AS -4125GS-TNRT

L40

SYS-221H-TNR /AS -2115HS-TNR

SYS-221BT-HNTR / SYS-621BT-HNTR• FHFL DW• PCIe 4.0 x16• 300W• 48GB GDDR6All-in-one Platform for Cloud Data Centers• 2 NVIDIA L40 PCIe or4 NVIDIA L4 PCIe
• 12 3.5” SATA drives• 16 DIMMs DDR5-4800

Fanless and Wallmount Edge

Compute and AI Performance at the Edge

Medium Workload:
2U CloudDC

Flagship Performance Rackmount System Designed for Ulimate Flexibility• 4 NVIDIA L40 PCIe
• 8 NVMe drives• 32 DIMMs DDR5-4800• FHFL DW• PCIe 4.0 x16• 300W• 48GB GDDR6• FHFL DW• PCIe 4.0 x16• 350W• 48GB GDDR62U Hyper Systems

L40S• FHFL DW• PCIe 4.0 x16• 350W• 48GB GDDR6L4• 4 NVIDIA H100 PCIe or
L40S
NVL 8 E1.S + 2 M.2 drives

L40S• PCIe 5.0 x16• 350W• 80GB HBM2e

SYS-421GE-TNRT / AS -4125GS-TNRT /SYS-521GE-TNRT

2U MGX System

10 GPU Systems

Short-Depth 5G/Edge & Hyper E
L40• 1 NVIDIA L4 PCIe per node
• 6 2.5” NVMe drives per node
• 32 DIMMs DDR5-4800 per node

• 8 NVIDIA L40S/L40 PCIe
H100 PCIE• 3 NVIDIA ConnectX-7• 2 FHFL H100 GPU with NVLinkBridge • • 16FHFL
U.2DW
NVMe drives

Modular Building Block Platform Supporting Today’s and future GPUs, CPUs, and DPUs

Medium Workload:
4U 10-GPU

SYS-821GE-TNHR /AS -8125GS-TNHR(Codenamed: Delta-Next)• NVIDIA HGX H100 SXM 4-GPU• 10 U.2 NVMe Drives• 10 PCIe 5.0 x16 networking slots

H100 NVL

SBI-411E-1G/5G• NVIDIA HGX H100 SXM 8-GPU• 16 U.2 NVMe Drives• 8 PCIe 5.0 x16 networking slots

6U SuperBlade®

H100 SXM5• 4 or 8 H100 GPU Board• NVLink & NVSwitch Fabric• PCIe 5.0• 700W per GPU• 80GB HBM3 per GPU• Up to 2 (double-wide blade) or1 (single-wide blade)
H100 PCIe per blade• 2 M.2 NVMe Drives• 2 E1.S Drives• 200G HDR InfiniBand

Recommended NVIDIA GPUs

Large Workload:
8U 8-GPU System

Medium Workload:
4U/5U 4-GPU

SYS-421GE-TNRT / AS -4125GS-TNRT /SYS-521GE-TNRT(Codename: Delta-Next andRedstone Next)

8U SuperBlade®

Systems

Large Workload:
BigTwin® 2U 4-NodeHighest Performance, Tailored for NVIDIA Omniverse

Large Workload:
4U/5U 8 GPU (PCIe)
H100 NVL

Petabyte Scale HDD:

4U 60/90-Bay
Top-LoadingAward Winning Multi-Node System with Resource Saving Architecture
Recommended NVIDIA GPUs

Omniverse Optimized Systems• 8 NVIDIA H100 NVL or 10 H100 PCIe
• 8 NVMe and 8 SATA Drives• 32 DIMMs DDR5-4800

Highest Density Multi-Node Architecture for HPC, AI and Cloud Applications

SSG-640SP-E1CR60 /SSG-640SPE1CR90

Systems

10 GPU Systems

4U/5U 8 or 10 GPU PCIe — Highly Flexible Architecture

Extra Large Workload:
8 -10 GPU (PCIe)

August 2023

SYS-E100-13AD