Supermicro Liquid Cooling solution
Sr. Principal Solution Architect
Junichi Tashiro
11/16/2021

Better Faster Greener™ © 2020 SupermicroTogether We Are Strong$3,500

Revenue (in millions)$3,000$2,500$2,000$1,50025 Years of Non-stop
Revenue Growth andProfitability$1,000$500

Silicon Valley HQ, USA$FY94 FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 FY13 FY14 Fy15 FY16 FY17 FY18Revenue$3.6B (2020)

WorldwidePresence

Silicon Valley (HQ), Taiwan, Netherlands,Japan, China

Human Resource

4000+ Worldwide, 50% R&D staff

Corporate Growth#18 Fastest Growing Company (Fortune 2016), #1
Fastest Growing IT Infrastructure Company

Product Volume

3rd Largest Server System Provider Worldwide (IDC2018), ~1.2M units annually2Our Progression
Presence

EMEA

Domestic

Asia Pacific

3.0
2.0

1.0

Verticals:
CSPs, OEMs

Targets:
Enterprise
Brand Recognition

MarketChannel:Distributors/VARs

Innovation

Subsystems

Server Systems

Total Solutions

Motherboards

BigTwin, NVMe,
Storage, Ultra,
SuperBlade, GPUSolutions

Management Software
Global Service
Resource-Saving Solutions
RoT, PFR and….
New Business ModelChassis andserver building
blocks3

Cloud & Edge Infrastructure Building Blocks
Blade
MicroBlade™

Form Factor Optimized

SuperBlade®

Rack-mount

1U 20 Bay

SuperStorage

Top-Loading
Scale Storage

DCO
WIO

GPU SuperBlade®

All-Flash NVMe

Max I/O

JBOF
NF1/EDSFF

Multi-node
FatTwin™

GPU SuperServers

4U 20 single-width
GPUs

TwinPro2®

1U 4GPU

MicroCloud

BigTwin™

2U 6GPU

10U 16GPU

Edge ComputingEnterprise

Cloud / Web

Storage / HCIAI / ML / HPC

Workload Optimized
4

Intel Data Center Use caseConfiguration
SBI-6119P/ 6219P with SBE610J
CPU: SKL 6136 12C 3.0G

Memory: 12 x 32GB DDR42666
Storage: 2 x 2.5"1.2TB SAS
Network: 10G switch moduleIntel Data Center Strategy: Leading Intel’s Business Transformation
Intel’s critical business functions—Design, Office, Manufacturing, and Enterprise (DOME)—while operating our data
centers as efficiently as possible, Intel IT has engaged in a multiyear evolution of our data center strategy. Sepermicro& Intel IT has developed a disaggregated server architecture. The architecture separates the CPU/DRAM module andthe NIC/Drives module on the motherboard.
the disaggregated design offers the following benefits:• No need to replace perfectly good components.
• No need to reinstall the OS.
• Cuts refresh costs by a minimum of 44%.
• Reduces technician time spent on refresh by 77%.
• Decreases refresh materials’ shipping weight by 82%.

White Paper
https://www.intel.com/content/dam/www/centrallibraries/us/en/documents/intel-it-green-computing-at-scale-paper.pdf

Liquid Cooling Solution

What’s Liquid Cooling

Better Faster Greener™ © 2021 SupermicroLiquid Cooling Demand from High-Heat CPU & GPU
1. Intel: 3rd Gen Xeon Scalable Processer (Codename: Ice Lake) released in 2Q 2021, can reach up
to 40 Cores, and run at 3.40GHz (Intel® Xeon® Platinum 8380 Processor), TDP up to 270W per socket.
3. AMD: The current TDP is also as high as 280W

2. NVIDIA: Take the latest Ampere Architecture GPU published in May 2020. TDP of A100 is 400W.

NVIDIA AMPERE ARCHITECTURE
11/18/2021

10Better Faster Greener™ © 2020 Supermicro

Data Center Power Usage Effectiveness (PUE) Breakdown

Source: The Green Grid, 2012; Zhao et al., 2014; Shehabi et al., 201612CONFIDENTIAL

Cold Plate Solution cases
CPU2Coolant outCoolant in2U 4 node Twin Pro node
11/18/2021

13

Better Faster Greener™ © 2020 Supermicro1-Phase Immersion Cooling System

Coolant

Coolant

1-Phase Dielectric Coolant

Coolant Selections
ItemCoolant
composition
1 3M FC40 (USA)fluorochemical
2 Submer (Spain)
vegetable oil
3 Mivolt(UK)Ethanol (醇)4 GRC
mineral oil
5 ITRI(TW)mineral oilSuccessful PoC for KDDI project at ITRI (GRC solution)

GRC Oil Tank

15

2-Phase Immersion Cooling System

2-Phase Dielectric Coolant

Air Cool vs. Liquid Cool Power Efficiencyx9
SMC 4U 8 GPUs servers x9(72 GPUs/ Rack)

IT Equipment Energy (W)
Non-IT Facility Energy (W)[Chillers, CRACs, pumps, etc…]
Total Rack-Level Power (W)x9x9
In Row Air Cooling

Cold Plate

Immersion Cooling(pPUE:1.6~1.4)(pPUE:1.4~1.15)(pPUE: <1.1)

1 server1 Rack1 server1 Rack1 server1 Rack

3,600

32,4003,300

29,7003,200

28,800

1 server1 Rack1 server1 Rack1 server1 Rack-27%

4,780
34,480-34%

2,100
30,90014,58046,9801.45
11/18/2021

1.16
17

1.07
Better Faster Greener™ © 2020 SupermicroComparison of Liquid Cooling and Air Cooling
Liquid Cooling

Advantage

Disadvantage

11/18/2021

Air Cooling➢Lower OPEX

➢ Lower CAPEX(OPerating EXpense)(Capital Expenditure)➢Quieter➢Lower PUE (Power-Saving)➢ Easy Deployment➢ Easy to Install➢ Higher CAPEX(Coolant / Water Needed➢ Leakage Concern➢ Higher OPEX
➢ Air Flow Mixing

19

Better Faster Greener™ © 2020 SupermicroWhere is the Demand of Liquid Cooling• Solution for high-density/high-heat device• GPU server, high-density server, …

• Energy saving requirement• Low PUE/Low power consumption• Field with water-cooling tower facility• General industrial plant, LAB, …

• Low-noise demand sites• Clean room, laboratory, office, …
11/18/2021

20Better Faster Greener™ © 2020 SupermicroLiquid Cooling Rack &Configuration

CONFIDENTIAL

Blade Server Advantages• Resource savings• Reusable of management module, chassis enclosure and switch module• Space Savings• Up to 65% rack space saving• Power Savings• Up to 20% energy cost savings• Maintenance savings• Hot swap – blades, PWS, CMM, switch• TCO, TCE savings• Cabling savings with blade-switch• Refresh cost savings

SBI-420P-1T3N/1C2N

SBI-4119SSG/4114S/SG• Reusable enclosure• Operation Resources• Rack and stack• Network engineering• OS configuration engineering
11/18/2021

8U SuperBlade® 820H Enclosure
23

Better Faster Greener™ © 2020 SupermicroCONFIDENTIAL

Liquid Cooling Rack Solution (I)123

Per Rack Including

Rack dimension:
1991mm(H) x 750mm(W) x 1200mm(D)Net weight(including all parts): 1104 kg1•

42U 750mm Wide x 1200mm

2•

InRackCDU•3

Facility Hose Pairs Camlock

Better Faster Greener™ © 2020 Supermicro
11/18/2021

24

CONFIDENTIAL

Liquid Cooling Rack Solution (II)Per Rack Including

45554564•

1U PassThrough Tray (support 10 blades)•51U PassThrough Tray(support 20 blades)•6CDM 42U 76-node

55Hot
LiquidColdLiquid44
CDM: Coolant Distribution Module
25

CONFIDENTIAL

Liquid Cooling Rack Solution (III)Per Rack Including
Blade Server System* 4pcs••••

SBE-820H-822: Enclosure for 20 Blades w/ 8x2200W
SBM-IBS-H4020: HDR 200G InfiniBand Switch
MBM-CMM-FIO-V: CMM
MBM-XEM-002: Blade 10G Ethernet Broadcom switch
with 2xQSFP and 4xSFP+, up to 2pcsPWS-DF005-2F: 12Vdc input redundant FAN module
power *3pcs
AOC BBP card: Blade Battery back up control module••Server Blade *76pcs••SBI-420P: ICX Blade SFT-DCMS and SN on Asset tag forRMACold Plate: 1U Liquid Cooling Module for Blade B12DPT
8U20N
RMA: Return Merchandise Authorization

11/18/2021

26

Better Faster Greener™ © 2020 Supermicro

8U SuperBlade ICX DP Blade with Passive Cooling Loop

2600

8 DDR4 DIMM
slots per socket

Passive Cooling LoopIntel® 4th Xeon® SPAir Cool: 220W
- SNK-P0077V- SNK-P1044V
Liquid Cool: up to
270W w/ PCL

CPU1

CPU1

CPU2

CPU2

M.2 2280

M.2 2280

SATA

PCIe Gen4 x 4

PCIe Gen4 x 4

PCIe
Gen4 x 4

KVM
2.5”

2600
Mezz1

TPM

Mezz125G

C621A

VROC

25G

C621A

SBI-420P-1C2N

TPM

Connector for Mezz AOC
card (IB 200G or ETH 25G)

VROC

SBI-420P-1T3N
Onboard 25G
LOM

PCL or 2.5”
2.5”

SSD

SAS

PCIe
Gen4 x 4

SSD

KVM
2.5”

2.5”

SSD

SSD

SAS or SATA board
KVM board

3rd SATA HDD/SSD
3rd 2.5” HDD/SSD spaceexchanging with liquid coolcomponent

11/18/2021

27Better Faster Greener™ © 2020 SupermicroPerformance, Power, and Cost Comparison
8U 820H Enclosure
CPUMSRP($)
TDP (W)
CoresBase Frequency (GHz)
Cache/core (MB)GFLOPS per CPU
Nodes per enclosure
16G@3200 RDIMM
Power Consumption (W)
GFLOPS per enclosure
GFLOPS/Per 42U rack
Power/Per 42U rack (W)
GFLOPS/Watt

Air Cooling
8352Y
3450
205
32
2.2
1.5
2252.820
320x
13,672
90,112
360,448
54,688
6.59

Liquid Cooling
8368
6302
270
38
2.4
1.5
2918.420
320x
16,922
116,736
466,944
67,688
6.9

1. 90% heat can be recovered by liquid cool.
2. Air condition efficiency: consume 18KW per 40KW powerImprovement

Up 29.55%

Up 23.77%
Up 29.55%

Up 4.7%

CONFIDENTIAL

Blade Server Advantage @ Data Center
REUSE (New Longevity PWS & FAN Design)
Multi-generation infrastructure for up to 65% CAPEX
Savings

REFRESH (45% ~60% save in Hardware Refresh Costs)
Modular upgrades for maximum performance and efficiency
REDUCE (June 2018 White Paper)
Optimized shared resources for up to 50% reduction in Power and Cooling TCO

11/18/2021

Disaggregated ServerArchitecture

Multi-Node Powerand CoolingResourcePooling

Rack ScaleManagement

33

Better Faster Greener™ © 2020 SupermicroThank You

www.supermicro.com

11/18/2021

34

Better Faster Greener™ © 2020 SupermicroDISCLAIMER
Super Micro Computer, Inc. may make changes to specifications and product descriptions at any time, without notice. The
information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions
and typographical errors. Any performance tests and ratings are measured using systems that reflect the approximate
performance of Super Micro Computer, Inc. products as measured by those tests. Any differences in software or hardware
configuration may affect actual performance, and Super Micro Computer, Inc. does not control the design or implementation of
third party benchmarks or websites referenced in this document. The information contained herein is subject to change and may
be rendered inaccurate for many reasons, including but not limited to any changes in product and/or roadmap, component andhardware revision changes, new model and/or product releases, software changes, firmware changes, or the like. Super MicroComputer, Inc. assumes no obligation to update or otherwise correct or revise this information.
SUPER MICRO COMPUTER, INC. MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THECONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THATMAY APPEAR IN THIS INFORMATION.
SUPER MICRO COMPUTER, INC. SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY ORFITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE TO ANYPERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OFANY INFORMATION CONTAINED HEREIN, EVEN IF SUPER MICRO COMPUTER, Inc. IS EXPRESSLY ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION© 2020 Super Micro Computer, Inc. All rights reserved.