Configurable Liquid Immersion Cooling System
Scalable Energy Efficient HPC / Data Center Solution

Introducing ZettaScaler Core Technologies

Value provided by these technologies

Unique liquid immersion cooling technology

High performance

All circuits are fully immersed and cooled by
3MTM FluorinertTM inert liquid circulation
Operating temperature is under 30 oC ( equal to 86oF ).
Safe, no need for sealing, highly-efficient cooling solution.
Easy to maintain with much less failure rate.

Our proprietary many-core MIMD processor, PEZY-SC2
Overwhelming computing power
4.1 TeraFLOPS ( Rpeak ) at 1GHz
2,048 cores of original design and 6 MIPS64® cores
16nm FinFET process
56MB of cache memory
TCI 3D stacked memory with bandwidth of 2.0TB/sec
DDR4 memory with bandwidth of 153.6GB/sec
PCI-Express Gen4 with I/O bandwidth 64GB/sec
Significantly reduced transmission power loss using 48V DC input.

Sufficient interconnect bandwidth
32 processors are connected using twelve hierarchical PCI-Express
fabric switches to form a Brick.
Every link between two processors in a Brick is PCI-Express Gen3 x16.
Bricks are interconnected with high-speed InfiniBand EDR, etc.

Ultra-high-density modules
Extreme densities realized by liquid immersion cooling.
Short-distance signal transmission allows for high-speed circuit operation.
Over one million processing cores in one cubic meter ( 1㎥ ).
Able to select a variety of high density modules, PEZY-SC2 module, All
FLASH-SSD module, Intel® Xeon® module, etc.

One immersion tank with 16 Bricks/512 processors delivers 2.1
PetaFLOPS ( Rpeak ), 1.4 PetaFLOPS ( Rmax )High energy-efficiency
World top class efficiency with over 10GFLOPS/W

Compact and silent
Compact size, silent, no air blow, no temperature nor moisturecontrol required.
Can be installed in a laboratory office and even in a private
researcher office.

Scalable
A system with multiple immersion tanks
Configuration example:
Connecting 16 immersion tanks can create 30 PetaFLOPS ( Rpeak ),20 PetaFLOPS ( Rmax ) system

Configurable
An immersion tank can be configured with a variety of modules
Configuration example:
The system can be configured 256 Xeon® processors/Brick with
2PB of SSD storage/Brick in a tank.

Configurable Liquid Immersion Cooling System
ZettaScaler is an extensible system that adopts ExaScaler's unique liquid immersion cooling technology. In conventional systems,
the equipment along with the entire room has to be cooled through air conditioning, but air conditioning is not necessary for thisliquid immersion cooling system. It is a very small and silent supercomputer, it can be even installed in laboratories or offices.
The basic block of our system is called a brick, mounted with ultra-high density processor modules. These extremely densebricks are installed in the immersion tanks and are efficiently cooled in our immersion cooling system.
The inside of the immersion tank is filled with fluorine inert liquid exhibiting excellent thermal characteristics. By circulating thisliquid, it is possible to efficiently cool all immersed electronic circuits. It provides excellent electrical insulation and it is a highly
safe liquid with non-combustibility, non-toxic, odorless and zero ODP ( Ozone Depletion Potential ) characteristics.
Furthermore, it has a high boiling point and hardly evaporates, so sealing is not necessary. This simplicity of our system structure
along with the ease of use of our cooling system simplifies maintenance.
ZettaScaler is also extremely extensible and scalable. By connecting the immersion tanks in tandem, it is possible to scale to thedesired performance level. For example, with 16 immersion tanks, the system can be configured to reach 30 PetaFLOPS ( Rpeak )/ 20 PetaFLOPS ( Rmax ).

ZettaScaler-2.0 system configuration

Tank

Tank / EPU / Brick / Module Card

（ A prototype image. This prototype has 24 bricks and the final product has 16 bricks. ）16 Bricks

BrickBrick

DC48V

4 EPUsEmbedded Power UnitEPU

DC48V power supply from thebottom of Tank to the Brick

Module CardTankThree phase AC200VCarrier BoardScalable（ Multiple Tanks Interconnected ）32 module cards, 4 network cards and 1 control card ( optional ) are mounted on the carrier board ( backplane ) and areconnected to each other using twelve hierarchical PCI-Express fabric switches interconnect.
Inter-Brick connections are facilitated using high-performance interconnects such as InfiniBand.
All features and specification are subject to change without prior notice.

PEZY-SC2 Module
Our high-performance compute nodes areequipped with PEZY-SC2, the successor to thePEZY- SC many-core processor developed byPEZY Computing K.K.
Its predecessor PEZY-SC is the Green500 Top
1-3 awarded processor used in the systems
Shoubu, Suiren, SuirenBlue and Satsuki.
PEZY-SC2 doubles the number of corescompared to PEZY-SC. Also, significantly
reduces transmission power loss using 48V
DC input.

W160mm, D100mm, H28mm

PEZY-SC2 Processor
・2,048 MIMD cores / 16,384 Thread / 1GHz・L1 12MB / L2 12MB / L3 40MB
・Theoritical performance : 4.1TFLOPS ( DP ) / 8.2TFLOPS ( SP )/ 16.4TFLOPS ( HP ) ( Rpeak )・MIPS64R6 ( P6600 ) 6Cores / L1 I:64KB+D:64KB ( each core ) / L2 2MB・Coming soon :Near Field Wireless Communication
between Processor-Memory link with TCI* Technology*ThruChip Interface

PCIe Gen3/4 x16 * 2CH ( x8 * 4CH )DDR4 64bit ( ECC ) * 4CH / 3,200Mbps
BW=100GB/sec Up to128GB
Power Consumption：130W ( Average at standard load )Die Plot

A Brick with 32 PEZY-SC2 modules
4 slots ( top right ) for 100Gbps
high speed interconnecting cards

Photo courtesy of JAMSTECIntel® Xeon® E5-2600 v3/v4 Module
Dual processor node for high-performance computing and virtual machineservices equipped with two QPI connected Intel® Xeon® processors

A Brick with 16Xeon® E5 modules
32 Xeon® E5 processors

W160mm, D115mm, H60mm

4 slots ( top-right )for 100Gbps
high speed interconnecting cards

Dual Xeon® D Module
One module contains two Xeon®D nodes for high-performancecomputing and storage control
A Brick with 32 Xeon D modules
64 Xeon® D processors

W160mm, D100mm, H28mm

4 slots ( top-right ) for 100Gbps
high speed interconnecting cards

ALL FLASH - High capacity SSD module
Storage module with 16 NVMe M.2-2280 SSDs
32TB with 2TB SSDs

A Brick with 64
ALL FLASH modules
Capacity up to 2PB

W160mm, D100mm, H14mm

4 slots ( top-right ) for 100Gbps
high speed interconnecting cards

High capacity FPGA Module
Compute node with Xilinx’s highest capacity FPGA
Virtex UltraScale XCVU440
A Brick with 32 High

PCIe Gen3 x8* 4CH
DDR4 SO-DIMM 64bit * 1CH / 2,400Mbps
BW=19.2GB/sec , 16GB memory ( max )W160mm, D100mm, H28mm

capacity FPGA modules
4 slots ( top-right ) for 100Gbps
high speed interconnecting cards

The right of all brand names, product names and trademarks belong to their respective owners.

ExaScaler Inc.
Kimura Bldg. 3F, 2-1 Kanda-Ogawamachi, Chiyoda-ku, Tokyo 101-0052, JAPAN

TEL：+81 (0)3-5577-3835 / Mail：info@exascaler.co.jp

http://www.exascaler.co.jp/