1DDN Update for 2019
第十八回PCクラスタシンポジウム

DataDirect Networks Japan, Inc.
2018/12/13

橋爪 信明 (nhashizume@ddn.com)

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

22018年主な出来事＠DDN• Declustered RAID(DCR)対応製品へラインナップ刷新► SFAOS11世代のHWへの切り替え元年► SFA200/400NV, SFA7990, SFA18K• 6月 Intel社よりLustre部門を買収► DDNのWhamcloudディビジョンとして活動► Open Source Community Edition開発► Lustre L3サポート• 9月 Tintri社を買収► DDN内の独立したTintriディビジョンとして活動► 仮想化・マルチクラウド自動最適化フラッシュストレージ

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

2DDN SFA
3ALL-FLASH AND HYBRID BLOCK STORAGE PLATFORMS
2019年より全グレードで新ラインナップを展開

200NV

400NV

7990

14KX

18K

23GB/s1M IOP/s

46GB/s2M IOP/s

23GB/s1M IOP/s

60GB/s4M IOP/s

92GB/s
3.2M IOP/s

24 NVME Slots

24 NVME Slots
Up to 450
SSD/HDD

48 NVMe Slots
Up to 1872
SSD/HDD48 NVMe Slots
Up to 1872
SSD/HDDEDR IB (4), OPA (2)FC32 (8), FC (8)EDR IB (8), OPA (4)EDR IB (4), OPA (2)FC16 (8)EDR IB (12|8)
OPA (4), FC16 (24)EDR IB (16), OPA (8)NEW 2018

COMING 2019

NEW 2018

To be EOS

COMING 2019

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

3

4DDN SFA | 製品名と位置付け
位置付け

用途200NV

400NV

7990

18K

総称

全体をまとめて呼ぶ場合

SFA200NVSFA400NV

SFA7990

SFA18K

ブロックストレージ

ファイルサーバに接続

SFA200NV

SFA400NV

SFA7990

SFA18K

エンベデッドアプライアンス

コントローラ上にファイルサーバを実装

SFA200NVE

SFA400NVE

SFA7990E

SFA18KE

ExaScalerアプライアンス

ExaScaler(Lustre)エンベデッド

ES200NV

ES400NV

ES7990

ES18K

GridScalerアプライアンス

GridScaler(GPFS)エンベデッド

GS200NV

GS400NVGS7990

GS18K

A3Iアプライアンス

AI向けエンベデッド

AI200

AI400

AI7990

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

45

SFA18KDDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

6MAXIMUM SCALABILITY WITH NVMe + SAS
FOR CAPACITY & BANDWIDTH WITH DRIVE TYPE FLEXABILITY

SFA18K

Block 92GB/s
Embedded 78GB/s•Start in as little as 4U and Scale up & Scale out;
Up to 1872 drives per system:o 72 Slots in “head”, 48 NVMe/SAS + 24 SASo Add up to 20x SS9012 Enclosures (90 SAS slots)• Flexible Access for File: IB or Ethernet (16 ports), OPA(8 ports)
Integrates high-performance, low-latency, RDMA capable
networks. EDR InfiniBand and 100Gbps (or 40Gbps)
Ethernet and OPA• Access for Block: IB (16 ports)
Block systems attach via EDR IB to external servers* Cached

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

6

7SFA18K Overview
Broad Connectivity Options
EDR IB
40/100 GbE
OmniPath

Flash and NVMe Enabled in‘head’
Up To 48 NVMe devices
Up to 72 SAS SSD

Real-Time, Multi-CPU RAID Engine
Full Implementation of Real-Time Storage OS,
Interrupt-Free; Massively Parallel I/O Delivery
SystemReACT Cache Management

SFA Embedded
Low-Latency VirtIO data path for DMA IO accessto local VM

Real-Time I/O Data-Aware Cache Optimizer,
Small IOs Go to Cache; Large & Streaming
I/Os Bypass Cache

DeClustered RAID
Storage Fusion FabricHighly-Over-Provisioned Backend RAIDed
Fabric Withstands Failures of Drives,
Enclosures, Cables, etc

8+2 rebuild vs DCR rebuild in112 disk pool and 210 disk pool

High Density Array
Up to 972 drives in 44U rack.
SS9012 enclosure is 90 HDDs / SSDs in4U

Data Integrity
Real-Time Data Integrity Verification for every I/ODDN Storage | © 2018 DDN Storage

0500

1000

DCR 210

DCR 112

Legacy 8+2© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

7

8SFA18K CONFIGURATIONSSFA18K
Only+ 1 SS9012+2 SS9012+4 SS9012+5 SS9012+6 SS9012+8 SS9012+10 SS9012+16 SS9012+20 SS9012

2.5” NVMe or
SAS SSD
Slots

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

72(NVMe:48)

3.5” HDD or
SAS SSD
Slots

090180

360

450

540

720

900

1440

1800

Enclosure
Connection

NADirect

DirectDirectDirectDirectDirectDirectDaisy Chain

Daisy Chain

最大物理容量NA

1.26PB

2.52PB

5.04PB

6.30PB

7.56PB

10.08PB

12.6PB

20.16PB

25.20PB

ConfigSFA (BLOCK)ESGS

Expansion
Options※ 最大物理容量は14TB NL-SAS HDDをSS9012に実装した場合

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

89EXAScaler ES18K Early Performance - IOR 1MB FPP• EXAScaler IOR Client Benchmarking with an ES18K EDR with 408 NL-SASDrives as 8 DeClustered RAID Pools of 51 drives.

70,00060,00050,00040,00030,00020,00010,000
0ES18K IOR(FPP)
Read: 408x NL-SAS, 8 x51/MR=161GB/s
Write

Bandwidth (MB/sec)

Bandwidth (MB/sec)

ES18K IOR(FPP)
Write: 408x NL-SAS, 8 x51/MR=170,000

58GB/sRead

60,00050,00040,00030,00020,00010,0000

Tuning isstill inprogress

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

910SFA7990

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

11SFA7990 WORLD’S MOST EFFICIENT HYBRID STORAGE
OPTIMIZED FOR SSD, HDD CAPACITY, MULTICORE & MULTI GPU

SFA7990

Block 23GB/s
Embedded 20GB/s
DDN’s ES7990 and GS7990, EXAScaler® and GRIDScaler® appliances,offer scalable, high performance parallel file systems in an integrated
package. Designed, deployed and supported by the experts in data intensive
workloads, these appliances are the gold standard for eliminating bottlenecks andmaximizing application performance.

• Start at 4U and Scale Up or Scale Out
Mix Disk and Flash SSD for Hybrid Performance• Flexible Access for File: (IB, Ethernet, OPA)
Integrates high-performance, low-latency, RDMA capable
networks. Both EDR InfiniBand and 100Gbps (or 40Gbps)
Ethernet plus OPA• Flexible Access for Block: (IB, FC)
Block systems attach via FC or IB to external servers* Cached

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

1112SFA7990 OVERVIEW

SFAOS
Mirrored, battery-backed write-back cache
ReACT Cache Management
Fully Declustered Data Placement for rapid rebuilds
Flexible Connectivity
Block Target: 4x EDR IB OR 8x FC16
Embedded File System: 4 x EDR IB/40/100 GbE
OR 2 x OPA

EMBEDDED FILESYSTEM APPLIANCES
EXAScaler®
GRIDScaler®

Full SAS Performance for all
configurations
Balanced SAS Architecture provides
saturation level performance for all
expansion options

High Density HDD and SAS SSD
Redundantly Connected
Enclosures
All configurations feature directly
attached, redundant links toSFA7990

SS9012 Enclosures and SFA7990 house
up to 90 3.5” Devices.

High Performance SAS
Backend Bandwidth supported by12Gbps SAS Links from allenclosures to SFA7990

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

1213SFA7990 CONFIGURATIONS

Configuration

SFA7990 Only

+ 1 SS9012+2 SS9012+4 SS9012

90180

270

450

1.26PB

2.52PB

3.78PB

6.30PB

SFA (BLOCK)SupportedSolutions

ES7990
GS79903.5” HDD or SAS
SSD Slots
最大物理容量ExpansionOptions※ 最大物理容量は14TB NL-SAS HDDをSS9012に実装した場合

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

1314ES7990 WORLD’S MOST EFFICIENT HYBRID STORAGE
OPTIMIZED FOR SSD, HDD CAPACITY, MULTICORE & MULTI GPU

20GB/sES7990ES7990 Sequential IO
Performance

ES7990 Random IO Performance(10xSSD)
IOPS
900780K IOPS@4KB

800

1820000

10400

8300

6200

4100

2004KB

8KB

16KB 32KB 64KB 128KBThroughput (MB/sec)

12500

Throughput (MB/sec)

14600

Read25000

20
16700
IOPS (ops/sec)

Write

B/W

13.7GB/s Write
205MB/s Per Disk

15000

10000

5000

080xNL-SAS

160xNL-SAS

IO Size (Byte)

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

1415GS7990 WORLD’S MOST EFFICIENT HYBRID STORAGE
OPTIMIZED FOR SSD, HDD CAPACITY, MULTICORE & MULTI GPU

GS799020GB/s* CachedDDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

15

16

SFA200/400NV

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

17WORLD’S MOST EFFICIENT HPC FLASHOPTIMIZED FOR NVMe IOPS, MULTICORE & MULTI GPU

SFA200NV* Cached

DDN Storage | © 2018 DDN Storage

Block 23GB/s
Embedded 20GB/s• Start at 2U and Scale-out
All NVMe Flash, high density, extreme
performance• Flexible Access for File: (IB, Ethernet,OPA)
Integrates high-performance, low-latency,
RDMA capable networks. EDR InfiniBand and100Gbps (or 40Gbps) Ethernet and OPA• Flexible Access for Block: (IB, FC)
Block systems attach via FC or IB to external
servers© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

17

18WORLD’S MOST EFFICIENT HPC FLASHOPTIMIZED FOR NVMe IOPS, MULTICORE & MULTI GPU

SFA400NV

Block 46GB/s
Embedded 40GB/s• Start at 2U and Scale-out
All NVMe Flash, high density, extreme performance• Flexible Access for File: (IB, Ethernet, OPA)
Integrates high-performance, low-latency, RDMA capable
networks. EDR InfiniBand and 100Gbps (or 40Gbps)
Ethernet and OPA* Cached

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

1819GS200NV : SCALABLE FILE PLATFORMS

OVER 23GB/s
THROUGHPUT EVEN
FOR RANDOM READ
WORKLOADS• Saturation level random readperformance with small I/Otransfers• Ideal for Analytics platforms
requiring Enterprise Features

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

1920SFA200NV AND SFA400NV | SCALABLE FILE PLATFORMS• Choose your starting option
from as small as 38TB or up to
152TB in just 2 RU
• Scale-out with NVMe toPetabytes of Flash and TB/s ofperformance

1400

Raw Capacity (TB)

Start with a complete All
NVMe Flash, Parallel File
System from DDN in just
2RU

SFA200NV and SFA400NV Capacities
1200
1000
800
600
400
2000

SM

L2L

4L

8L

38TB

76TB

152TB

304TB

608TB

1.2PB

Coming soon
15.36TB NVMe

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

20

21

A3I
Accelerated, Any-scale AI Solutions From DDN

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

22DDN PARALLEL FILE STORAGE APPLIANCES FOR AI AND DL

AI200

AI7990All NVME parallel file storage appliance
Optimized for the most intensive workloads
30TB, 60TB, 120TB in 2RU

Hybrid parallel file storage appliance
Optimized for capacity, intermix with flash
1PB in 4RU

23GB/s, 395K IOPS • 100Gb Ethernet or EDR InfiniBand • Start with 1, Scale out and mix

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

22

23

®

DDN| ©2018
Networks,Inc.
DDNStorage
Storage| ©DataDirect
2018 DDNStorage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.24®

DDN| ©2018Networks,Inc.
DDNStorageStorage| ©DataDirect
2018 DDNStorage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

DDN Storage | © 2018 DDN Storage

A3I with DGX-1 POD

A3I with quad DGX-1

A3I with single DGX-1

25DDN A3I WITH DGX-1 SCALABLE REFERENCE ARCHITECTURES© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

25

26 DDN A3I Solutions: Performance to a single container

Over 100K IOPs and10GB/s to 1 Container
EXAScaler client demonstrates over 10GB
per second to a single container and over
100K IOPs.
Typical Deep Learning Codes will perform
IOs around 128K IOs – and see around
6GB/s with this pattern.

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

2627DDN A3I Solutions: Fully-optimized with widely-used DL frameworks

60% Faster Consistent, Linear AI
Performance
AI benchmarking of AI Frameworks withpopular test sets demonstrates images per
second rates 60% higher than competingsystems

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

27

28

AI Benchmarking: DDN enables and accelerates

Images PerSecond

CAFFE GOOGLENET TRAINING THROUGHPUT
2.4X MORE IMAGES WITH DDN STORAGE
4344
17960500

1000

1500

2000

DDN Storage

2500

3000

3500

40004500

5000

Other Storage Products

Time (inminutes)

CAFFE GOOGLENET TRAINING TIME
2X FASTER WITH DDN STORAGE
197
379050100150

DDN Storage

200250300350400

Other Storage Products

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

28

29

AI Benchmarking: IO during TensorFlow training

Full GPU Saturation,Maximum Productivity
DDN AI200 delivers a steady highthroughput, low-latency stream of data tothe DL training application
Achieving full GPU saturation ensures
every compute cycle is put to productiveuse

DDN true end-to-end parallelism enables
acceleration at scale for distributed
computing with multiple multi-GPU nodes

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

29

AI Benchmarking: Optimized TensorFlow Data Sets

Optimized Data Sets,Accelerated Training
Convert discrete data and metadata asset
collections into series of streamlined binary files forTensorFlow

TFRECORD CONVERSION BENCHMARK
3X FASTER WITH DDN STORAGE
Time (in minutes)

302367010203040

DDN Storage

Maximize DL training and validation efficiency while
minimizing data management and preparation
overhead607080

Other Storage Products

TENSORFLOW RESNET-50 TRAININGPERFORMANCE
2.5X FASTER WITH TFRECORDS
Time (inseconds)

Achieve more accurate results by eliminating
manual data shuffle bias

671
1614
0200400600800

TFRecord Data Set

DDN Storage | © 2018 DDN Storage

501000

1200

1400

1600

1800

Discrete Data Set© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

30

31 DDN PARALLEL ARCHITECTURE AND PROTOCOL vs NFS
FULL GPU SATURATIONMAXIMUM AI PRODUCTIVITYGPU STARVATIONCRIPPLED AI PERFORMANCE

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPUGPUGPUGPUGPUGPU

GPU

GPU

GPU

GPU

GPU

GPU

GPUGPUGPUGPUGPUGPU

GPU

GPU

GPU

GPU

GPU

GPU

GPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUGPUDDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

31

32 DDN PARALLEL ARCHITECTURE AND PROTOCOL vs NFS

MAX GPUUTILIZATIONDDN FULL GPU
SATURATION
DGX-1 GPU UTILIZATION (%)

2X FASTER
Consistent DL performance
Achieves full GPU utilization
Efficient loading of data set

AI200 READ (MB/S)MAX GPUUTILIZATIONNFS GPUSTARVATION

DGX-1 GPU UTILIZATION (%)

DDN Storage | © 2018 DDN Storage

NFS STORAGE READ(MB/S)© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

3233DDN A3I SOLUTIONS PERFORMANCE ENHANCEMENTS

AI200 and AI400 (Random Read IOPS andThroughput)
50

1400
1200

40

1000

30800
600

20400

AI200-Throughput
AI400-Throughput
AI200-IOPS
AI400-IOPS

10200004KB

DDN Storage | © 2018 DDN Storage

Throuhgput (GB/sec)

Performance improvements forRandom Read up delivers up to toaround 750K IOPs per RU for AI400
and over 22GB/s per RU

60

1600

IOPS (Kops/sec)

AI200 and AI400
Optimizations for AI

1800

8KB

16KB 32KBIO Size(Byte)

64KB

128KB

© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.

33

34

Thank You!
Keep in touch with us

Team-JPSales@ddn.com

Tokyu Bancho Bldg. 8F
6-2 Yonbancho Chiyoda-ku,Tokyo 102-0081@ddn_limitless+81-3-3261-9101+81-3-3261-9140

company/datadirect-networks

DDN Storage | © 2018 DDN Storage© 2015 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.