NVIDIA DGX SuperPOD: IBM Storage
Scale and IBM 3500
Reference Architecture
Featuring NVIDIA DGX H100 and A100 Systems

DG-10981-001 v1
June 2023

Abstract
The NVIDIA DGX SuperPOD™ with NVIDIA DGX™ H100 or NVIDIA DGX A100 systems is
an artificial intelligence (AI) supercomputing infrastructure, which provides thecomputational power necessary to train today's state-of-the-art deep learning (DL)models and to fuel future innovation. The DGX SuperPOD delivers groundbreaking
performance, deploys as a fully integrated system, and is designed to solve the world's
most challenging computational problems.
This DGX SuperPOD reference architecture (RA) is the result of collaboration between
DL scientists, application performance engineers, and system architects to build asystem capable of supporting the widest range of DL workloads. The performancedelivered by the DGX SuperPOD with DGX systems enables the rapid training of DL
models at great scale. The integrated approach of provisioning, management, compute,
networking, and fast storage enables a diverse, multi-tenant system that can span data
analytics, model development, and AI inference.
In this paper, the IBM Storage Scale System 3500 was evaluated for suitability insupporting DL workloads when connected to the DGX SuperPOD. IBM 3500 is a modern
implementation of software-defined storage with low latency NVMe physical storage,advanced erasure coding, connected using InfiniBand and Ethernet networking. Multiple
IBM 3500 systems can be aggregated to create a high-performance cluster filesystem,
tiered for data economics, or connected to multiple clusters for geographic and crossplatform data sharing in a single global data platform. The IBM 3500 is a 2U building
block that makes it easy to deploy, manage, and grow fast storage for AI with NVIDIA
DGX systems.
The DGX SuperPOD is a turnkey solution validated at scale with scale-out IBM 3500
network attached storage (NAS) systems. Joint testing and integration ensure the
NVIDIA DGX SuperPOD is a rapidly deployed and a robust solution for scalable AI
development. NVIDIA and IBM jointly test, plan, and install the system, with the storage
backed by IBM global deployment and support services.
As configured, tested, and deployed in the NVIDIA SuperPOD, IBM 3500 systems can beused for all DL workloads including:>Training models efficiently with directly from IBM Storage Scale.

>

Automatically leverage local resources as cache to minimizing rereading data acrossthe network.

>

Workspace for long-term storage (LTS) of datasets.

>

A centralized repository for the acquisition, manipulation and sharing of results using
standard protocols like NFS, SMB, and S3.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | i

Contents
Storage Overview.............................................................................................................................................. 1
Storage Caching Hierarchy ..................................................................................................................... 1
Storage Performance Requirements .................................................................................................. 2
About the IBM Storage Scale System E3500 ........................................................................................ 3
Validation Methodology and Results ........................................................................................................ 4
Microbenchmarks ....................................................................................................................................... 4
Hero Benchmark Performance .............................................................................................................. 5Single-Node, Multi-File Performance .............................................................................................. 5
Multi-Node, Multi-File Performance ................................................................................................ 6
Single-File I/O Performance................................................................................................................ 6
Application Testing..................................................................................................................................... 7
ResNet-50 .................................................................................................................................................. 7
NLP—BERT ................................................................................................................................................ 8
Recommender—DLRM.......................................................................................................................... 8
Summary .............................................................................................................................................................. 9

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | ii

Storage Overview
Training performance can be limited by the rate at which data can be read and reread
from storage. The key to performance is the ability to read data multiple times, ideallyfrom local storage. The closer the data is cached to the GPU, the faster it can be read.
Storage must be designed considering the hierarchy of different storage technologies,
either persistent or nonpersistent, to balance the needs of performance, capacity, andcost.

Storage Caching Hierarchy
The storage caching hierarchy of the DGX system is shown in Table 1. Depending ondata size and performance needs, each tier of the hierarchy can be leveraged tomaximize application performance.
Table 1. DGX system storage and caching hierarchy
Storage Hierarchy Level

Technology

Total Capacity1

Performance1

RAM

DDR4

2 TB per system2> 200 GB/s

NVMe

30 TB per system3> 50 GB/s

Internal Storage

1. Total capacity and performance values are per system.
2. Shared between the operating system, application, and other system processes
3. PCIe Gen 4 NVMe SSD storage

Caching data in local RAM provides the best performance for reads. This caching istransparent after the data is read from the filesystem.
While local storage is fast, it is not practical to manage a dynamic environment with local
disk alone in multi-node environment. Functionally, centralized storage can be as quick
as local storage on many workloads.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 1

Storage Performance Requirements
Performance requirements for high-speed storage greatly depend on the types of AI
models and data formats to be used. The DGX SuperPOD has been designed as acapability-class system that can manage any workload both today and in the future.
However, if systems are going to focus on a specific workload, such as natural language
processing (NLP), it may be possible to better estimate performance needs of thestorage system.
To enable customers to characterize their own performance requirements, some generalguidance on common workloads and datasets is shown Table 2.
Table 2. Characterizing different I/O workloads
Storage Performance
Level Required

Example Workloads

Dataset Size

Good

NLP

Most all datasets fit in cache

BetterImage processing withcompressed images,
ImageNet/ResNet-50

Many to most datasets can fit within
the local system’s cache.

Best

Training with 1080p, 4K, or
uncompressed images, offlineinference, ETL

Datasets are too large to fit into cache,massive first epoch I/O requirements,
workflows that only read the dataset
once

Performance estimates for the storage system necessary to meet the guidelines inTable 2 are in:•Table 4 of the NVIDIA DGX SuperPOD Reference Architecture—DGX H100 Systems.

•

Table 8 of the NVIDIA DGX SuperPOD Reference Architecture—DGX A100 Systems.

Achieving these performance characteristics may require the use of optimized fileformats such as TFRecord, RecordIO, or HDF5.
The high-speed storage provides a shared view of an organization’s data to all systems.
It needs to be optimized for small, random I/O patterns, and provide high peak system
performance and high aggregate filesystem performance to meet the variety ofworkloads an organization may encounter.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 2

About the IBM Storage Scale System
3500
The IBM Storage Scale System 3500, shown in Figure 1, combines the performance ofNVMe storage technologies with the reliability and the rich features of IBM Storage
Scale in a powerful 2U storage system that scales out for performance and capacity.
Figure 1. IBM Storage Scale System 3500

IBM Storage Scale on NVMe is designed to be the market leader in all-flash performance,and scalability with read performance of up to 125 GB/s and 1.3 million IOPs per NVMe
all-flash appliance. Providing data-driven multicloud storage capacity, the NVMe all-flash
appliance is deeply integrated with the software-defined capabilities of IBM Storage
Scale to seamlessly plug it into for analytics, scalable cluster, or AI workload.
Available with multiple drive options and advanced erasure coding, the IBM 3500
provides options to optimize costs for different installation sizes. As with all IBM Storage
Scale solutions, capacity and performance can be scaled and tiered. Combining IBM
3500 all-NVMe systems provides nearly linear performance scalability. IBM 3500
solutions may also be used as an all-flash NVMe performance tier on slower storage,
including tape or object storage.
IBM Storage Scale is an industry leader in high-performance file systems. The underlying
general parallel file system (GPFS) provides scalable throughput and low-latency data
access, as well as superior metadata performance. Unlike other systems that can easilybottleneck, the distributed architecture of a parallel filesystem provides reliable
performance for multi-user sequential and random read or write. This is particularly
important in AI clusters where multiple compute nodes may need to read or write to thesame file. IBM Storage Scale has been proven on the largest AI clusters in the world,
including the US National Labs supercomputers Summit and Sierra, as well as the Circe
supercomputer built by NVIDIA.
IBM Storage Scale provides Container Native Access and Operators to support
Kubernetes driven DevOps and Data Ops practices. In addition, IBM Storage Scaleprovides enterprise features such as call-home proactive support, encryption, and audit
file logging that works with enterprise security information and event management(SEIM) platforms.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 3

Validation Methodology and Results

Three classes of validation tests are used to evaluate a particular storage technology
and configuration for use with the DGX SuperPOD: microbenchmark performance, realapplication performance, and functional testing. The microbenchmarks measure key I/Opatterns for DL training and are crafted so they can be run on nodes with CPU only. This
reduces the need for large GPU-based systems to validate storage. Real DL training
applications are then run on a DGX SuperPOD to confirm that the applications meet
expected performance. Beyond performance, storage solutions are evaluated forrobustness and resiliency as part of functional testing.
The NVIDIA DGX SuperPOD storage validation process leverages a “Pass or Fail”
methodology. Specific targets are set for the microbenchmark test. Each benchmark
result is graded as good, fair, or poor. A passing grade is one where at least 80% of thetests are good, and none are poor. In addition, there must be no catastrophic issues
created during testing. For application testing, a passing grade is one where all cases
complete within 5% of the roofline performance set by running the same tests with data
staged on the DGX RAID. For functional testing, a passing grade is one where allfunctional tests meet their expected outcomes.

Microbenchmarks
In the Storage Performance Requirements section, there are several high-level
performance metrics that storage systems must meet to qualify as a DGX SuperPOD
solution. Current testing requires that solutions meet the “Best” criteria discussed in thetable. In addition to these high-level metrics, several groups of tests are run to validate
the overall capabilities of the proposed solutions. These include single-node tests where
the number of threads is varied and multi-node tests where a single thread count is used
and as the number of nodes vary. In addition, each test run in both Buffered andDirectIO modes and when I/O is performed to separate files or when all threads andnodes operate on the same file.
Four different read patterns are run. The first read operation is sequential where no data
is in the cache. The second read operation is executed immediately thereafter toevaluate the ability for the filesystem to cache data. The cache is purged and then thedata is read again, this time randomly. Lastly, the data is reread again randomly, toevaluate data caching.
IOR benchmark for single-node and multi-node tests was used.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 4

Hero Benchmark Performance
The hero benchmark helps establish the peak performance capability of the entire
solution. Storage parameters, such as filesystem settings, I/O size, and controlling CPU
affinity, were tuned to achieve the best read and write performance. Storage devices
were expected to demonstrate that quoted performance was close to measured
performance. Other tests are crafted to demonstrate performance of real workloads.
The delivered solution for a single SU had to demonstrate over 20 GiB/s for writes and
65 GiB/s for reads. Ideally, the write performance should be at least 50% of the read
performance. However, some storage architectures have a different balance between
read and write performance, so this is only a guideline and read performance is more
important than write.

Single-Node, Multi-File Performance
For single-node performance, I/O read and write performance is measured by varying thenumber of threads in incremental steps. Each thread writes (and reads) to (and from) its
own file in the same directory.
For single-node performance tests, the number of threads is varied from 1 to the ideal
number of threads to maximize performance—typically more than half the cores (64),but no more than the total physical cores (128). The I/O size is varied between 128 KiB
and 1 MiB and the tests are run with Buffered I/O and Direct I/O.
The target performance for these tests is shown in Table 3.
Table 3. Single-node, multi-file performance targets
ThreadCount

Buffered or
DirectIOI/O size(KiB)

1

Buffered1
1

Performance
Write

Read

RereadRandomRead

RandomReread

128

512

1,024

1,536

256

1,536

Buffered

1024

800

3,072

4,608

768

1,024

Direct

1024

1,024

1,024

-

1,024

-

When maximizing single-node performance, the thread count may vary, however it isexpected that performance does not drop significantly when additional threads are usedbeyond the optimal thread count.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 5

Target performance for single-node performance with multiple threads is in Table 4. The
optimal number of threads may vary for any storage configuration.
Table 4. Single-node, multi-threaded performance targets
ThreadCount

Varies

Buffered or
DirectIOI/O size(KiB)

Buffered

Performance
Write

Read

RereadRandomRead

RandomReread128

8,000

12,00018,00012,00018,000

Direct128

8,000

15,000-15,000-Buffered1024

10,00020,00030,00020,00030,000

Direct1024

10,00020,000-20,000-

Reread performance relative to read performance can vary substantially between
different storage solutions. The reread performance should be at least 50% of the read
performance for both sequential and random reads.

Multi-Node, Multi-File Performance
The next test performed is multi-node I/O read and write test to make sure that thestorage appliance can provide the minimum required buffered read and write per system
for the DGX SuperPOD. This benchmark determines the capacity of a filesystem to scale
performance of different I/O patterns. Performance should scale linearly from one to a
few nodes, reach a maximum performance, then not drop off significantly as morenodes are added to the job.
The target performance for a single SU of 20 nodes is 65 GiB/s for reads with I/O size of
128 KiB or 1,024 KiB, and if the I/O is Direct or Buffered. The write performance should be
at least 20 GiB/s, but ideally it would be 50% of the read performance. Results from thesetests must be interpreted carefully as it is possible to add more hardware to achieve theselevels. Overall performance is the goal, but it is desirable that the performance comesfrom an efficient architecture that is not over-designed for its use.

Single-File I/O Performance
A key I/O pattern is reading data from a single file. Often the fastest way to read data iswhen all the data is organized into a single file, such as the RecordIO format. This canoften be the fastest way to read data because it eliminates any of the open and close
operations required when data are organized into multiple large files. Single-file reads
are a key I/O pattern on DGX SuperPOD configurations.
Targeted performance and expected I/O behavior is that the single-node,
multi-threaded, writes can successfully create the file, that sequential read and random
read performance is good, and that read performance scales as more nodes are used.
Multi-node, multi-threaded, single file writes are not evaluated. In addition, it is expectedthat buffered reread performance will be like the multi-file reread performance.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 6

Target performance for single file I/O is in Table 5.
Table 5. Single-file read performance targets
Node
Count

Buffered or
DirectIOI/O size(KiB)

1

Buffered1

Performance
Read

Reread1

RandomRead

RandomReread1

128

2,500

1

2,500

1

Direct128

15,000-15,000-1

Buffered

1024

3,000

1

3.000

1

1

Direct1024

20,000-20,000-20

Buffered128

65,0001

65,0001

20

Direct128

65,000-65,000-20

Buffered1024

65,0001

65,0001

20

Direct1024

65,000-65,000-

1. Reread performance of cached data should be near in performance to the results from the multi-filereread test

Application Testing
Microbenchmarks provide indications of the peak performance of key metrics. However,
it is application performance that is most important. A subset of the MLPerf Training
benchmarks is used to validate storage performance and function. Here, both
single-node and multi-node configurations are evaluated to ensure that the filesystem
can support different I/O patterns and workloads. Training performance when data is
staged on the DGX RAID was used as the baseline for performance. The performancegoal is for the total time to train when data is staged on the shared filesystem to bewithin 5% of those measured when data is staged on the local RAID. This is not just forindividual runs, but also when multiple cases are run across the DGX SuperPOD at thesame time.

ResNet-50
ResNet-50 is the canonical image classification benchmark. Its dataset size is over
100 GiB and it has a requirement for fast data ingestion. On a DGX system, a single nodetraining requires approximately 3 GiB per second and the dataset is small enough that it
can fit into cache. Preprocessing can vary, but the typical image size is approximately
128 KiB. One challenge of this benchmark is that at NVIDIA the processed images are
stored in the RecordIO format (i.e. one large file for the entire dataset) because thisprovides the best performance for MLPerf. This one large file can stress sharedfilesystem architectures that do not distribute the data across multiple targets orcontrollers.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 7

NLP—BERT
BERT is the reference standard NLP model. In this test, the system is filled with twoeight node jobs and four single node jobs (or less if not all nodes of the SU are available).
It is expected that the total time to train is within 5% of that measured when training
from the local raid. This test does not stress the filesystem but does ensure that local
caching is operating as needed.

Recommender—DLRM
The recommender model has different training characteristics than ResNet-50 and
BERT in that the model trains in less than a single epoch. This means that the data set isread no more than once, and local caching of data cannot be used. To achieve fulltraining performance, DLRM must be able to read data at over 6 GiB/s. In addition, thefile reader uses DirectIO that stresses the filesystem differently than the other two files.
The data are formatted into a single file.
This test is only run as a single node test; however, several tests are run where thenumber of simultaneous jobs vary from one to the total number of nodes available. It is
expected that the shared filesystem only sustains performance up to the peak
performance measured from the hero test.
For 20 simultaneously cases, the storage system would have to provide of over 120
GiB/s of sustained read performance, more than what is prescribed in the Storage
Performance Requirements section. Even the best performance outlined in that section
is not meant to support every possible workload. It is meant to provide a balance of highthroughput while not over-architecting the system.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 8

Summary
NVIDIA evaluations show that the IBM Storage Scale System 3500 with Storage Scale
file system meets the DGX SuperPOD performance and functionality requirements. It isa great choice to pair with a DGX SuperPOD to meet current and future storage needs.
As storage requirements grow, IBM 3500 building blocks can be added to seamlessly
scale capacity, performance, and capability. The combination of NVME hardware and
IBM Storage Scale parallel file system architecture provides excellent random read
performance, often just as fast as sequential read patterns. Testing has validated thateach IBM 3500 can deliver the highest levels of per node performance and meet all ourapplication performance requirements. The IBM Storage Scale parallel file systemprovides a platform that is fully supported with the NVIDIA DGX SuperPOD and has been
deployed in production.

NVIDIA DGX SuperPOD: IBM Storage Scale and IBM 3500

DG-10981-001 v1 | 9

NoticeThis document is provided for information purposes only and shall not be regarded as a warranty of a certain functionality, condition, or quality of a
product. NVIDIA Corporation (“NVIDIA”) makes no representations or warranties, expressed or implied, as to the accuracy or completeness of theinformation contained in this document and assumes no responsibility for any errors contained herein. NVIDIA shall have no liability for theconsequences or use of such information or for any infringement of patents or other rights of third parties that may result from its use. This document
is not a commitment to develop, release, or deliver any Material (defined below), code, or functionality.
NVIDIA reserves the right to make corrections, modifications, enhancements, improvements, and any other changes to this document, at any time
without notice.
Customer should obtain the latest relevant information before placing orders and should verify that such information is current and complete.
NVIDIA products are sold subject to the NVIDIA standard terms and conditions of sale supplied at the time of order acknowledgement, unless otherwise
agreed in an individual sales agreement signed by authorized representatives of NVIDIA and customer (“Terms of Sale”). NVIDIA hereby expressly objects
to applying any customer general terms and conditions with regards to the purchase of the NVIDIA product referenced in this document. No contractual
obligations are formed either directly or indirectly by this document.
NVIDIA makes no representation or warranty that products based on this document will be suitable for any specified use. Testing of all parameters ofeach product is not necessarily performed by NVIDIA. It is customer’s sole responsibility to evaluate and determine the applicability of any information
contained in this document, ensure the product is suitable and fit for the application planned by customer, and perform the necessary testing for theapplication to avoid a default of the application or the product. Weaknesses in customer’s product designs may affect the quality and reliability of the
NVIDIA product and may result in additional or different conditions and/or requirements beyond those contained in this document. NVIDIA accepts noliability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any mannerthat is contrary to this document or (ii) customer product designs.
No license, either expressed or implied, is granted under any NVIDIA patent right, copyright, or other NVIDIA intellectual property right under thisdocument. Information published by NVIDIA regarding third-party products or services does not constitute a license from NVIDIA to use such products
or services or a warranty or endorsement thereof. Use of such information may require a license from a third party under the patents or otherintellectual property rights of the third party, or a license from NVIDIA under the patents or other intellectual property rights of NVIDIA.
Reproduction of information in this document is permissible only if approved in advance by NVIDIA in writing, reproduced without alteration and in fullcompliance with all applicable export laws and regulations, and accompanied by all associated conditions, limitations, and notices.
THIS DOCUMENT AND ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS(TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR
OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT,MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY LAW, IN NO EVENT WILL NVIDIA BE LIABLE
FOR ANY DAMAGES, INCLUDING WITHOUT LIMITATION ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES,HOWEVER CAUSED AND REGARDLESS OF THE THEORY OF LIABILITY, ARISING OUT OF ANY USE OF THIS DOCUMENT, EVEN IF NVIDIA HAS BEEN
ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s
aggregate and cumulative liability towards customer for the products described herein shall be limited in accordance with the Terms of Sale for the
product.

Trademarks
NVIDIA, the NVIDIA logo, NVIDIA DGX , and NVIDIA DGX SuperPOD are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. andother countries. Other company and product names may be trademarks of the respective companies with which they are associated.

Copyright© 2023 NVIDIA Corporation. All rights reserved.

NVIDIA Corporation | 2788 San Tomas Expressway, Santa Clara, CA 95051

http://www.nvidia.com