INDUSTRYAI Research

CHALLENGES• Requirement for 580 Petaflopsof GPU+CPU performance.
• Trusted partner with theability to deliver high-end
servers with GPUs.

Introduction
NEC has been a leader in designing and distributing a wide range of systems for manyyears. In addition, NEC has a track record of half a century in AI technology R&D andcreated many of the world's highest levels of AI in various AI fields, such as image andvideo recognition, language and semantic understanding, data analysis, predictive
detection, and optimal planning and control.
NEC is the world's No.1 in face authentication and iris authentication, and the world'sNo.8 company in terms of the number of acceptances at top international conferences onmachine learning. NEC conducts AI research at all its laboratories, and the number of AI
researchers is in the hundreds. In this way, NEC has one of the world's leading AI researchand development capabilities.
NEC chose Supermicro GPU and Ultra systems to strengthen the technologicalcompetitiveness of AI and to maintain and strengthen business competitiveness in thefield of AI.

Challenges
NEC recognized that its challenge is accelerating the creation of advanced AI andspeeding up social value creation. In the actual development of AI, while changing data,
algorithms, etc., trial and error will be repeated thousands of times. However, due to theincrease in the required amount of computation of deep learning, there are some cases© Copyright 2022 Supermicrowhere learning consumes up to several thousand hours per training run, and in somecases, even more.

SOLUTION
Supermicro GPU and Ultra
Servers
SYS-420GP-TNAR•

Dual 3rd Gen Intel®
Xeon® Platinum 8358
Processor•

1 TB Memory/Node•

8 x NVIDIA A100 80GB
Tensor Core GPU SXM•

NVIDIA HGX™ platform
with NVIDIA NVLink™
and NVSwitch™

SYS-120U-TNR•

Dual 3rd Gen Intel Xeon
Gold 6342 Processors•256GB Memory/NodeFor example, GPT-3, a huge language model, took up to 355 years of GPU time to learn
with a single NVIDIA Tesla V100 GPU. The time needed for this type of training run meantthat results were not usable for many industries, where training needed to becompleted in hours, not years. This real-world example shows that an AI supercomputeris necessary for many training models, which requires in-house expertise or a trustedsupplier to design and implement such a large cluster.
Therefore, by using AI supercomputers, it is expected that advanced AI algorithms canbe researched and developed without the restriction of limited computing resources.
Additionally, an extensive AI system needs to have the capacity so that hundreds ofresearchers have simultaneous access to the system to advance their specific needs.

Solution
NEC decided to work with Supermicro to design and deliver the NEC AI supercomputer.
There were two main reasons that NEC chose Supermicro as a strategic partner.
1) NEC was able to freely customize and configure the internal configuration on
the GPU server according to the architecture for the AI supercomputer requiredby NEC. However, to optimize for AI learning workloads and deep learning, NECneeded to change or reconfigure many things, including the server's physicalhardware configuration, BIOS settings, and fan settings. Therefore, the ability
to work with Supermicro to modify these settings was critical to selecting theSupermicro GPU servers.
2) NEC realized that as the technology moves forward, the hardware that would
be used needed to be extensible and able to house future generations of AIacceleration. In addition, as Deep Learning evolves dramatically from year toyear, the freedom and expandability of hardware that could freely change andextend the physical configuration were very attractive to NEC.
Since NVIDIA's GPUs are the de facto standard for many applications in deep learning
frameworks such as PyTorch and TensorFlow, there are a lot of challenges or difficulties
with using non-NVIDIA GPUs, and it is difficult to conduct research on deep learning at
this time if their underlying infrastructure was to be changed. Therefore, NEC definitely
believes that their AI researchers would best be able to perform their research on NVIDIAGPUs.
NEC choose to use the NVIDIA A100 Tensor Core GPUs. The main reason is that the total
calculation speed for AI operations using the NVIDIA A100 is exceptional. In addition, theGPU memory bandwidth of the A100 is 2TB/s, but in deep learning, it is basically amemory bandwidth limit. So, with the TF32 used in the A100, the range is the same as
the FP32, but the precision can be treated as FP16, and although the accuracy is notcompromised, the memory bandwidth bottleneck can be alleviated. In other words, it iseasier to take advantage of the calculation performance of the A100. In particular, manygroups in NEC are researching and developing AI using images, such as biometricauthentication, image recognition, and video recognition, to be more memory-limitingfor GPUs. Therefore, NEC has adopted the A100 with the TF32 supported.

© Copyright 2022 SupermicroThe second reason that NEC adopted the HGX A100 is that it is equipped with an NVIDIA
A100 Tensor core GPUs, and it uses the third generation NVIDIA NVSwitch™ tocommunicate between GPUs at 600GB/sec. This internal switch is required because
data communication between GPUs can be performed at high speed in the server, so it
can learn at a very high rate.
BENEFITS•

Faster and MoreComplete AI Workloads•Future ready for next
generation GPUs

Additionally, NEC decided that the GPUs needed to communicate with each other for
training applications. With the GPU-to-GPU communication within, multiple GPUs cancommunicate at the high speed of 600 GB/s, NEC was confident that they could fully
utilize the computing performance of the GPU with distributed learning processing, sothe AI R&D efficiency will be improved significantly.
Another reason that NEC chose NVIDIA's products for network switches and NICs is theConnextX-6 solution, which supports end-to-end RoCEv2 communication from theserver to the switch, and the actual performance can be close to the limit of 200GbE. In
distributed deep learning processing, it is necessary to process each iteration ofAllReduce to exchange parameters, and a significant amount of communication occurs
within the clusters. Therefore, in recent large-scale AI models, the expected processing
time will not be achieved if the communication speed is lower due to the narrow
bandwidth and higher latency. As a result, the distributed efficiency isn't met, no matter
how many servers are being used. To bypass the CPU and reduce latency, it is important
to be able to perform NVIDIA GPUDirect® RDMA using RoCE v2. Supermicro GPU serverand NVIDIA technologies can easily connect all switches and servers end-to-end, andNEC has adopted those products and solutions.

NECNEC established itself as a leader
in the integration of IT andnetwork technologies while
promoting the brand statementof “Orchestrating a brighterworld.” NEC enables businesses
and communities to adapt torapid changes taking place inboth society and the market as itprovides for the social values ofsafety, security, fairness andefficiency to promote a moresustainable world where
everyone has the chance to reach
their full potential. For more
information, visit NEC athttps://www.nec.com

SYS-420G-TNAR

SYS-120U-TNR

Solution Specifics:
Quantity

SupermicroServer

116 nodes
with a totalof 928 GPUs

SYS-420GP-TNAR

119 nodes

SYS-120U-TNR

CPU Per Node2 x 3rd Gen Intel
Xeon Platinum
8358 Processors
32 cores, 2.6GHz
2 x 3rd Gen IntelXeon Gold 6342
Processors
24 cores, 2.8GHz

Memory PerNodeGPU Node1,024GB

8 x NVIDIA
A100 80GB
NVLink,NVSwitch

256GB© Copyright 2022 SupermicroBenefits
After running many tests, NEC determined that with the new AI supercomputer,compared with one Tesla V100, the deep learning performance will be up to 4,600 times
faster, NEC decided that the AI development time could be greatly reduced. As a result,
For example, GPT-3, a huge language model, took up to 355 years of GPU time to learn
NEC researchers will now be able to achieve significantly more research into many AI
domains with the new AI Supercomputer from Supermicro.
"Computational capability is a source of competitiveness in the age of AI. We are very
pleased to be able to significantly accelerate AI research by providing researchers with a
distributed deep learning environment with amazing performance with GPU servers by
NVIDIA and Intel technologies. NEC continues to work closely with Supermicro toaccelerate the creation of social value by enhancing our AI research." - Takatoshi
Kitano, Senior AI Platform Architect at NEC
For more information, please visit:https://www.supermicro.com/en/products/gpu
https://www.nec.com/en/global/rd/aisupercomputer/© Copyright 2022 Supermicro