Applications on TSUBAME3.0
Fast Algorithms and Large Scale CFD
Fast Algorithms for HPC and Deep Learning
Cache Blocking for Low-Rank Matrices

Structured low-rank matrix
operations result in a batchof small and rectangularmatrices. This causes
standard BLAS libraries toperform poorly. Evenbatched MKL is suboptimal
when the small matrices
are skinny. We develop ahighly optmized innerkernel for such matrices
and show that we canoutperform batched MKL
by 2x, which can be usedin structured low-rankmatrix operations.

Memory Efficient Deep LearningTSQR on TensorCores

The compression ofH-matrices involves a QR
decomposition of tall andskinny matrices --TSQR.
Since the goal of thecompression is toperform a low-rankapproximation of theoriginal dense matrix, theaccuracy required for thisoperation is not so high.
This means that we canuse the TensorCores on the latest GPUs, to perform the TSQR much
faster than usual. We investivate the amount of performance we canachieve when using different precision (FP16 and FP32) along with theTensorCore. We find that we can get close to 15 TFlops for tall matrices.

Training ImageNet in 2 minutes on 2048 GPUs

The size of deep neural networks
Data-parallel training of deep
continues to grow rapidly, whereneural networks suffers from themodels like GPT-3 andlarge-batch problem, where theMegatron-LM exceed the memory
generalization gap increases as
capacity of a single GPU. The trend
the mini-batch size increases
in computer architecture where theproportional to the number ofarithmetic throughput is growing
GPUs. Using the Kronecker
faster than memory capacity,
Factorization mention on the right,
suggests that memory consumption
we are able to use a second order
is a critical issue in deep learning.
optimization method with minimum
We reduce the memory usage ofoverhead, which allows us to retain the convergence even for extremely
deep neural networks by scattering, large batch sizes. We trained ImageNet in 2 minutes on 2048 GPUs,
recomputing, and offloading theusing a batch size of 131,072.
model parameters and activations.

Large-scale Mesh-based and Particle-based Simulations
Weak-Compressible Flow Computations for
Gas-Liquid Two-Phase Flows

Large-scale Granular Simulation

Adaptive Mesh Refinement for Multi-phase Flows

Dynamic Load Balancing using A Space-filling
Curve

Discrete element method(DEM) is often used toA weakly
simulate granularcompressible
dynamics and its simple
scheme withalgorithm with the contact
interface-adaptedinteraction is suitable for
AMR method forGPU computing.
incompressible
However so manygas-liquid two-phaseparticles are included andflows without solving
the particle distributions
the Poisson equation
arechanging
intimeandhas been developed. The fully explicit time integration is achieved byspace.
Adynamicdomain
solving pressure evolution equation derived from compressible
Naiver-Stokes equation under the condition of isothermal and low-Mach decomposition has to be introduced for multiple-node computing. In abunker shot, the sand wedge does not hit to the golf ball directly andnumber. To reduce volume oscillation, the conservative Allen-Cahntransferring the force through the sand to the ball in order to reduce theequation coupled with Level-set method is solved. The results ofimpact. In this simulation, 16.7 millions of DEM particles are used tobenchmarks agree well with those of semi-implicit incompressible
represent the dynamics of the sands with 256 GPUs.
solvers and simulated 2D/3D soap bubble forming problem.
Simulations for multi-phase flows
require high-resolution grids tocapture phenomena at the interface.
By using the adaptive meshrefinement (AMR) method, which
dynamically adapts high-resolution
grids to interfaces, computational costand memory usage are educed. The
spatial distribution of a computationalload change in time; therefore,
dynamic domain partitioning using aspace-filling curve is introduced for
multi-GPU computing to assign anequal number of grid points to eachGPU. The figures show thelarge-scale free-surface flow simulations for the dam-breaking process
and corresponding domain decomposition for 64 GPUs.

For large-scaleparticle-based
simulation and Adaptive
Mesh Refinement(AMR), it is a criticalissue to achieve
computational loadbalance and equal
memory usage on
multiple computenodes. A domain
partitioning in terms of a space-filling curve(SFC) is one of promising
candidates and it is recognized that a 1-dimensional mapping of
3-dimensional space by cutting the equal length. Due to low cost ofSFC domain partitioning, it is suitable for frequent re-partitioning in thesimulations of unsteady phenomena.

A Hilbert Curve

http://www.gsic.titech.ac.jp/sc20