Applications on TSUBAME3.0
Deep Learning and Large Scale CFD
Fast Algorithms for HPC and Deep Learning
Structured Low-Rank Matrices

Error Correction for TensorCores

Structured low-rank matrices areTensorCores
multiply
two16bitdense matrices that can bematrices and accumulate into ahierarchically subdivided to yield
32 bit matrix. This conversion tolow-rank off-diagonal blocks.
16bit results in a significant loss
in precision. The use of an
HSS matrices are a special typeauxiliary 16bit matrix to storeof structured low-rank matrixthemantissa
loss
isknowntowhere only the diagonal blocks
partially correct this error.
are recursively subdivided.
However, the round-to-zero in
For such matrices Cholesky/LU
TensorCore accumulation isanother source of error that has
factorization can be done without
not been accounted for in
dependency on trailing submatrices.
previous work. We develop a novel method that also corrects for thisWe extend this highly parallel factorization from HSS matrices to more error, which allows us to exactly match an GEMM in single precision,
general H2-matrices, which can be applied to 3-D problems.
while exploiting the compute capability of TensorCores.

Large Language Models
Pretraining Vision Transformers
Pretraining of vision transformers We are currently involved in multiple international efforts to pretrain
on Synthetic Images
requires a huge amount of realimages, but huge datasets suchas JFT-300M are not publiclyavailable. In order to address
this issue of data availability
we investigate the possibility
to train vision transformers on
synthetic images generated frommathetical equations. Such images are free of copyright, privacy issues
and societal bias. We are able to match the pretraining effect of real
images when using our synthetic image dataset VisualAtom.

some of the largest language models. The systems we are using
include Fugaku, LUMI, and soon Frontier and Aurora. Pretraining ofthese large language models requires many HPC tecniques for
reducing memory consumption and I/O, while extracting the fullpotential of low-precision matrix engines. Optimal combination ofdata parallel, tensor parallel, pipeline parallel, and sharded paralleltechniques are the key to achieving high throughput.

A Large-scale Foam Simulation
using a Multi-phase Field LBM and AMR

Unraveling Mystery of Sweeper

AMR for Multi-phase Flows

Dynamic Load Balancing using A Space-filling
Curve

Large-scale Mesh-based and Particle-based Simulations

It is well known thatMLB Los Angeles
Angels Shohei
Foam formation witha stable thin liquid
Ohtani shows
film is very difficult toincredible
simulate using
performances
forconventional
both batter andmethods due to thepitcher. He throws
limitation of meshresolution that cana kind of slider
be used. We“Sweeper” (breaking
addressed this issue by using a Multi-phase Field Lattice Boltzmannball) and its ball trajectory has more than 40cm horizontal movement butMethod and Adaptive Mesh Refinement. Using the Multi-Phase Field
small movement in vertical direction. We study the aerodynamics of theLBM, we can prevent a “numerical coalesce” phenomenon that leads tosweeper by using a simulation based on the Lattice Boltzmann Method
bubble break-up. Adaptive Mesh Refinement has been introduced sowith a cumulant collision term suitable for a large-eddy simulation model.
that the thin liquid film can be simulated efficiently. Herein, we
demonstrate a simulation of foam formed from 200 air bubbles using
It is found that a lift force appears when the spin axis inclines to batterour proposed method.
direction with 50-60 degree.
Simulations for multi-phase flows
require high-resolution grids tocapture phenomena at the interface.
By using the adaptive meshrefinement (AMR) method, which
dynamically adapts high-resolution
grids to interfaces, computational costand memory usage are reduced. The
spatial distribution of a computationalload change in time; therefore,
dynamic domain partitioning using aspace-filling curve is introduced for
multi-GPU computing to assign anequal number of grid points to eachGPU. The figures show thelarge-scale free-surface flow simulations for the dam-breaking process
and corresponding domain decomposition for 64 GPUs.

For large-scaleparticle-based
simulation and Adaptive
Mesh Refinement(AMR), it is a criticalissue to achieve
computational loadbalance and equal
memory usage on
multiple computenodes. A domain
partitioning in terms of a space-filling curve(SFC) is one of promising
candidates and it is recognized that a 1-dimensional mapping of
3-dimensional space by cutting the equal length. Due to low cost of
SFC domain partitioning, it is suitable for frequent re-partitioning in thesimulations of unsteady phenomena.

A Hilbert Curve

http://www.gsic.titech.ac.jp/sc23