Computational physics project (A04) report
Interdisciplinary algorithms and computer simulations
http://bridge.kek.jp/A04/

Hideo Matsufuru (KEK)
QUCS 2012, 13-16 December 2012, Nara1

GoalsSupport projects in this area through promoting
collaboration and developping algorithms,
computational techniques and environment
Understand physics over different scales fromviewpoints of computational physics

Approaches:
Algorithms– Sharing information/know-how– Collaboration with applied mathematitians– Development of new algorithms

Architecture (computaional techniques)– Find out best architecture– Fully make use of processor's performance– Support porting/tuningEnvironment (Data Grid)
Lattie QCD common code (software engineering)2

Members
H.M. (KEK)
Tomoteru Yoshie (Tsukuba)
Takashi Kaneko (KEK)
Shoji Hashimoto (KEK)
Ken-Ichi Ishikawa (Hiroshima)
Tatsumi Aoyama (Nagoya)
Jun-ichi Noaki (KEK)
Naoya Ukita (Tsukuba)
Shinji Motoki (KEK)
Kazuhiro Yabana (Tsukuba)
Noritaka Shimizu (Tokyo)
Hideyuki Suzuki (Tokyo U. Science)
Kosuke Sumiyoshi (Numazu Call Tech)
Akira Mizuta (KEK)
Tetsuya Sakurai (Tsukuba)Hiroto Tadano (Tsukuba)
Daisuke Takahashi (Tsukuba)

3

Related proposed projectH21-22, H23-24: T.Sakurai
Large-scale numerical linear algorithms for elementary
particle, nuclear, and astrophysics
H21-22: T.Ishikawa
Study of dedicated system for computation of higher
order corrections– Multi-order precision computation– Dedicated hardware– Computations more than double precision needed– Cf. Multi-precision forum: http://suchix.kek.jp/mpcomp/4

Interdisciplinary algorithms
Matrix analyses of linear problems– Supernova/lattice QCD– Advices/hint to Bridge++ codeSeminar series (2010,2011)– Relativistic hydrodynamics (A.Mizuta)– Krylov subspace method (T.Sakurai)– Dynamical Density Matrix Renormalization Group(K.Iwano, KEK)
– (TD) DFT (K.Yabana)“Door to High Performance Computing”– Compilation of information/know-how– Wiki, under development (forever...)– Supported by SPIRE Field 5– Large contribution by A.Imakura for linear solver
algorithms: advice to choose best algorithm5

Interdisciplinary algorithms
Supernova explosion (Sumiyoshi, Sakurai, Imakura, HM)– MPI paralleized– Algorithm improvement
Weighted Jacobi preconditioner for iterative solver→ Talks by Imakura (Sat 15th) and Sumiyoshi (Sun 16th)

Relativistic hydrodynamics (A.Mizuta, talk on Sun 16 th)– Development of 3D/2D relativistic hydrodynamic code
– Nice lecture note (published in Genshikaku-kenkyu)

Also see

– Talk by K-I. Ishikawa– Poster by T.Kaneko

6ArchitectureRunning GPGPU servers– Totally 12 servers were installed:
NVIDIA Tesla/Fermi, AMD Radeon, Cell B.E.
– For prgram development and practical computation– Many flavor phsics (most heavily used)
several papers/proceedings published→ N.Yamada's talk (Sun 16th)– Supernova EOS project: next page
– Multi-precision computation– Developement of OpenCL programs→ bases of supernova EOS, Bridge++Lectures (by bender companies)– CUDA programming– CELL B.E. ProgrammingWorkshop (Jun 2009)
School on accelerator board (Dec 2009)7

Acceleration by GPGPU
Supernova-EOS Project (Togashi’s talk)
Construction of a new nuclear Equation of State (EOS) table
for supernova simulations with the variational method
In this study, the expectation values of the three-body Hamiltonian
with the degenerate Fermi-gas wave function <H3 α > F are necessary
for various densities and proton fractions.
V αijk : t h e c o m p o n e n to f th e U I X th r e e - b o d y p o te n tia l(α = 2π , R)

The three-body
Hamiltonian

These expectation values <H3α > F are expressed as double integrals ofanalytic functions.

Computation on GPU• Code ported to OpenCL• Nvidia Tesla C2050 @KEK

Acceleration is attained with GPU. (40 min → 5-10 sec)
8Architecture
Recent techinologies that may be efficient:(just for information)
HMPP/Open ACC– If you don't like OpenCL nor CUDA ...
– Directive based framework to use accelerators

XMP: directive based parallelization– If you don't like MPI ...
– Directive based framework for distributed memoryparallelization– Developed in Tsukuba Univ. etc.
– Introductory slide is available in Bridge++ website(by M.Sato, Tsukuba Univ.)To be tested before the end of project9

Data Grid

Advanced computational environment is necessary———Japan Lattice Data Grid (JLDG)
http://www.jldg.org/—————

Large amount of data (lattice QCD, etc.)
Fast data transfer among SC sites
Sharing data inside/among collaborations

Provides public data and high speed data transfer
Official service started May 2008
Tsukuba, KEK, Osaka RCNP, Hiroshima, Kanazawa, (Kyoto)+ Tokyo (Kashiwa), Nagoya
Virtual private network on SINET
Gfarm grid file system: ver.1 → ver.2
FUSE mount (NFS-like) is now possible

Continuously being improved
User group and stored data are increasing– Lecture for users (27 Jan 2010)

Application to other field?
10Japan Lattice Data Grid2008

As of 2012/12

http://www.jldg.org/Total 2.6PB(7sites,21servers)
450TB

New site:- Tokyo (Kashiwa)- Nagoya
New support:- SPIRE Field 5Storage:- 260TB → 2.6PB

5TB

part of data can beaccessed from ILDG

KEK

Kanazawa
CCS

Hiroshima

480TB
Tsukuba

200TB
Tokyo(Kashiwa)

300TB
Osaka

Nagoya

900TB

300TB
11

Lattice QCD common code
http://suchix.kek.jp/bridge/Lattice-code/





Now ver.1.0 is public [24 July 2012]
Development started in Autumn 2009
Object oriented, C++, MPI parallelized
Still in active development phase:Your advices/contributions are welcome !
Why new code ?

– Maintenance by ourselves– Detailed documents– Share ideas and skills in software
design and algorithms

Goals– High performance: on massively
parallel, accelerators, etc.
– Easy to understand for beginners– Easy to modify/extend for new ideas



Details: S.Motoki's talk (Sat 15th)12

Summary/outlook

A04 team—————

Approaches from viewpoints of computational physics
Algorithms
Architecture dependent techniques
Environment for data sharing
Lattice-common code (software engineering)

Outlook————This kind of activity is effective and may help your work
Not the end, but basis for further communication
SPIRE Field 5 continues similar activity
Please visit “Door to HPC” or JICFuS consult and give yourcomment !

13