Activity of A04 teamInterdisciplinary algorithms and computer simulations
http://bridge.kek.jp/A04/

Hideo Matsufuru (KEK)

3 December 2011, workshop@Nemu-no-sato

1Members

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato2

GoalsSupport researches in this area through developmentof algorithms, computational technologies andenvironments
Understand physics over different scales fromviewpoints of mathematical structure and algorithms

Approaches

Algorithms
Architecture dependent techniques
Environment (Data Grid)
Common code (lattice QCD)

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

3

Members

HM (KEK)
Koichi Yabana (Tsukuba)
Tomoteru Yoshie (Tsukuba)
Noritaka Shimizu (Tokyo)
Takashi Kaneko (KEK)
Hideyuki Suzuki
Shoji Hashimoto (KEK)(Tokyo U. Science)
Ken-Ichi Ishikawa (Hiroshima)
Kosuke Sumiyoshi
Tatsumi Aoyama (Nagoya)(Numazu Call. Tech.)Jun-ichi Noaki (KEK)
Akira Mizuta (KEK)
Naoya Ukita (Tsukuba)
Shinji Motoki (KEK)
Tetsuya Sakurai (Tsukuba)
Hiroto Tadano (Tsukuba)
Daisuke Takahashi (Tsukuba)

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

4Recent activity
Improving algorithms and parallel computation
Supernova explosion (Sumiyoshi, Sakurai, Imakura, HM)– Iterative linear algorithm for parallel comp.
– talks by Sumiyoshi and Imakura
Linear algorithms for lattice QCD (Sakurai, Tadano, et al.)General relativistic fluid dynamics/thermal radiation in
the post process– tuning and parallelization (Mizuta)
Lattice QCD commmon code (Talk by Noaki)

In the following, some details ofGPU computation
KEKSC (announcement)
Data Grid (JLDG)

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato5Algorithm● Goals:– Sharing know-how of algorithms– Collaboration with applied mathematicians– Bases toward inter-field application– Development of new algorithms● Example of target algorithms– Linear equation solver, eigenvalue problem– Evolution equation, variational methods, Monte Carlomethods● Algorithms suitable to each architecture:– For massively parallel/coarse grained parallel machines– For accelerator boards (GPGPU, Cell, etc.)••

mixed precision algorithms
Less communication with host CPU

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato6Computation on GPU● GPU is now commonly used for numerical calculations– Ex. Lattice QCD: extensively used in practice– Many body / Integration / Multi-precision calculations– Many libraries are available– Now double prec. is enough fast / ECC available● How to implement ?

– CUDA by NVIDIA has been extensively used
– CUDA Fortran now available in PGI compiler– OpenCL: common framework for multi-core computation –applicable to NVIDIA, AMD, Cell, etc.

● Which application ?

– Decomposed into large num. of independent computations– Less memory access, many arithmetic operations

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato7

Computation on GPU● Running computation servers– Tesla cluster, Fermi/AMD GPU/Cell B.E. servers– Test bed for code development– Providing computational resources:

Lattice QCD, multi-precision calculation, etc.
● Integration in nuclear eq. of state (Takano, Togashi +HM)
– Code ported to OpenCL– Nvidia Tesla C2050 (double prec.)@KEK– Acceleration: 40 min → 5-10 sec (roughly)● Goals: sharing techniques and fundamental parts– Materials are compiled at our website– Prescription guide– Development of 'framework'● If you are interested in, please contact us.

– Direct contact or via JICFuS support:– You can ask whether your application MAY become fast.

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

8

KEK supercomputer
http://scwww.kek.jp/

KEKSC is now available / fully installed soon
– System-A in service at data center (~Jan)– System-A+B: March 2012~● System-A: Hitachi SR16000– Power7, 54.9 TFlops, 14TB memory– 56 nodes: 960GFlops, 256GB/node– Automated parallelization on single node
● System-B: IBM Blue Gene/Q– 6 racks (3 from March, 3 from Oct 2012)– 1.258PFlops, 96TB in total– Rack: 1024 nodes, 5D torus network

209.7TFlops, 16TB memory● Application– Large-scale simulation program (http://ohgata-s.kek.jp/)– scbridge: for getting started for members in this projectif you are interested in, please contact Noaki-san or HM
H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

9

Data Grid
Computational Science
requires advanced data handling

http://www.jldg.org/– archive large amount of data
– fast data transfer among SC sites– research data sharing amongcollaborations nation (world)wide●Japan Lattice Data Grid (JLDG)– provide fast data transfer via

VPN on SINET4– connect major LQCD research sites(Tsukuba, KEK, RCNP, Hiroshima, Kanazawa)– provide uniform file system (single directory-file tree)
realized by Gfarm (Grid Data Farm)– connecting to International Lattice Data Grid (ILDG)
to provide public gauge configurations
H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

10

JLDG: peculiarity● User interface– mount as a file system (only at U. Tsukuba)– grid-ftp to servers (for other sites)– direct file operations (ls, rm, mv ...) from client machine– public data download via ILDG interface– faceted navigation (ensemble search)● File system– user/group access control,cooperated with VOMS
– automatic file replica creation(among different machines)– quota

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

11

JLDG: status and statistics● Status– users: 4 research groups, O(60) users– disks: 440TB (11 servers distributed over 5 sites)– data: 3M files, 120TB– public data: 54 ensembles, 70K config’s, 12TB

● Statistics (# grid-ftp sessions, this year)
100

2719

1014

90
80
70

RCNP

Tsukuba
Tsukuba60

KEKRCNP50

Hiroshima
Kanazawa

40

Kanazawa

30
20
10Tsukuba
KEKRCNP
Hiroshima

KEKTotal

422
10637563042870

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

12

JLDG: future plan
● This FY– new JLDG sites: U. Tokyo, Nagoya U.
– install more disks ~600TB (in Kansai area)●Next FY and on– mounting JLDG file system (at several sites)– improve stability/availability– outsourcing of operation/maintenance– application to other fields– cooperation with HPCI data grid

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

13

Summary/outlook● A04 teamApproaches from viewpoints of computational physics– Algorithms– Architecture dependent techniques– Environment for data sharing– Lattice QCD common code

● Outlook– Further collaborations are necessary– Increase practical applications– Accumulate documents and code
● If you are interested in, please contact us.

– Direct contact for machine usage / collaboration– JICFuS support: www.jicfus.jp/field5/en/promotion/user/

prodides advices for programming, tuning, seeking for
better algorithms, and so on

H.Matsufuru / 3-5 Dec 2011 / Nemu-no-sato

14