XcalableMP(XMP) https://xcalablemp.org
What’s XcalableMP (XMP for short)?
 A PGAS programming model and language for
distributed memory , proposed by XMP SpecWG
 XMP Spec WG is a special interest group to
design and draft the specification of XcalableMP
language. It is now organized under PC Cluster
Consortium, Japan. Mainly active in Japan, but
open for everybody.

Project status (as of June 2019)
 XMP Spec Version 1.4 is available at XMP site.
new features: mixed OpenMP and OpenACC,libraries for collective communications.
 Reference implementation by U. Tsukuba andRiken R-CCS: Version 1.3 (C and Fortran90)
is available for PC clusters, Cray XT and K
computer, and Fugaku. Source-to-Sourcecompiler to code with the runtime on top of MPIand GASNet.
 HPCC class 2 Winner 2013, 2014Language Features
Directive-based language extensions for Fortran and C forPGAS model
Global view programming with global-view distributed datastructures for data parallelism

SPMD execution model as MPI
 pragmas for data distribution of global array.
 Work mapping constructs to map works and iteration with affinity todata explicitly.
 Rich communication and sync directives such as “gmove” and“shadow”.
 Many concepts are inherited from HPFThe spec of XcalableMP 1.x is now converged.
We are now moving to XcalableMP 2.0 with global
task-based parallel programming and PGAS

Co-array feature of CAF is adopted as a part of the language
spec for local view programming (also defined in C).
int array[YMAX][XMAX];

Code example#pragma xmp nodes p(4)#pragma xmp template t(YMAX)#pragma xmp distribute t(block) on p
#pragma xmp align array[i][*] to t(i)main(){
int i, j, res;
res = 0;

data distribution

add to the serial code : incremental parallelization#pragma xmp loop on t(i) reduction(+:res)for(i = 0; i < 10; i++)for(j = 0; j < 10; j++){work sharing and data synchronizationarray[i][j] = func(i, j);
res += array[i][j];}}3XcalableMP as evolutional approachWe focus on migration from existing codes.


Learn from the pastSpec WG is organized under the PC Cluster
Consortium, Japan.

Design based on PGAS model andCoarray (from CAF)

XMP applications Experiences

 IMPACT-3D: 3D Eulerian fluid code, which performs

compressible and inviscid fluid computation to simulate
converging asymmetric flows related to laser fusion (NIFS) RTM code: Reverse-time Migration Method for Remote

Sensing applications (Total, France)

PGAS is an emerging programming model for
exascale!

 SCALE-LES: Next-generation Climate Code developed byUsed as a research vehicle forprogramming language/model research

 GTC-P: Gyrokinetic Toroidal Code , which is a 3D PIC codeGlobal-view for data-parallel apps. Japanese community had experience of HPF for Global-view model.

Specification designed by community

Directive-based approach to enable parallelization by adding directives/pragma.
Also, should be from MPI code. Coarray may replace MPI.



XMP 2.0 for task programming and PGAS
Extension to accelerator (XACC)AICS Tomita’s Teamto study the micro turbulence phenomenon in magneticallyconnected fusion plasma (Princeton Univ. and Univ. Tsukuba)

CloverLeaf: a hydrodynamics mini-application



Fiber mini-apps

Performance of XcalableMP on Fugaku
 XcalableMP was taken as a parallel

programming language project for improving
the productivity and performance of parallel
programing.

 XcalableMP is now available on Fugaku and theperformance is enhanced by the Fugaku
interconnect, Tofu-D.

QCD (Local-view, Coarray)

256

512

128

XMP

256

MPI

128

Speedup (MPI/K on 2 = 2)

Speedup (MPI on 8 = 8)

512

64
32
168

XMP/K
MPI/K
XMP/Fugaku

64

MPI/Fugaku

32
1684832128# XMP nodes

Impact-3D (Global-view, Directives)
Fusion simulation code, stencil apps

51222481632# XMP nodes

NT-Chem (Local-view, Coarray)64