XcalableMP
Directive-based language eXtension for Scalable Parallel ProgrammingOverview
What’s XcalableMP ?

Implementation Status
XcalableMP specification ver. 1.0 is available

XcalableMP is a directive-based PGAS language for
distributed memory system

Omni XcalableMP compiler ver. 0.5.3 is available
from University of Tsukuba

Designed by XcalableMP Specification Working
Group

Download from http://www.xcalablemp.org
Supported platforms are linux cluster, Cray platform, ..
Interface of Scalasca & tlog profiling tools
For accelerators(GPU, etc)
XcalableMP Parallel I/Owork in progress
Interface of MPI library

Members from academia (U. Tsukuba, U. Tokyo, Kyoto U., andKyusyu U.), research labs(RIKEN, NIFS, JAXA, andJAMSTEC/ES), and industries(Fujitsu, NEC, Hitachi) in Japan

Omni XcalableMP compiler is developed in "Seamlessand Highly-productive Parallel Programming Environment for High performance computing" project fundedby MEXT in JapanFor K computer
XcalableMP will be used to program to K computer

Programming Model
Language Features

Global-view Programming

Local-view Programming

Language extension of C99 andFortran 95
Node0
Node1
Node2
SPMD as a basicDirectivesexecution model
Communication,synchronization,and work-mapping occur when
directives are encountered
All actions are taken by directives for being “easy-to-understand” in performance tuning(different from HPF)

Supports typical parallelization
based on the data parallel paradigm and work mapping

Also includes Co-Array Fortranlike feature

Example: a[12] is distributed onto 4 nodes

Comm, sync, and work-mapping

int a[12];#pragma xmp nodes p(4)#pragma xmp template t(0:11)#pragma xmp distribute t(block) onto p#pragma xmp align a[i] with t(i)#pragma xmp loop on t(i) reduction(+s)
for(i = 1; i < 10; i++) {a[i] = func(i);s += a[i];}global index 0 1 2 3 4 5 6 7 8 9 10 11

Node 0Node 0Node 1

Node 1

Node 3

Node 3Distributed Arraydouble a[5], b[5];#pragma xmp coarray a // Declartion:b[0:2] = a[3:2]:[1];// Communicationa[]b[]

Node 0Node 1

Extends C for an array section
array_name[start : length[:step]]:[node_number]

Node 2

Node 2

Example: Declaration & communication of coarray

Each node computes red elementsin parallel

Many concepts of XcalableMP
are inherited from HPF

The array_name[start:length]:[node_number]means elements from the array_name[start]
to the array_name[start+length-1] located oncompute node whose name is node_number

Benchmarks
Laplace Solver by Global-view#pragma xmp reflect uu#pragma xmp loop (x, y) on t(x, y) threadsfor(y = 1; y < YSIZE-1; y++)for(x = 1; x < XSIZE-1; x++)u[y][x] = (uu[y-1][x] + uu[y+1][x] +uu[y][x-1] + uu[y][x+1])/4.0;

Define shadow area and its width
Specify additional thread parallelization
XcalableMP also supports hybrid parallelization for multicore cluster

Synchronize data only on shadow area
300
250
200

100503264128
256
Number of CPUs(x 4Cores)

2500
2000
1500
1000
500
0

Exchange data by using Co-array

XcalableMP
MPI

163264128
Number of Cores

For more information, please visit

multi-threaded XcalableMP
flat MPI

1500int key[SIZE];#pragma xmp coarray key:#pragma xmp barrierfor( i=0; i<comm_size; i++ )key[recv_displ[i]:count[i]]:[i]= buff[send_displ[i]:count[i]];Performance(Mop/s)

Define two-dimensional process grid

Performance(GFlops)

double u[YSIZE][XSIZE], uu[YSIZE][XSIZE];#pragma xmp nodes p(N_Y, N_X)
#pragma xmp template t(0:YSIZE-1, 0:XSIZE-1)#pragma xmp distribute t(block, block) onto p#pragma xmp align u[y][x] with t(x, y)#pragma xmp align uu[y][x] with t(x, y)#pragma xmp shadow uu[1:1][1:1]:#pragma xmp loop (x, y) on t(x, y) threadsfor(y = 1; y < YSIZE-1; y++)for(x = 1; x < XSIZE-1; x++)uu[y][x] = u[y][x];Integer Sort of NPB by Local-view

512

T2K Open Supercomputer Alliance (#5007@Level 6)
Center for Computational Sciences, University ofTsukuba (#923@Level 4)
http://www.xcalablemp.org

256