30 October 2007
Review of CCS

Activities and Results
2004～2007
Akira Ukawa
Center for Computational Sciences
Executive Advisor to the President(Research and Information)

 Personal note
 From computational physics to computational
science in 2004
 Major Projects since 2004
 Toward petascale computational science1Personal note
Particle physics, in particular lattice QCD(my 1st paper in 1980, 1st simulation paper in 1982)

Involved in R&D of parallel computers since 1991






CP-PACS Project
Research for the Future Project
PACS-CS Project

FY1992-1996 (Leader Y. Iwasaki)
FY1996-2000
FY2004-2006

Director

CCP April 1998-March 2003
CCS April 2004-March 2007

Executive Advisor to the President since April 2007

Office in the Central Admin Building
Center forComputationalPhysics

1980

19851990

1995

2000

Center for
ComputationalSciences

2005

1978 PACS-9
1980 PAX-32

1989 QCDPAX
1996 CP-PACS

2006 PACS-CS

2

Focus of this presentation
The current Review concerns the period 2004-2007,i.e.,since Center for Computational Sciences (CCS) wasfounded.
So, rather than actual research results, try to provide
background on:
 The reorganization and expansion fromCenter for Computational Physics (CCP) toCenter for Computational Sciences (CCS) in 2004
 Roles of the major research projects since 2004 fordeveloping and establishing CCS
 The Japanese Petaflops Project (2006-2012) and futureof CCS

3

Three decades of R&D of massively parallel
PAX/PACS systems at U. of Tsukuba1977 research begins (Hoshino, Kawai)
1978 1st system
1996 6th system CP-PACS(top of Top500)
2006 7th system PACS-CS

2006
1996

7th gen. PACS-CS

6th gen. CP-PACS

1989

1978

1980

2nd gen. PAXS-32

5th gen. QCDPAX

1st gen. PACS-9

year

namespeed

1978

PACS-97 kflops

1980

PAX-32

0.5 Mflops

1983

PAX-128

4 Mflops

1984

PAX-32J3 Mflops

1989

QCDPAX

14 Gflops

1996

CP-PACS614 Gflops

2006

PACS-CS

14.3 Tflops

Application-driven R&D of MPP
through collaboration of

Scientists

Computer scientists

Industry

Concentrated use of computer
power on fundamental scienceproblems4 PAX

Parallel Array Experiment PACS

Parallel Array Continuum Simulation
Parallel Array Computer System

Several boards on display in the Hall5

Our stance

We have a long history of interdisciplinary research incomputational science, including HPC systems
development, by a tight collaboration between scienceand computer scienceWe believe that this approach is a key to bring out
breakthroughs in the field of computational scienceResearch activities and organizational actions at
CCP/CCS have been set up toward advancing
computational science along this approach6

Chronology of CCS (from M. Sato’s slide)1992 April

1996 OctoberNovember
1997 April

2002 April

2002

Founding of Center for Computational Physics (CCP)(10 year term / 10 faculty members and 3 visiting faculties)
development of massively parallel computer CP-PACS begins
Massively parallel computer CP-PACS completed
Ranked as No. 1 in the Top 500 World Supercomputer List
JSPS research for the Future Project “Computational Science”「Development of Next-Generation Massively ParallelComputers」 begins
The Second 10 year term of Center for Computational Physics
begins( 11 faculty members and 3 visiting faculties)
Government decides to change the National University
System in 2004

2002 July-2003Dec. Planning on reorganization and expansion of CCP
2004 April
Founding of Center for Computational Sciences (CCS)(31 faculty members and 3 visiting faculties)
2005 April
Development of Massively Parallel Cluster PACS-CS in the project「Discovery, Synthesis and Emergence of Novel Knowledge
through Computational Sciences」 begins （3years, FY2005～FY2007)
7

Change in 2004
 Prior to JFY2004
National universities were a part of the Government strictly
controlled by the Ministry of Education, Culture, Sports,
Science and Technology (MEXT) Since JFY2004
The status of the national universities was changed to thatof “independent institutions”


Pressure toward a reformation of the Government itself
Pressure toward a more effective university system in the era ofglobal competition

Budget still provided by the Government, but universities
are granted much wider freedom of action on their own
6 year cycle of PDCA;need midterm evaluation of thepresent cycle JFU2004-20098

Strategy of CCP
 CCP viewed the transition as a welcome opportunity toreconsider its future plan

A 20 year history of R&D of high performance computers,and forefront computational physics research with themA unique interdisciplinary research environment with a tightcollaboration between physicists and computer scientists

 Not only physics but a wider area of science would benefit
from the interdisciplinary approach pursued by CCP, so
reorganize and expandIn terms of science area covered
In terms of faculty University Executive Office welcomed the CCP proposal toexpand CCP, and provided strong support within theUniversity and during the negotiations with MEXT (winter2002-summer of 2003)
9Expansion to Center for Computational Sciences
Center for Computational Physics

1992-2004

Particle Phyiscs Division2

Astrophysics Division2

Condensed matter Division

2Biophysics Division2

Computer science Division

3

Center for Computational Sciences

April 2004

Particle physics and Astrophysics

6

Materials and Life Sciences

11

Geoenvironment and Biology

3High Performance Systems

5

Computational Informatics

6

11 faculties(5P/4AP/3L)

 An expansion (11Æ31 fuculty) keeping the unique feature ofcollaboration between computer science and applications
Materials and Life Science Division sizably expanded (2Æ11)

2 new divisions added (Geo-environment and Biology, ComputationalInformatics)

31 faculties（11P/12AP/8L)

 Of the 20 members added, 14 are faculty members who moved fromthe Graduate School; 6 new positions were provided by the University
and filled by new recruitments

January 2007

10

CCS within University of Tsukuba

Vice pres. general

Vice pres. education

Vice pres. finance

Vice pres. organization

Vice pres. researchmember of Councilz

University of Tsukubapresident research (budgetand faculty)

Science and engineeringLibrary and media
Sociological science

11institutes

Business science
Human science

Univ. Hospitall

Library

Center for info.and media

Advanced research alliance

Plasma research center

Center for Comp. Sciences

Pure and applied science

Life and environmental science

System and information engineering

Graduate schools

Graduate schools
Masters course
Doctoral course●●●Undergraduateschool
Research centers

University Council
directly supervised by vicez

Central Office
President
top of the research centers ofz

Major projects of CCS
 「PACS-CS」Project (2005-2007)

Computational science as a whole, in particular particle physics andmaterials/life science and computer science
PACS-CS = Successor to CP-PACS

 「FIRST」Project (2004-2007)「1st generation cosmic objects through heterogeneous multicomptersystem」
 Astrophysics and computer science FIRST = special purpose cluster with GRAPE-6 embedded nodes

 ILDG/JLDG Project （2002-）
International/Japanese Lattice Data Grid

Grid project for lattice QCD (particle physics)
Particle physics and computer science

Other projects pursued by each areas/divisions
12

PACS-CS Project
Parallel Array Computer System for Computational Sciences

Official Project title: “Discovery, Synthesis, and Emergence ofNovel Knowledge through Computational Science”

Funding: Special Fund for Education and Research of National
University Institutions (Buildup of Centers)

Amount and Period: 2.3 billion Yen over JFP 2005～2007 (3years)

Project goal: to serve as a kickoff project of CCS, in particular

Expand the interdisciplinary approach of a collaboration of scienceand computer science from particle physics/astrophysics areas,
where CCP has established itself, to materials/life sciences andGeo-environmental/biological sciences, thereby advancing theforefront of computational science

Develop a massively parallel cluster PACS-CS as a successor to CPPACS to provide a computational engine to meet the computing
needs of the applications
13Brief chronology
2003 June-AugustDecember
2004 January

initial blueprint of the PACS-CS Project discussed
Government budget for JFY2004 includes the founding of CCS
1st negotiation with MEXT toward budget application of the PACSCS Project in JFY2005

April

Formal founding of CCS

June

Formal application procedure within University of Tsukuba

June-August

Negotiation with MEXT

AugustMEXT request to Ministry of Finance for JFY2005 budget includes
the PACS-CS Project

September

Evaluation by CSTP (Committee for Science and Technology Policyof the Japanese Government )October

Announcement of results by CSTP (ranked “A” of the SABC scale)

December

Government budget for JFY2005 includes the PACS-CS Project2005 April

PACS-CS Project formally starts

14

PACS-CS Project time chart
2004 Summersetup working groups in each science target area with computational
science and started collaborative research : pacs-cs (particle physics),comas-dft (materials/life sciences)、gpv-jma(geo-environment)、mol-evol(biological science)

2005 April

official start of the PACS-CS ProjectJuly

official contract with Hitachi Ltd. for systems development

Augustofficial contract with Fujitsu Ltd. for network driver develoment
Set up Systems Working Group and Network Working Group(members from CCS and vendor engineers ) to discuss technologicalissues in R&D of PACS-CS at 2 to 4 weeks intervals

2005 Fall ～2006 Spring2006 June

System Planning/Verification/Manufacturing

full system test at Hitachi Factory/ranked 34th in the June 2006 Top
500 list (ranked 2nd after Earth Simulator among Japanese systems)

2006 July 1

full system installation at CCS/start of operation for user applications

2007 September 3 Official commissioning of PACS-CS
2007 Octoberstart of Interdisciplinary Computational Science Promotion Program15PACS-CS Project organization
 Project leader

CCS director
Akira Ukawa (2005-2006)
Mitsuhisa Sato (2008)

 Systems development leader

Taisuke Boku

 Collaborative research WGs
Pacs-cs (particle physics)

A. Ukawa et al (7+5)

Comas-dft(materials/life)

A. Oshiyama et al (7+3)



Gpv-jma (geo-environment)

H. Tanaka et al (2+5)

Mol-evol(biology)

Y. Inagaki et al (2+1)(science, computer science)
16

FIRST Project (I)Research on 1st generation celestial objects inthe Universe
Astrophysical simulation includinggravity/radiation/hydrodynamics of matter
neededDevelopment of Heterogeneous multi-computer
system (HMCS) combining GRAPE-6 for gravity
calculation and a general-purpose CPU forradiation and other calculations
100My

14By

1By

Dark age ofUniverse

50My

Big Bang

ビッグバン
Cosmicbackground radiation

1stgeneratio
n objects

reionization

galaxies
Massive
blackbol
esstars
Stars/pla17
nets

FIRST Project (II)Project leader: M. Umemura

Members:

4 astrophysicists and 4 computer scientists,
post docs, students

Funding: Specially Promoted Research in Grants-in-Aid for Scientific
Research, 0.33 billion Yen over 2004-2007

FIRST system
Total system 256nodes general purpose 3.5TFLOPS＋gravity 35TFLOPSSystem started operation in October 2006
Front view of FIRST
Blade-GRAPE board(136.8GFLOPS)

18

ILDG/JLDG Project


UK
ＵＫＱＣＤ
networkR& D of International Lattice Data Grid(ILDG) and JapaneseLattice Data Grid (JLDG) to share configuration data in the world
lattice QCD community and within the Japanese lattice QCD
community
University of Edinburgh
EPCC
University of Tsukuba
CCSFermi National Accelerator Lab
USA
ＳｃｉＤＡＣ
networkDESY
Von Neumann InstituteＷａｓｈｉｎｇｈｏｎ ＵIndiana
Edinburgh
Glasgow
Liverpool
SwanseaSouthampton

Utah
Berlin/Zeuthen
DESY/Neumann
Bielefeld
Regensburg

Future expansion to Asia/Oceania

MIT/Boston U

BNL/Columbia

ＵＣＳＢ

St．Louise

JLAB

Arizona

Germany
LatFor
networkJapan
LFT Forumnetwork

KEK

Future expansion to EU network
Italy, France, Spain, Denmark,…

ＦＮＡＬ

Kyoto

U Tsukuba

Hiroshima

19

Funding and members
 Various funding over the years
 JSPS core-to-core program (ILFTNet Project) 40 million Yen over 2004-2005

 NII(National Institute of Informatics) Cyber
Infrastructure Project About 40 million Yen over 2005-2007 Collaborations
 within CCS: particle physics (2+1 PDF) + computerscience (3) National: Tsukuba, KEK, Osaka(RCNP), Hiroshima

20

Next-generation supercomputer Project(Japanese Petaflops Project)
http://www.nsc.riken.jp/index-eng.html

 Development of a general-purpose 10-petaflops
class system by 2012 Shared use of the system by researchers in various
fields of computational science Buildup of a center for research and education incomputational science around the petascale system
in Kobe Continual R&D of supercomputers(i.e., next, next-next, …) as a key
national technology envisaged21Involvement of CCS“Element technology for next-generation supercomputer”
Project

National project funded by MEXT through JST
Collaboration with Hitachi Ltd. on low-power on-chip memoryarchtecture for next-generation systems

Collaboration with RIKEN on the petaflops system developmentFormal agreement between U. Tsukuba and RIKEN signed in
September 2006
Participation of several CCS faculty in the system design as RIKEN
Visiting Researchers
Collaboration in tuning ofrepresentative petascale
applications(QCD and RS-DFT)
Members in various committees22

Once more, Our stance, and …


We have a long history of interdisciplinary research incomputational science, including HPC systems development, by aclose collaboration between science and computer science

We believe that this approach is a key to bring out breakthroughsin the field of computational scienceResearch activities and organizational actions at CCP/CCS havebeen set up toward advancing computational science along thisapproachWe wish to have your assessment and advice on how well we havebeen doing in this approach since 2004, and how best we cancontinue to do well and continue to be a key organization
nationally and internationally in computational science in the
coming era of petascale computing
23