Report of the 2016 RIKEN AICS Advisory Council
Meeting of the Advisory Council of the RIKEN Advanced Institute for
Computational Science (AICS), August 29 – August 31, 2016, Kobe, Japan

1. Preliminary Information
Members of the AICS ACJack Dongarra
Prof. Dr.
University of Tennessee, USA
Computer Science
Kozo Fujii
Prof. Dr.
Tokyo University of Science , Japan
Deputy ChairEngineering
William Kramer Dr.
Director, Blue Waters, NCSA, USA
Computer Science
Thomas Lippert Prof. Dr. Dr. Director, Jülich Supercomputing Centre, Germany
Chair
Particle Physics
Haruki Nakamura Prof. Dr.
Osaka University, Japan
Life Sciences
Hiroshi Niino
Prof. Dr.
Atmosphere and Ocean Res. Inst., University of Tokyo, Japan Atmospheric Science
Satoshi Sekiguchi Dr.
Director General, Nat. Inst. of Adv. Ind. Science & Tech. , Japan
Computer Science

Information provided to the AICS AC
−
−
−
−
−
−

−
−

Meeting Program of AICS AC 2016
White Paper of AICS AC 2016 (Basic Information, Overview, Future Plans, Appendix)
Activity Report of AICS AC 2016 (Operation and Computer Technologies Division)
RIKEN AICS Brochure (K Computer: System, Facilities, Research, Future, Post K)
AICS’ Research Teams Brochure (Research Highlights of R&D Activities)
Presentations of RIKEN and AICS Leading Scientist: Yoichiro Matsumoto, RIKEN Executive
Director; Kimihiko Hirao, AICS Director; Fumiyoshi Shoji, Director of Operations und Computer Technologies Division; Akira Ukawa, Director of AICS Research Division and Deputy
Director of AICS; Yutaka Ishikawa, Project Leader of Flagship2020 Project;
Presentations of Research Teams
Interviews with Team Leaders and Researchers

Tasks of the AICS AC
The AICS AC is tasked with reviewing the activities of AICS from 2010 to 2016 following the ToR
for review as laid out by the RIKEN President, which is supplemented by the ToR laid down by
the AICS Director. Both ToR are included in the White Paper of the 2016 AICS AC.
Information and recommendations from previous reviews
Since the RIKEN AICS was founded on July 2010, no previous review of the AICS by the AC under
the RAC framework has taken place. Hence, the present review is the first one.
Glossary
AC
AICS
HPCI
JLESC
MTBF
MTTR
PUE
RAC
RIST
ToR

Advisory Council
Advanced Institute for Computational Science
High Performance Computing Infrastructure
Joint Laboratory on Extreme Scale Computing
Mean Time between Failures
Mean Time to Recovery
Power Usage Effectiveness
Riken Advisory Council
Research Organization for Information, Science and Technology
Terms of Reference12. Introduction
The meeting of the Advisory Council of the RIKEN Advanced Institute for Computational Science(AICS) took place from August 29 to August 31, 2016, in Kobe, Japan. The period under review,
2010 to 2016, enjoyed the internationally celebrated highlight of the inauguration of the K
Computer System of 2011, from this time on for two periods being at position one of the fastest
supercomputer systems worldwide on the Top500 list, and being present among the top 5
ever since then. This success is not accidentally; as corroborated by the first position of the K
computer in the Graph500 list and the second position in the new HPCG benchmark, it
demonstrates Japan’s strategic vision as to the K computer together with high operational quality
and impactful accompanying research at AICS.
In section 3 of this report, the AC presents the evaluation of AICS. Section 4 conducts a SWOT
analysis of strengths, weaknesses, opportunities and threats as distilled in Section 3. In section
5, a summary of specific measures is given, while, Section 6 lists the AC’s recommendations.

3. Evaluation and Findings
In the following, the evaluation of AICS by the AC is described in the order of the ToR of the AICS
director. The report of the AC is taking into account the ToR of the President of RIKEN in theevaluation comments.

3.1

Operation and Management of the K Computer

The first major objective in the threefold mission of the AICS is “to operate the K computer andto provide its top level resources to users both in academia and industry”. These tasks comprise
operation, management and R&D for the enhancement of the system along with the peripheral
facilities like networks system, air conditioning, chillers, and power generators. This objective
entails in addition the technical coordination of the HPCI system operation along with R&D for
its enhancement.
First of all, the AC acknowledges that the AICS Operation and Computer Division is delivering
outstanding work in operating, managing and improving the K computer, having reached aworld class level. Given the sheer size and complexity of the machine as well as of its
technical and support periphery, the AC considers the well balanced division structure with24 technicians and researchers for this task supported by about 30 technicians of Fujitsu as
fully adequate, absolutely necessary and as the major strength of the division. The users enjoy
high system availability with an MTBF of more than 18 days, excelling most leadership classsystems in China, Europe and the USA. At the same time, its MTTR is only about 10 hours. In
addition, a careful tuning and optimization of power generation, consumption and heatreuse has been successfully achieved, leading to a remarkably low PUE. The development ofa smart energy management system is a major opportunity that the AC encourages the teamto pursue.
The division has managed by various means, e.g. resolving the early time file system problems
and introducing asynchronous file staging, to boost the job filling rate from 60% to 75%, a fully
adequate value, given the more effective block-wise node allocation chosen. Still, the ACconsiders it as an important opportunity to continue the efforts in improving the job filling rate
without compromising the capability of the system. This might be realized by a
combination of improved back filling methods with a closer involvement of the users in themonitoring of the system, by offering a 3D realistic web based system allocation panel.
The AC is concerned about the apparent separation of AICS with respect to the end users of the
K computer. The AC sees this separation as a structural weakness that concerns both the AICS
Operation and Computer Division and the AICS Research Division. The AC has the feeling that
only an indirect communication channel to the users via the RIST 2nd level support team isestablished at this time. Both, 1st level support as provided by the AICS Operation and Computer2

Division, as well 3rd level support as provided by the AICS Research Division throughcommunity oriented methodological research would strongly profit from more transparent anddirect ways of user feedback. A unified trouble ticket system might be a first step in thisdirection. The positive effects on the co-design activities of the Flagship2020 Division throughdirect user feedback with the AICS teams seem evident.
While one might consider the number of publications per person of this division as a weaknessof the division, the AC recognizes that technical work very often must be prioritized against
research publications. The AC welcomes further that the division is prepared to increase their
publication rate according to their self-assessment, which is certainly in the genuine interest of
the majority of team members with fixed time contracts.
Indeed, the AC is concerned about limited permanence of positions, which poses some threat ofloosing or not being able to attract good people in the future, especially in the operational sector.
This was discussed in the interviews of researchers.

3.2

Research in Computer Science and Computational Science

The second major objective of the mission of the AICS is “to carry out leading edge research
toward breakthroughs in science and technology through the close cooperation ofcomputational science and computer science”. This comprises “research of basic technologies
for advancing the usage of the K computer” and “advanced research in computer andcomputational sciences”. As detailed below in the summary of the team leader presentations,this young division is able to present a huge mass of very impressive results in research anddevelopment for many use cases. On the one hand, the algorithms, tools, models and methods
developed to advance the usage of the K computer, funded through MEXT’s grant-in-aid, enjoy asteadily increasing utilization by the user community (including more than 140 companies). In
fact, the AC is absolutely convinced that the performance improvements reported pay off by
factors through a much more effective exploitation of the system by the users of thesesoftware instruments, compared to the expenditure required to fund about 110 researchers in
the 16 teams of the division.
On the other hand, all these developments created are, at the same time, investments into the
future, as they also help defining scientific requirements on the post K architecture, see section
5.3, and will be available for the post K system right from its start. 18 computer scientist and 14
computational scientists from the Research Division are also involved in the activities of the
Flagship2020.
What is more, the advanced research of the division in computer and computational sciences, cofunded through bottom up external funding by the Hyogo Prefecture and the City of Kobe,
competitive funding through JST, JSPS, MEXT, and a small management expenses grant by MEXT,
has considerably increased AICS’ excellence in science.
The unifying paradigm and unique feature of the division’s research strategy is the living
interaction between computer science and computational science, which is – to the satisfactionof the AC – realized in a multitude of activities between the fields. Many scientific highlights, 32
codes at full scale on the K computer and an impressive number of papers and citations prove thestrength of the division.
The group has created important cooperations with the leading supercomputing centers world
wide showing its international standing; one example is the JLESC consortium. Quite a few
international scientific symposium have taken place in the reporting period.
Finally, the AC would like to comment on weaknesses, opportunities and threats for the division:
Smaller weaknesses appear in the educational activities: the training would certainly have higher
impact if it was held in English and transmitted via web broadcast. A systematic visitor’s program
would be desirable as well.

3

While there are discussions on building a Software Center, at this time there is no tangible longterm software maintenance plan, which is considered as a weakness of the AICS.
Data science is ever deeper dominating the entire field of science. The AC considers this an
important opportunity for the AICS. Given the huge activities in the direction of data sciences inJapan and worldwide, the absence of an environment and support structure that enables data
science focused projects on the K computer, including machine learning, is considered as a threat
for the AICS. In fact, the SWOT analysis in Section 6 shows that the AICS can easily compensate
threats by its strength and is able to turn weakness into pursuing opportunities.

3.3

Development of the Post K Computer

The third major objective of the mission of the AICS is “to lead the development of Japan’s
future strategy for computational science, including the development of the follow up to
the K computer”, i.e., the so-called Flagship2020 project. The ACs judgment is mainly
based on information not being under nondisclosure, as given in the division leader’spresentation. From this information, the AC is impressed by the progress in the design
efforts achieved. The AC is convinced that expertise and experience in AICS is a huge strength
and will enable it to successfully carry out its Flagship2020 responsibilities todevelop the post K supercomputer together with the right combination ofcommunities, skills, and collaborations. The selection of ARM as the core technology is
an innovative chance for AICS and potentially will have great impact in the entire HPC
ecosystem. The post K system based on ARM gives AICS
the opportunity for much larger
community impact and therefore should be highly visible. It is however important, to startacquiring competence with ARM processor technology for a broad range of tools andapplications codes very soon.
It is foreseen that other systems in HPCI will help to fill the provision gap when the machines
are replaced. A delay in the provision of post K certainly poses a risk to Japan’s science
infrastructure. The AC thinks, that longer delays could be compensated by provisioning anintermediate system with similar capability as the present K at much smaller footprint andmodest expense. Furthermore, the Flagship2020 project encompasses the development of awide range of application codes to run on post K, in order to solve major social and scienceissues. Nine co-design fields have been selected, ranging from protein sciences via weather
prediction to fundamental physics.
The AC acknowledges the progress achieved and welcomes the co-design as a most effectivemeans to pursue an application oriented development of post K.

3.4

Feedback from Staff Interviews

The AC has asked for having a direct unbiased discussion with staff. The AC considers thesituation, the concerns, the constructive proposals and suggestions of younger team leaders
and researchers as very important for inclusion into this evaluation, to sharpen therecommendations. The conclusion from the interviews is provided to the AICS AC management inform of a separate document.

5

4. SWOT Analysis

SWOT

Internal

External

HelpfulHarmfulStrengths

Weaknesses

1. Strong and well balanced
operation and technology teams

1. Only indirect interaction with end
users via RIST

2. Excellence in computer andcomputational science

2. Competences for data curation anddiscovery are underdeveloped

3. Convincing R&D structure andorganization for post K

3. ARM needs considerable adaptionof existing codes

Opportunities

Threats

1. User transparent monitoring can
improve K’s effectiveness1. Decreased interest of young technicians w/o permanent positions

2. Supporting data science will
expand the impact of K systems

2. Only few users in societally relevant
data sciences (medicine etc.)

3. ARM processor opens door towider community

3. In case of delay of post K, there is a
detrimental gap in capability

The AC focuses its SWOT analysis on a few relevant topics in order to derivestrategic advice. The discussion follows the numbering in the table for each of the five items:
1. Operation and Management
The AC considers the outstanding operation and technology teams an important strength of AICS.
The interviews made evident, that the members of staff want to be able to plan their carrier at
AICS, as it is harder to find new job opportunities for technicians than for researchers. Some morepermanent position would mitigate this risk.
The AC considers the purely indirect interaction with users as a weak point. Measures likemore transparent web based monitoring or a uniform User Service Request Ticket Systemaccessible by all AICS staff are perfect opportunities to improve on the user interaction andimprove effectiveness like the job fill rate.

2. Research
AICS excellence in computational and computer science is a good starting point to engage
in attracting more users in emerging data sciences, such as medical imaging or other fields
relevant for society.
The AC considers data science an important opportunity to enhance the impact of the K
systems. An important step can be accomplished by developing proper data curation competences.

3. Post K
A delay in the provision of post K is considered less probable, given the strength of theconsortium. Still it is a threat of high impact. While HPCI is foreseen to fill the unavoidable
provision gap when the machines are replaced, longer delays could be compensated byprovisioning an intermediate system with similar capability as the present K,
at much smaller footprint. While the transition to ARM requires substantial adaption ofexisting codes, this weakness will turn into an opportunity, as the new processor technology will
attract a much broader scientific ecosystem.
For all three fields discussed in the SWOT analysis, weakness can be turned into opportunityand strength can immunize the AICS against risks.
6

5. Mid to Long Term Policy (5 to 10 Years)
The AC strongly encourages the AICS to continue to strive being a world classsupercomputing center as a hub for science and technology innovation by doing leadingedge research of basic technologies for advancing the usage of the K systems as well as
advanced research in computer and computational sciences to increase excellence. The ACrecognizes that the AICS is following the five strategic topics of the initiative on scientificexcellence of the RIKEN president and recommends continuing in this direction in a ten
years perspective.

5.1

Comments on the Future Plan of AICSThe AC fully endorses the future plan of AICS, presented by Director Hirao. The plan isbased on the three mission statements of AICS as presented in sections 5.1, 5.2 and 5.3 ofthis document. The AC is convinced that, following the plan, AICS will develop a worldwide
leading large scale supercomputer system in Japan, i.e. the post K supercomputer, AICS is will
become a world hub for computer and computational science, and AICS will foster
breakthroughs in simulation, ensemble computations, data analytics, and data
assimilation. The AC is looking forward to seeing how this vision will materialize.

5.2

Policy on Software Sustainability

The impact of the work at the AICS on novel algorithms for the next generation
Exascale computers can be greatly increased if key computational kernels developed at
AICS can be made available as open source software to be used by the community atlarge. Development of such software can provide great visibility and reputation to the
AICS and help cement the reputation of RIKEN as a “destination”. However, this isbest done by professionals specialized in the development of high quality software
rather than graduate students or post docs, whose focus, rightly, is on advancing
research. Even though hiring professional software engineers can be quite expensive,
the AC highly recommends allocating resources to address this need.

5.3

Policy on Data Science

While the AICS has focused on simulation and key mathematical tools such as linear
systems solvers, there is an excellent opportunity to expand the focus substantially into thegeneral area of data analytics. This is important for a number of reasons as outlined below.
Progress in science and engineering is increasingly dependent upon our ability to analyze
very large data sets that are becoming available due to advances in sensors and instruments
as well as computational simulations. This is often referred to as the fourth paradigm ofscientific discovery, with theory and experimentation being the first two paradigms andcomputational simulation as the third paradigm.
Even the massive amounts of computational simulation output create opportunities fornovel data analytics. In particular, novel algorithms are needed to extract patterns fromspace time data sets that can identify features and regions of interest (e.g., vortices in a CFD
simulation) and even help steer the computational simulations.
AICS’s expertise in the development of algorithms that are power efficient and minimize
data motion (as opposed to arithmetic operations) can be very valuable in improving a hostof tradi-tional data analytics algorithms including those that are not based on linear algebra(e.g., association pattern mining, k means clustering, and deep learning kernels).
In addition, the AICS should carry out further efforts to bridge the integration of simulationand experimental science by data assimilation, in order to accelerate discovery across many
areas of science using the K systems.

7

6.

Response w.r.t. ToR of the RIKEN President

The ACs has received evaluation criteria by the RIKEN President: 1. Does the Center
possess R&D achievements and personnel meeting international standards, produce worldclass results and benefits for society, what is the standing in the field along with strengths/
weaknesses and a policy for 5 to 10 years? (Criterion 2. not applicable); 3. Alignment of theactivities with five strategic goals, prospects to fulfill them and new policies: (1) pioneer aresearch management model – (2) lead the world in R&D – (3) become a hub for S&T – (4) befocal point for global brain circulation – (5) foster world class leaders in research; 4. How
effective and appropriate does AICS maximize RIKEN’s achievements as a whole, including
collaboration between the RIKEN centers.
1. The AC is impressed by all AICS divisions delivering world class results with respect tointernational standards: (i) AICS’ Operation and Computer Division is an international rolemodel as to operating, managing and improving the K computer; (ii) AICS’ Research in Computerand Com-putational Science Division is pushing the effectiveness of the K computer throughalgorithms, software and libraries, is highly ranked in the international TOP500 andGraph500 lists, and shows impressive numbers of papers, many in societally relevant fields;(iii) AICS’ Flagship2020 Project Division brings together strong expertise in AICS and the rightcombination of communities and collaborations as a sound basis for co-design. As to theknowledge of the AC, there is no comparable co-design project worldwide. Besides strengths,
the AC could find only few weaknesses and risks for AICS. By means of a SWOT analysis, theAC became convinced that any weakness can be turned into an opportunity and that its
strengths immunize the AICS against risks.
2. Not applicable.

3. The AC acknowledges that AICS is closely following the five strategic topics of the initiative on
scientific excellence of the RIKEN President and recommends continuing on this path: (1) The
tripartition of the AICS and the paradigm of interaction between computer and computational
science enables most effective R&D management at AICS. The AC gives recommendations 1 and
2 for closer interaction with the user communities as a new policy; (2) The AC recognizes highflexibility at AICS in pioneering new research, e.g. data assimilation. In addition, the AC gives
recommendations 3 and 4 to establish a Software Center and to pursue methodologies for Data
Science including Machine Learning; (3) The AC sees RIKEN AICS as existing hub of technology
and innovation, e.g. proven through co-design with industry for the K/post K systems. The AC
proposes recommendation 5 to sustain sufficient attraction of excellent developers; (4) In recommendation 5, the AC advices to introduce international recruiting programs; (5) International
cooperation with world class centers exist, including an exchange program (JLESC), demonstrating AICS’ pursuit of international brain circulation. In recommendation 7, the AC proposes tointernationalize training at AICS by using English as the language and broadcasting via the web.
4. The AICS substantially contributes to RIKEN’s achievements and visibility from both a national and an international perspective. The exponentially growing field of data science is a unique
opportunity for RIKEN to utilize AICS’s resources and competences for RIKEN as a whole.
The AC congratulates RIKEN for having established AICS as a marvel of the Japanese research
infrastructure, with a focal role in the HPCI ecosystem and highest worldwide reputation.

7.
8.

9.
10.

11.

12.

13.
14.

Recommendations
The AICS has the responsibility for the quality of usage of the K computer. The AC
believes, that this entails to have direct interaction with the end users. A uniform User
Service Request Ticket System for all aspects of the K computer service, comprising users,
AICS and RIST personnel, would be an effective first step to improve the integration of
support activities.
Additionally, the AC encourages AICS to set up a yearly workshop of K computer users,
AICS experts and RIST support experts for open discussion on important operational
topics.

AICS has established system and application software development activities for a variety
of software capabilities proven relevant for the HPC user community. In order to create
longterm impact, the AC endorses to establish support structures for maintenance andimprovement of these software products like the planned Software Center. In this manner,
AICS will gain international recognition as a home for important community based
software capabilities.

The AICS should start to substantially extend data science activities during the remainderof the K computer operation in order to actively push the convergence of simulation anddata analytics – expected by many international experts – for the post K phase, thustranscending existing research fields. In particular this concerns provisioning novel data
analytics and ma-chine learning services, and the suitable design of post K software andwork flow environments to support data science workloads.

AICS faces work force recruitment and retention challenges within the fields ofcomputational science that are relevant to the mission of AICS. The AC recommends
establishing three high profile programs: (i) a highly prestigious postdoctoral fellowship
program in the areas of computer science and computational science; (ii) programs toengage more graduate students within the structure of AICS; (iii) a robust visitor program.
As a general recommendation, valid for any similar institution, AICS should implement
succession planning and leadership development activities at all levels of staff.

The AICS should carry out training courses in English language as a means forinternationalization and in addition should broadcast the courses on the web.

Acknowledgments

The AC members would like to thank the AICS scientists and administration for a very effective
organization of the review and the friendly, constructive and pleasant atmosphere during
the review. Furthermore, they would like to thank the RIKEN for having established AICS as amarvel of the Japanese research infrastructure, with a focal role in the HPCI ecosystem andhighest worldwide reputation.
Jülich, October 15th , 2016

Prof. Dr. Dr. Thomas Lippert
on behalf of the AICS Advisory Council, Prof. Dr. Jack Dongarra, Prof. Dr. Kozo Fujii, Dr. William
T. Kramer, Prof. Dr. Haruki Nakamura, Prof. Dr. Hiroshi Niino, Dr. Satoshi Sekiguchi
8