AI with Super-computed Data forMonte Carlo Earthquake Hazard Classification
Tsuyoshi Ichimura1,2 , Kohei Fujita1,2 , Takuma Yamaguchi1 ,Muneo Hori1,2 , Maddegedara Lalith1,2 and Naonori Ueda3
1Earthquake Research Institute & Department of Civil Engineering, The University of Tokyo
2
Advanced Institute for Computational Science, RIKEN
3
Center for Advanced Intelligence Project, RIKEN

Abstract—Many problems associated with earthquakes are yet tobe solved using heroic computing, which is defined as computing at
the largest scale possible using the best supercomputers and algorithms. Thus, a continuous effort has been pursued in HPC to solvethese problems. However, even when heroic computing is applied,
its practical use is difficult without considering the uncertainties inmodels. In this study, we constructed an AI methodology that uses
super-computed data generated using heroic computing. We applied
this AI to an earthquake hazard classification including uncertainty
analyses in order to demonstrate its utility. This study can be regarded
as an innovative step towards realizing high quality computing for
Earthquakes by exploiting the potential of HPC through AI.

I.

I NTRODUCTION

Evaluation of earthquake damage scenarios (i.e., estimationof damage to cities and buildings) is required to effectively
mitigate the devastating effect of earthquakes. Knowledge ofthe distribution of earthquake shaking (i.e., the earthquake
hazard) at the ground level of whole cities with an area of
100 km2 is required to generate these damage scenarios (e.g.
[1], [2]). Because the soil at a depth of 100 m from the surface
consists of soft soils having complex geometry that changes
every 10 m, ground shaking due to earthquakes becomes highly
nonlinear and varies significantly with local position (e.g. [3]).
To improve earthquake mitigation scenarios, earthquake
hazard maps of whole city areas should be developed at a very
high resolution (less than 50 m). An approach to generate suchhazard maps is to use observed ground shaking data; however,this approach is difficult to implement at such a high resolution.
Three-dimensional ground information is currently being accumulated for whole urban areas with the objective of improving
earthquake hazard distribution estimation (e.g., [4]). Simulations that use such data can be useful for generating hazard
maps; however, this requires a very large-scale analysis, which
will be very challenging to accomplish. In addition, subsurface
ground cannot be measured directly and will therefore not behighly accurate. The uncertainty of this data must be quantified
in the hazard maps. As ground responds highly nonlinearly
during earthquakes, conducting Monte Carlo simulations aresuitable for considering uncertainty of data; however, this iscurrently impractical because it requires 102 – 103 repetitionsof a large-scale analysis.
In this study, we develop a practical approach to enable
uncertainty analysis of large-scale analyses. Here we enable
the above super-large scale analysis by heroic computing, anduse this super-computed data to construct an AI methodology.
This AI methodology is applied to conduct Monte Carlosimulations to estimate the uncertainty of the results. Even
if HPC were to enable large scale analysis with verifiedresults, this would not be sufficient to qualify as high-quality
computing (HQC) without the estimation of uncertainty. By
considering uncertainty thorough AI, the importance of HPC
that enables heroic computing is strengthened. Thus HPC cancontribute towards accomplishing HQC for Earthquakes.
II. AI WITH S UPER - COMPUTED DATA FOR M ONTEC ARLO E ARTHQUAKE H AZARD C LASSIFICATION
To analyze the distribution of surface ground shaking in acity, it is necessary to obtain the solution to the nonlinear wave
equation for the ground for the whole area, i.e., Eq. (1). We
use a low-order unstructured finite element method because it
analytically satisfies the traction-free boundary condition andis suitable to accurately model complex ground structures withstrong local nonlinearity. We use implicit time integration tokeep the analysis stable. Because low-order elements are used,
this becomes a challenging computation that is dominated bysparse matrix computation with random access.
()4/dt2 M + 2/dt Cn + Kn δun =()f n − qn−1 + Cn vn−1 + M an−1 + 4/dt vn−1 , (1)where, δu, u, v, a, q, and f are incremental displacement,
displacement, velocity, acceleration, internal force vectors, andexternal force vectors, respectively; M, C, and K are theconsistent mass, damping, and stiffness matrices, respectively;
and dt and n indicate the time step increment and time step
number. For Tokyo, Eq. (1) becomes the order of 0.1 trillion
degrees of freedom (DOF), which requires to be solved foreach of the 6,000 time steps. This size is considered to bevery challenging.
To enable such a massive analysis, we developed a fastand scalable solver [5] using the adaptive conjugate gradientmethod, mixed precision arithmetic, a multi-grid method, anelement-by-element method, structuring of an unstructured
mesh, and steady and contiguous memory access computation.
This solver attains high performance up to the full K computer
system with 663,552 CPU cores on problems with up to 1.08
trillion DOF. This study was nominated as an SC15 Gordon
Bell Prize finalist and is presently considered to be state-of-theart analysis. Using this supercomputing method, we analyzed
earthquake ground shaking in a 10,250 m × 9,250 m area ofCentral Tokyo. This is the state-of-the-art heroic computation
with 0.133 trillion DOF and 6,600 time-steps, which have beensolved using the full system of K computer with a maximum
user time of 8 hours. A large domain with strong nonlinearity
was solved at a high spatial resolution of 1 m; the total data
size analyzed was 0.133 trillion DOF × double precision ×
6,600 time steps = 6,600 TB.

7000

7000
166000145000

76000

65000
5124000

4000

43000

3103000

82000

62000

1000

41000

2
10

00Fig. 1.

2000

4000

6000

8000

02000

4000

6000

8000a) Distribution of meanb) Distribution of 3σMonte Carlo earthquake hazard classification using an AI with super-computed data. Both the mean and 3σ (σ is standard deviation) differs locally.

It is not practical to conduct 1,000 cases of heroic computing to estimate the effect of uncertainty of the groundstructure. Thus, using super-computed data, we constructed anAI methodology that classifies ground motion severity and use
this AI to conduct Monte Carlo simulation. Here, the groundmotion is classified into 20 levels for 50-m grid points inside
8,350 m × 7,200 m (24,360 points) area of Central Tokyo.
It is essential that the developed AI methodology possesses
the ability to perform classification with high correct ratios.
Generally, nonlinear wave propagation problems are sensitive
to the structure within one wavelength of the observation
point. Thus, we assume this physical point of view to selectand normalize data for training a highly accurate AI system.
Here, we conducted training by Chainer [7] on Pietra (SGI,Tesla P100 × 4, Intel E5-2690 v4 × 2). Using this AI, we
conduct Monte Carlo earthquake hazard classification. The
target ground structure consists of three layers; we change
the geometry of the interfaces between the first and secondlayers and that of the interface between the second and thirdlayers using a normal distribution with a mean of 0 m and astandard deviation of 5 m on a 1,000 m grid. We conduct aMonte Carlo simulation with 1,000 samples on this stochastic
model using the constructed AI. Fig. 1 shows the mean andstandard deviation of the obtained distribution. We can see thatboth the mean and the standard deviation differ largely fromlocation to location. Such information cannot be obtained bya single case of heroic computing; thus, the distribution was
first evaluated via uncertainty analysis using AI; this showsthe effectiveness of our method. Because construction of an AI
methodology with high correct ratios requires super-computed
data obtained from heroic computing, the contribution from
HPC was essential for enabling this analysis. From this pointof view, we can say that we have revealed the potential of
HPC with the development of such an approach.
III.

C ONCLUDING R EMARKS

We performed heroic computing using our proposed supercomputing method and developed an AI methodology basedon the the obtained super-computed data; this methodology
was made highly accurate by selecting and normalizing thetraining data with the consideration of the underlying physicalcharacteristics of the ground motion problem. We used thismethodology to conduct a Monte Carlo earthquake hazard
classification procedure and demonstrated the effectiveness of

our methodology. In the future, we plan to improve both theAI methodology and the settings for the problem to further
improve the performance of our analysis.
Because this problem is highly nonlinear, the resulting
distribution becomes very complex. Thus, it is important toconsider uncertainty for this problem. On the other hand, it
is evident that conducting 1,000 cases of heroic computing
using the full K computer with the maximum available timefor each case is highly impractical. Many of other earthquake problems are highly nonlinear and results in complex
responses. Although we cannot completely eliminate the needfor multiple simulations, we have shown that an AI that canconfirm attainment of a permissible level of accuracy canenhance HPC via an uncertainty analysis. This demonstrates
the potential of HPC and is a big step toward achieving of
HQC for earthquakes.
Acknowledgments: Our results were obtained using the K computer atthe RIKEN Advanced Institute for Computational Science (proposal numbers:
hp160221, hp160160, hp160157, and hp170249). We acknowledge the support
provided by the Japan Society for the Promotion of Science (15K18110,26249066, 25220908, and 17K14719). We thank the Operations and Computer
Technologies Division of RIKEN AICS and the High Performance Computing
Infrastructure helpdesk for their support in the use of the K computer.

R EFERENCES[1] Second report of the Nankai Trough Large Earthquake Model Committee, Cabinet Office, Government of Japan (August 28,2012). [Online].
http://www.bousai.go.jp/jishin/nankai/model/index.html[2] Disaster
assessmentofTokyoduetolarge
earthquakes such as the Nankai Trough Earthquake, Tokyo
Metropolitan
Government(May14,2013).
[Online].
http://www.bousai.metro.tokyo.jp/taisaku/1000902/1000402.html[3] T. Ichimura, K. Fujita, M. Hori, T. Sakanoue, and R. Hamanaka. “Threedimensional Nonlinear Seismic Ground Response Analysis of Local Site
Effects for Estimating Seismic Behavior of Buried Pipelines,” Journalof Pressure Vessel Technology, American Society of Mechanical Engineers, 136, Paper No: PVT-13-1131, 2014, DOI: 10.1115/1.4026208.
[4] National Digital Soil Map, The Japanese Geotechnical Society,[Online].http://www.denshi-jiban.jp/[5] T. Ichimura, K. Fujita, P.E.B. Quinay, L. Maddegedara, M. Hori, S.
Tanaka, Y. Shizawa, H. Kobayashi, and K. Minami. “Implicit nonlinear
wave simulation with 1.08T DOF and 0.270T unstructured finite elements to enhance comprehensive earthquake simulation,” Proceedingsof the International Conference on High Performance Computing,
Networking, Storage and Analysis, (SC’15), 2015.
[6] H. Miyazaki, Y. Kusano, N. Shinjou, F. Shoji, M. Yokokawa, and T.
Watanabe. “Overview of the K computer system,” FUJITSU Sci. Tech.
J., 48, 3, 302-309, 2012.
[7] Chainer, Preferred Networks, inc., https://chainer.org/