Chapter 6

Discrete-Event Simulation
Research Team
6.1

Members
Nobuyasu Ito (Team Leader)Hajime Inaoka(Research Scientist)
Yohsuke Murase(Research Scientist)
Naoki Yoshioka (Research Scientist)
Takeshi Uchitane (Postdoctoral Researcher)
Shih-Chieh Wang (Postdosctoral Researcher)
Tomio Kamada (Guest Researcher)

6.2

Research Activities

Discrete-event simulations cover much wider fields than discretized simulations of continuous models.
They comprise various kinds of models, for example, particles, agents, automata, games and so on,
and their applications are from material and biomedical sciences to ecological and environmental
problems. Social designs and controls have becoming the more interesting target since so-called the”big data sciences” became popular.
One characteristic feature of discrete-event simulations is their variety both in model parameters
and behaviors. Different parameters of discrete models often result in qualitatively quite different
behaviors. For example, two particles just pass through when they do not collide with each other,
but they will be scattered to different orbits when they collide. A automaton reacts specificallywhen their inputs satisfy its activation condition. A system with such discrete elements will behaves
unpredictable way. This feature is much different from the case of ”continuous” simulations which areoften characterized by continuous change in behaviors when input parameters are slightly modified.
Another feature of discrete-event simulations is network structure. Relations between elements
are often characterized by graphs and the graphs have usually nonuniform. For example, in a systemof hard particles, colliding particles are connected and noncolliding ones are not. The connectionchanged at every collision. Another example is human relations. Some are friendly connected, andsome are competing. Such human relations are known to be characterized by a small-world structure.
Activities of the Discrete-event simulation research team(DESRT) are challenging these two features with the K and the post-K scale massive parallel supercomputers. The target problems arepotentially in very vast fields of sciences, technologies and humanities, and the DESRT is now focusing mainly on social systems.

51

Chapter 6. Discrete-Event Simulation Research Team

6.2.1

Parameter-space explorer

The DESRT has been developing job management tools named OACIS and CARAVAN to challenge
the huge-parameter space, the first feature. The name OACIS is an abbreviation of ”Organizing
Assistant for Comprehensive and Interactive Simulations”[5].
The OACIS have been released for public use as an AICS software[25], and the CARAVAN is
now being tested with its prerelease version. Both of these two tools are used with user’s simulation
and analysis softwares. A user register one’s simulation and analysis softwares together with their
input parameters and available host computers to these tools, then the tools control submitting andanalyzing jobs.
Each simulation and each analysis for many parameters will be properly processed by computers
if they are specified correctly, but error will easily be introduced by human side. In this sense,
supercomputers are demanding not only greater programming skill but also more reliable operationand smarter decision of simulation parameters. The OACIS and CARAVAN are designed to solve
this demand. A difference of the two tools is number of jobs and/or parameter sets. The OACIS isdesigned for jobs up to 106 ∼ 107 different parameters, and the CARAVAN to 109 and more.
The OACIS, a job management tool for simulations and analyses are designed and developed in
DESRT. It is coded using Ruby, Ruby-on-rail framework and MongoDB. After installation, users
register their applications for simulations and analyses, and their computers from PC to supercomputers like K to the OACIS. Then they can design and order executions of simulations and analyses
on its web-browser front end. The ssh connection is used to operate the registered remote computers
and Job states are supervised by the OACIS. Current prototype transfers output files of simulations
and analysis to the local computer operating the OACIS from remote computers. The results andhistorical data are preserved in local computer using MongoDB. The first version was released lastyear, and the second version in this year 2015. Now users are extending to universities and research
institutes. Lecture courses have been organized on demand not only in AICS but also in these sites.
In this year, the OACIS activities are mainly to support user groups and minor version-ups havebeen continued.
The CARAVAN is coded with a PGAS language X10 implemented to the K computer. It is now
under development and it will be released in following years.

6.2.2

Graph simulation

A challenge to network structures, the second feature of the discrete-event simulation, have beencultivated in contexts of phenomenological layers of social systems, especially, traffic, economics
and social relations. These three are major basic components of modern society, and agent-basedmodeling of them has been developed in these decades.
Car traffic simulations with agent-based model of each car on a single linear road will be simple. But real roads form irregular network, and therefore treatment of networks becomes a major
ingredient of simulations for real roads.
Agent-based models of car traffic simulations have been well developed since 1980s, and somesimulation softwares are available now. But parallelized simulation software is not available yet. So
the DESRT has been developing a parallelized car traffic simulator. Each computer process simulatesa part of a road map, and simulations are geometrically parallelized.
Although the simulator is still random-walking cars without routing, the simulator had achieved
simulations of all over Japan scale(see Table 6.1) and ones of all over the world was achieved in thisyear. Using quarter nodes of the K computer, realtime global simulation was achieved(see Table 6.2).
Preliminary survey of parallelized routing algorithm has been conducted this year, and full routing
will be equipped in the simulator in the following years.
To develop analysis of traffic simulation, the DESRT had developed a car traffic simulator of
Kobe city[2] using unparallelized simulator, SUMO. An ensemble of OD sets is assumed and Monte
Carlo sampling of OD sets are executed. For each OD set, car traffic was simulated with the Kobe
traffic simulator. Then the factor analysis of multivariate statistics had been done for traffic of eachroad segment[10]. Estimated factors(Fig. 6.1) are reasonably explain what we are experiencing inour daily lives in Kobe. Another issue of urban network traffic is whether it is stable and how it
behaves macroscopically. A network flow model assuming a basic fundamental diagram between car
density and flux was analyzed, and it was concluded that a network traffic is not stable in general,and therefore so-called the macroscopic fundamental diagram is fragile[11, 12, 24, 19, 23]. So thereal urban traffic is strongly influenced by the traffic rules and controls.
52

Chapter 6. Discrete-Event Simulation Research Team

Table 6.1: performance of our parallelized traffic simulator on the K computer is listed for the road
network of Japan using the Open street map(https://openstreetmap.jp/), which contains totally
1,284,452 Km of road with 8,143,352 road segments and 5,887,609 crossings. Totally 11,775,218 cars
are simulated and each car selects next way randomly at every corner. This simulator firstly executes
initialization mainly the map preparation and file I/O took 51 to 55 seconds. Then car movements
are simulated. Performance data of traffic for one hour are listed for several number of nodes up to
the quarter nodes of the K computer. Simulation time step was 0.01 second and therefore totally
360,000 steps were executed after the initialization.
number of
node
81
324
1296
5184
20736

simulationtime(sec)
75,02019,059
4,865
712
386

MPI time
(sec)
4,285
2,082
1,067
328
281

map generationtime(sec)
103
74.0
68.3
63.1
65.2

elapsedtime(sec)
75,1
19,1
4,96
789
499

Table 6.2:
performance of our parallelized traffic simulator using the quarter nodes of
the K computer is listed for the road network of all over the world using the Open street
map(https://openstreetmap.jp/), which contains totally 30,887,952 Km of road with 104,743,486
road segments and 79,441,144 crossings. Totally 100,000,000 cars are simulated and each car selects
next way randomly at every corner. This simulator firstly executes initialization mainly the map
preparation and file I/O took 561 seconds. Then car movements are simulated. Performance data of
traffic for 100 seconds are listed. Simulation time step was 0.01 second and therefore totally 10,000
steps were executed after the initialization.
number of
node
20736

6.2.3

simulationtime(sec)
116

MPI time
(sec)
41.4

map generationtime(sec)
74.8

elapsedtime(sec)
857

Other activities

Studies on ecosystems[6, 8], pedestrian and other self-propulsive and dissipative particles[3, 4, 16],
models of social-relation networks[1, 14, 15, 17, 18], molecular dynamics simulation of nonequilibrium
phenomena[13, 13, 20], material breakdown[21, 22] were also be achieved.
During this year, a post-doc(Dr. Shih-Chieh Wang) and a researcher(Dr. Naoki Yoshioka) havearrived for new projects of modelings and simulations of disease propagation and simulations ofquantum computers. Some preliminary research of these subjects are initiated.

6.3

Schedule and Future Plan

In the following, we describe our schedule and plan.
Users of the OACIS have been growing, and the more requests are coming. Formation of the
OACIS user group will be the next issue, together with international extension. In addition to theuser support, an API to control OACIS from a software will be developed to make intelligent andautomatic model analysis possible.
The first version of CARAVAN will be released in the following years. For this purpose, fixing
some problems in the X10 library is a key process.
Car routine with geometrically parallelized map over nodes will continue to be developed, andparallelized traffic simulator will be released. Simulations of disease propagation and quantum
computers will be conducted.
Together with these activities, future roadmap and perspective of social studies with discreteevent simulations are summarized in Fig.6.2.

53

Chapter 6. Discrete-Event Simulation Research Team

Figure 6.1: Road segments appeared in the 33 factors from 160 Monte Carlo events were shown withblue lines in a road map of the Kobe city. It is observed that the national roads, high way, and othermajor roads are extracted from the simulations.

6.4

Publications

Journal Articles[1]

Yohsuke Murase et al. “Modeling the role of relationship fading and breakup in social network
formation”. In: PLoS ONE vol.10, No.7, e0133005. doi: 10.1371/journal.pone.0133005 (2015).

Conference Papers[2]

Yuta Asano et al. “Traffic Simulation of Kobe-City”. In: Proceedings of the International Conference on Social Modeling and Simulation, plus Econophysics Colloquium edited by H.Takayasu,
N.Ito, I.Noda and M.Takayasu, Kobe held in Nov. 2014, pp.255-264, (Springer Proceedings in
Complexity, ISBN 978-3-319-20590-8, 2015). 2015.

[3]

Takayuki Hiraoka, Takashi Shimada, and Nobuyasu Ito. “Collective Dynamics of Pedestrians
with No Fixed Destination”. In: Proceedings of the International Conference on Social Modeling and Simulation, plus Econophysics Colloquium edited by H.Takayasu, N.Ito, I.Noda and
M.Takayasu, Kobe held in Nov. 2014, pp.243-253, (Springer Proceedings in Complexity, ISBN
978-3-319-20590-8, 2015). 2015.

[4]

Akiyuki Kuwabara et al. “Drag force on disks moving at constant speed in a two-dimensional
granular bed(in Japanese)”. In: Proceedings of the 21st Symposium of the Mathematical Societyof Traffic Flow(Dec. 10-11, 2015, Nagoya, Japan) p.103. 2015.

[5]

Yohsuke Murase, Takeshi Uchitane, and Nobuyasu Ito. “A tool for parameter-space explorations”. In: “Workshop on Recent Developments in Computer Simulation Studies in Condensed Matter Physics”, The University of Georgia, Athens, U. S. A., February 24-28, 2014,
Physics Procedia vol.57 pp.73-76. 2015.

[6]

Yohsuke Murase et al. “A Universal Lifetime Distribution for Multi-Species Systems”. In:
Proceedings of the International Conference on Social Modeling and Simulation, plus Econophysics Colloquium edited by H.Takayasu, N.Ito, I.Noda and M.Takayasu, Kobe held in Nov.
2014, pp.175-186, (Springer Proceedings in Complexity, ISBN 978-3-319-20590-8, 2015). 2015.
54

Chapter 6. Discrete-Event Simulation Research Team

Roadmap of traffic simulation
100000Computer resource per job[ TFLOPS day / job ][1EFLOPS day]

[1ZFLOPS day]

10000

1000

Whole Japan, full OD
Whole Japan, big event
Whole Japan, traffic control

100

whole Japan, partial OD

Whole Japan, analysis
Metropolitan, big event

10

Tokyo, big event[1TFLOPS day]

Metropolitan, traffic control

1

Tokyo, traffic control
million city, traffic control

0.1[1PFLOPS day]million city, road plan
million city, signal control

0.01
1

(1)

10

100

1K
10K
Number of jobs

100K

1M

10M

1M

10M

1M

10M

Roadmap of market simulation
100000Computer resource per job [ TFLOPS day / job ][1EFLOPS day]

[1ZFLOPS day]

10000

Systemic risk ofinter-bank network[1PFLOPS day]1000

whole Tokyo stock market
100

volatility distortion
Basel Capital Accords(100 name)10inter-name networks[1TFLOPS day]Basel Capital Accords(3 name)1
arbitrage
0.1

daily limit
tic size

0.01
1

(2)

10

100

1K
10K
Number of jobs

100K

Roadmap of evacuation simulation
100000Computer resource per job [ TFLOPS day / job ][1EFLOPS day]

[1ZFLOPS day]

10000
megalopolis
commute support[1PFLOPS day]1000

megacity
complex multiple disaster100

megacity
complex disaster

10

densely populated zone
complex disaster1

city zone
Tsunamicity zone
Tsunami and heavy snow

one building
evacuation

0.1[1TFLOPS day]
0.01

(3)

1

10

100

1K
10K
Number of jobs

100K

Figure 6.2: Roadmaps of agent-based simulations of (1) car traffic, (2) market and (3) evacuation areplotted. Horizontal axis shows estimated number of simulation jobs to achieve each task. Verticalaxis shows computer resources of each job in a unit of TFLOPS·day, Lines of -45 degree correspondto lines of equal computational effort[7].

55Chapter 6. Discrete-Event Simulation Research Team
[7]

Itsuki Noda et al. “Roadmap for Multiagent Social Simulation on HPC”. In: Proceedings of2015 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent
Technology (WI-IAT), Singapore, December 6-9, 2015, vol.3, pp.22-25, (DOI:10.1109/WIIAT.2015.178). 2015.

[8]

Takashi Shimada, Yohsuke Murase, and Nobuyasu Ito. “Do connections make systems robust?:
a new scenario for the complexity-stability”. In: Proceedings of the International Conference on
Social Modeling and Simulation, plus Econophysics Colloquium edited by H.Takayasu, N.Ito,
I.Noda and M.Takayasu, Kobe held in Nov. 2014, pp.99-109, (Springer Proceedings in Complexity, ISBN 978-3-319-20590-8, 2015). 2015.

[9]

Takashi Shimada, Naoki Yoshioka, and Nobuyasu Ito. “Linear stability of steady flow statesin networked roads withpiece-wise linear fundamental diagrams(in Japanese)”. In: Proceedingsof the 21st Symposium of the Mathematical Society of Traffic Flow(Dec. 10-11, 2015, Nagoya,Japan) p.43. 2015.

[10]

Takeshi Uchitane and Nobuyasu Ito. “Applying Factor Analysis to Results of Vehicle Traffic
Simulation and Discussion for Results of the Analysis(in Japanese)”. In: Proceedings of the 21st
Symposium of the Mathematical Society of Traffic Flow(Dec. 10-11, 2015, Nagoya, Japan) p.
2015.

[11]

Naoki Yoshioka, Takashi Shimada, and Nobuyasu Ito. “Stability and macroscopic fundamental
diagram in simple model of urban traffic(in Japanese)”. In: Proceedings of the 21st Symposiumof the Mathematical Society of Traffic Flow(Dec. 10-11, 2015, Nagoya, Japan) p.71. 2015.

Invited Talks[12]

Nobuyasu Ito. Network traffic simulation. VIIIth Brazilian Meeting on Simulational Physics
Florianopolis, Brazil, August 4-7th, 2015. 2015.

[13]

Nobuyasu Ito et al. Molecular dynamics simulation of liquid-gas transition. 28th Workshop“Recent Developments in Computer Simulation Studies in Condensed Matter Physics”, The
University of Georgia, Athens, U. S. A., February 23-27, 2015. 2015.

[14]

Yohsuke Murase et al. Modeling the role of relationship fading and breakup in social network
formation. 2015 Physics of Social Complexity workshop POSCO International Center, Pohang
University of Science and Technology, Pohang, Korea, Nov. 2-4, 2015. 2015.

[15]

Janos Torok Yohsuke Murase et al. Modeling the role of relationship fading and breakup insocial network formation. 2015 Physics of Social Complexity workshop POSCO International
Center, Pohang University of Science and Technology, (Nov. 2-4, 2015, Pohang, Korea). 2015.

Posters and Presentations[16]

Akiyuki Kuwabara et al. Drag force on disks moving at constant speed in a two-dimensional
granular bed. The 21st Symposium of the Mathematical Society of Traffic Flow(Dec. 10-11,
2015, Nagoya, Japan).

[17]

Yohsuke Murase et al. A universal lifetime distribution for multi-species systems. The CASSIA
3rd workshop on International workshop on complex phenomena from molecule to society (Nov.
24-25, 2015, Tokyo, Japan, )[18]

Yohsuke Murase et al. Multilayer weighted social network model. 2015 NCTS International
Workshop on Critical Phenomena and Complex Systems(Aug.15-17, 2015, Shinchu, Taiwan).

[19]

Takashi Shimada, Naoki Yoshioka, and Nobuyasu Ito. Linear stability of steady flow statesin networked roads withpiece-wise linear fundamental diagrams. The 21st Symposium of theMathematical Society of Traffic Flow(Dec. 10-11, 2015, Nagoya, Japan).

[20]

Hiroshi Watanabe et al. Scalable and Highly SIMD-vectorized Molecular Dynamics Simulation
Involving Multiple Bubble Nuclei. SC15, Austin, USA (Nov. 2015).

[21]

Naoki Yoshioka, Ferenc Kun, and Nobuyasu Ito. Epicenter jump distributions in thermally induced breakdown using kinetic algorithm. 6th Hungary-Japan Bilateral Workshop on Statistical
Physics of Breakdown Phenomena (Jan. 6-7, 2016, Debrecen, Hungary).
56

Chapter 6. Discrete-Event Simulation Research Team
[22]

Naoki Yoshioka, Ferenc Kun, and Nobuyasu Ito. Macroscopic fundamental diagram and stability in simple models of urban traffic. 21st International Symposium on Artificial Life andRobotics (Jan. 20-22, 2016, Beppu, Oita, Japan,)[23]

Naoki Yoshioka, Takashi Shimada, and Nobuyasu Ito. Stability and macroscopic fundamental
diagram in simple model of urban traffic. The 21st Symposium of the Mathematical Society ofTraffic Flow(Dec. 10-11, 2015, Nagoya, Japan).

[24]

Naoki Yoshioka, Takashi Shimada, and Nobuyasu Ito. Stability and macroscopic fundamental
diagram in simple model of urban traffic. The CASSIA 3rd workshop on International workshop
on complex phenomena from molecule to society (Nov. 24-25, 2015, Tokyo, Japan, )Patents and Deliverables[25] OACIS v2.0.0 - v.2.2.1(five release).

57