# How Effective are Tabletop Role-Playing (Serious) Games in Understanding and Validating the Predictive Capabilities of Disaster Response Agent-based Models?

Nimisingha Jacob Amakama, Gilles Dusserre, Axelle Cadiere, R. Wade Schuette, Grégory Zacharewicz

<figure><img src="image_1.png" /></figure>

## To cite this version:

Nimisingha Jacob Amakama, Gilles Dusserre, Axelle Cadiere, R. Wade Schuette, Grégory Zacharewicz. How Effective are Tabletop Role-Playing (Serious) Games in Understanding and Validating the Predictive Capabilities of Disaster Response Agent-based Models?. American Journal of Multidisciplinary Research and Innovation, 2024, 3 (4), pp.49-61. 10.54536/ajmri.v3i4.2937. hal-04658044

HAL Id: hal-04658044

https://imt-mines-ales.hal.science/hal-04658044v1

Submitted on 22 Jul 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# How Effective are Tabletop Role-Playing (Serious) Games in Understanding and Validating the Predictive Capabilities of Disaster Response Agent-based Models?

Nimisingha Jacob Amakama<fnref n="1" />*, Gilles Dusserre<fnref n="1" />, Axelle Cadiere<fnref n="2" />, R. Wade Schuette<fnref n="3" />, Gregory Zacharewicz<fnref n="1" />

## Article Information

Received: May 27, 2024

Accepted: June 29, 2024

Published: July 03, 2024

## Keywords

*Agent-Based Models, Disaster Response, Tabletop Exercise, Role-Playing Games, Validation*

## ABSTRACT

Tabletop Role-Playing Games (TRPGs) for disaster response management are collaborative exercises in which participants take on the roles of characters, guided by a “Game Master.” TRPG are useful tools for training, improving the preparedness of emergency responders, and understanding/validating Agent-Based Models (ABMs). This study evaluates the knowledge of participants and assessed the effectiveness of utilizing TRPG in validating a NetLogo model. Medical doctors and master’s students were divided into two groups, who participated in the exercise based on defined disaster response strategies. The central components of the ABM were transformed into a Disaster Response Tabletop Role-Playing Game (DRTRPG). Medical doctors and master’s degree students were assigned to two groups in the exercise that is based on defined disaster response strategies. Multichoice pre- and post-assessment tests were used to evaluate each group’s knowledge, while a performance scoring matrix was used to generate data from in-game activity. Data was analyzed using Pandas and MS Excel data analytical tools. Pre and post-assessment test results (n = 8) showed an 11.5% and 3.1% increase in knowledge for medical doctors and students on disaster and healthcare decision-making, respectively. Visual comparison between the ABM and DRTRPG results reveals partial model validity. However, statistical comparison shows a clear variation in the outcomes. The “Alternate” policy strategy resulted in better disaster response outcomes. The design and implementation of the DRTRPG resulted in a collaborative understanding and communication of the ABMS model with promising results. However, there are significant statistical differences between both methodologies, highlighting the challenge of validating the predictive capabilities of ABMS using RPGs.

## INTRODUCTION

Tabletop role-playing games (TRPGs) are collaborative exercises in which participants take on the roles of characters in a fictional game world and are guided by a human supervisor called a “Game Master.” The Game Master (GM) acts as both storyteller and guide, working with players to help shape the game world. Creativity is central to TRPGs and influences both the players and the game master (Katifori et al., 2022). These serious games aim to train or prepare users in a specific area, such as disaster management, thereby improving the preparedness of emergency and crisis responders (Guiamalodin et al., 2024; Kwadwo & Morkla, 2024; Tanguid & Tanguid, 2023; Simões-Marques et al., 2020). Tabletop role-playing games (TRPGs) provide instructors with the opportunity to monitor group dynamics and decision-making activities while serving as immersive and effective tools for promoting soft skills (Linehan et al., 2009; Suguitan & Natividad, 2022). They provide a valuable environment to improve understanding of abstract concepts such as drought and water-related challenges, while improving communication and conflict resolution skills (Alejandria et al., 2023; Pasinetti et al., 2015). Serious games have also been integrated into educational contexts to improve learning outcomes and promote social behaviour. An example of this is the recycling of knowledge through immersive virtual reality environments (Poděbradská et al., 2020).

Because they collect data from player interaction data, they can accurately predict knowledge changes based on player interactions in the game. Serious role-playing games are also effective tools for understanding and validating the predictive capabilities of agent-based models in scenarios with limited computational resources (Alonso-Fernández et al., 2021; 2019; C. Liu et al., 2021). Games like ANYCARE, a role-playing game (RPG), helped players understand the challenges and intricacies of dealing with weather-related emergencies, thereby increasing their confidence in making timely decisions (Terti et al., 2019; Raimi et al., 2022). Cheng et al. (2020) found that Role-Playing Games (RPGs) are valuable pedagogical tools for promoting students’ understanding and responsibility, as well as their ability to reach consensus in multiple role scenarios. In the context of disaster response, RPGs can serve as useful tools for disaster prevention and promote communication between public and academic stakeholders (Sakai et al., 2019). Role-play exercises such as RAMSETE provided participants with the opportunity to discuss and challenge concepts, policies, and practices in the areas of disaster risk reduction, management, and adaptation to climate change (Fleming et al., 2020; Deinkuro et al., 2021; Olalekan et al., 2018; 2022; Sanchez et al., 2021). The study by Hosseini et al. (2022) found that disaster and crisis games are a more efficient method for improving nursing students’ understanding and behavioural skills in crisis

<footnote n="1">¹ Laboratory for the Science of Risks (LSR), IMT Mines Ales, France</footnote>

<footnote n="2">² CHROME, University of Nimes, France</footnote>

<footnote n="3">³ Shewnexti LLC, USA</footnote>

<footnote>* Corresponding author’s e-mail: nimisingha.amakama@mines-ales.fr</footnote>
management compared to case-based approaches. The use of role-playing in the development and validation of agent-based models offers several advantages. These benefits include adapting game parameters and player demographics to real-world scenarios, incorporating experimental design into data collection processes, and integrating operational validation phases into modelling procedures (Utomo et al., 2022).

This study examines the effectiveness of integrating role-playing to engage both healthcare professionals and non-experts (students) in understanding and validating an agent-based model of disaster response strategies. This research is significant because it introduces a method to improve model validation and capture the complexity of agent-based models by leveraging the formal and informal knowledge and behavior of stakeholders, thereby bridging the gap between theoretical modeling and real-world situations. Hence, this study aims to evaluate the effectiveness of utilizing disaster response tabletop role-playing games (DRTRPGs) in facilitating understanding of disaster management concepts and in validating the predictive capabilities of an agent-based model for a hypothetical disaster setting. To achieve the study's goal, the following objectives were proposed (1) Evaluate the knowledge enhancement achieved through the training and implementation of the DRTRPG via multiple-choice pre- and post-assessment tests; and (2) To visually and/or statistically compare the outcomes of the DRTRPG against the ABMS model outcomes (validation), by examining the impact of ambulance deployment policies on mortality outcomes. The authors assert that lessons learned from this research can be integrated into disaster preparedness and response strategies, resulting in improved training, preparedness, and coordinated disaster responses. The subsequent aspects of this study consist of a literature review, the methods and materials adopted for the study. The final two sections present the results and discussion and draw conclusions, including limitations and recommendations for future studies.

## LITERATURE REVIEW

In a tabletop role-playing game (TRPG), players create characters and go on adventures under the guidance of a game master (Guzdial et al., 2020). Role-playing games (RPGs) are structured to instruct, guide, and equip players to overcome specific challenges by simulating real-world scenarios ("The Functions of Role-playing Games: How Participants Create Community, Solve Problems and Explore Identity," 2011; Rye & Aktas, 2022). Disaster response tabletop role-playing games have a variety of uses in improving preparedness and response skills. With the help of these emergency simulation games, players can test their decision-making skills, practice response plans, and better understand their role in disaster situations (McCourt & Watson, 2023). Suleman et al. (2022) assert that TRPGs significantly improve understanding of disaster management among healthcare workers, with an emphasis on strategies, communication,

and collaboration. Additionally, tabletop exercises have been shown to improve nursing students' knowledge and disposition during disaster exercises, making them useful tools for improving disaster response skills (Ademiluyi et al., 2010; Husna et al., 2020; Raimi et al., 2020; Raimi et al., 2019).

Agent-based simulation on the other hand is a computational modeling approach that positions autonomous agents within an environment to investigate emergent outcomes resulting from interactions between agents and their surroundings (Y. Liu et al., 2023). McDonald and Osgood (2023), states that these simulations are useful for deciphering macro-level phenomena by studying individual-level interactions in complex systems. They offer advantages such as scalability, flexibility in representing diversity and the ability to include static and dynamic network contexts in addition to spatial aspects (Antelmi et al., 2022). Ogie et al. (2022) highlighted the potential benefits of using ABM for collaborative translation in crisis communication. Role-playing games and agent-based models together provide a vantage point for exploring a variety of phenomena. According to (Guyot et al., 2007), a combination of ABM and RPGs can serve three main purposes:

(i) Assist stakeholders in decision-making in challenging scenarios;

(ii) Promoting group learning and negotiation between participants; and

(iii) Validating and refining a model by observing stakeholder behavior.

When it comes to modeling a specific case, RPG players are usually selected from the key stakeholders at the case study site (D'Aquino & Bah, 2014). However, if the research goal includes hypothesis testing and group comparisons, participants may be selected from a broader population (Meijer et al., 2006).

As an additional method for validating agent-based models, role-playing games (RPGs) facilitate validation triangulation by observing agents' decision-making within the game environment (Villamor et al., 2023). Utomo et al. (2022) argued that role-playing games (RPGs) provide valuable insights for model validation by allowing observation of player behaviour that can be contrasted with agent behaviour in a model. This was supported by Pira et al. (2017), who asserted that RPGs could improve the robustness and relevance of agent-based models by enabling stakeholder-driven validation, especially in complicated situations such as urban freight transport decision-making.

However, several studies suggest that validating ABMs using the classical empirical methods is very challenging, as there is typically a paucity of data and ABMS models are based on future predictions (Marks, 2007; Takadama, Kawai and Koyama, 2008; Anand et al., 2016; Onggo & Karatas, 2016; Gore et al., 2016; Utomo et al., 2022). These findings reveals that the classical empirical validation of agent-based models is challenging because of features such as path dependence, emergence, and
complex interactions, in addition to the lack of micro-
level data. Other aspects cited by the researchers border
on the complexity of social processes and the stochasticity
of the simulation. This the authors argue may lead to
disagreement in the result as compared to an experiment
that is well controlled.

However, Robinson et al. (2007) argue that models should
be subjected to validation using various techniques and
approaches and emphasize that the validity of ABMs
depends on the accessibility and quality of data for
model calibration and validation. They also advocate the
use of RPG-based validation to test and validate ABMs;
but they caution that the results should be verified with
historical data and validated using alternative methods.
According to Dubois et al. (2013), the settings of games
can influence participants' attitudes and behaviour patterns
and subsequently affect the outcomes of role-playing
games. In addition, the interchangeability of the terms
"player" and "agent" is debated as part of the discussion
about combining role-playing games and agent-based
models ("Introduction to Agent-Based Modeling," 2019).
Squazzoni et al. (2020), suggested that the realism and
validation in role-playing games (RPGs) depends on the
assumptions and rules built into the game design. They
suggested that if a role-playing game is well designed on
the basic rules and assumptions of empirical data, it can
realistically simulate human behavior and decision-making,
thereby improving the validity of the validation process.
These studies support the need for this current study by
highlighting the context of disaster response strategies
for disaster preparedness, training, and ABM validation.
To achieve our research objectives, this study evaluates
the effectiveness of using a disaster response tabletop
role-playing game in improving player knowledge and
validating the results of an ABM by assessing the impact
of ambulance policy on mortality outcomes - an aspect not
specifically addressed in previous studies.

MATERIALS AND METHODS

Research Methodology: DRTRPG Design and
Implementation

This research falls into the area of experimental research,
particularly at the interface of simulation and educational
research in the context of disaster protection strategies
to assessing disaster response strategies. The study aims
to improve the understanding of process and knowledge
representation in agent-based models (ABMs) and role-
playing games (RPGs) through the practical and interactive
involvement of participants, including both experts
and non-experts. In addition, the research addresses
a specific gap in the literature, which is to assess and
validate the impact of ambulance policy on mortality
rates to improve the understanding of the ABMS model
using a disaster response tabletop role- playing game.
As a validation strategy for this study, a scenario-based
disaster response tabletop role-playing game (DRTRPG)
was developed, as suggested by Ligtenberg et al. (2010) for
ABM model validation method using RPG. The process

of transforming ABMs into RPG for the purpose of
validation is summarized below:

* Develop a basic ABMS model,

* Develop a role-playing game based on the basic ABMS model in which players can engage in tasks that replicate the agent's actions in the ABMS model,

• Guide players in completing game tasks,

• Use the RPG to collect data,

* Analyze and contrast the results from the ABMS with the results from the RPG.

The DRTRPG exercise is divided into two phases. First,
participants complete preparatory training and take part
in exercises to familiarize themselves with the DRTRPG
concept in a group setting. The goal of this phase is to
refine both the game and the ABMS model components.
In phase 2, response results are generated, analysed and
compared between doctors, master's students and the
ABMS basic model, also in a group setting. The DRTRPG,
developed between 2023 and 2024, includes two exercises
with three training sessions each. Tabletop gaming sessions
took place in a standard classroom at the IMT Mines Ales
Clavières location that comfortably accommodate 10 to 15
participants. With this setup, each group experienced the
same crisis response scenario throughout the game. Prior
to participating in the DRTRPG exercise, participants'
understanding of disaster management and health-related
decision-making were assessed using a multiple-choice pre-
game questionnaire. A simulation is then run that represents
components of the ABMS model within the DRTRPG,
creating a scenario in which players collaborate and made
decisions. During the game, each group recorded the time
spent on tasks while observers rate the players' actions.
This approach help in the generation of substantial dataset
for analysis. Following the in-game activities, a debriefing
was conducted, along with a post-game questionnaire and
an observer rating assessment completed by players and
observers, respectively.

Participants

This study involves participants with diverse backgrounds
and experiences playing the DRTRPG game in teams that
were assigned specific roles (e.g., surgeons, triage nurses,
emergency responders) to manage a hypothetical explosion
event and provide optimal medical care. As part of the first
practice session, the game was first tested in four groups
with a total of 15 players (including observers). In phase
2, two groups with 11 participants are used to compare
the actions of students and doctors. The researchers
hypothesizes that the implementation of the DRTRPG has
the potential of improving the validity and effectiveness
of disaster response strategies by integrating agent-based
models and tabletop role-playing techniques.

Concepts, Principles, and Experimental Setup

Tabletop role-playing games designed for disaster response
are helpful resources for confirming the accuracy of agent-
based models used in disaster management. PREDIS
and other similar games facilitated the decision-making
processes of experts and non-experts alike, allowing for quicker and more precise responses (Rye & Aktas, 2022). According to Utomo et al. (2022), RPG data is used to validate ABMS models at both macro and micro levels

to ensure that the model accurately represents real-world knowledge and processes. The graphical user interface of the developed ABMS simulation model underlying the design concept of the DRTRPG is shown in Figure 1.

<figure><img src="image_3.png" /><figcaption>Figure 1: Graphical user interface of simulation utilized in the DRTRPG design (Amakama et al., 2023a;2023b)</figcaption></figure>

The graphical user interface (GUI) of the ABMS model represents a simulated (720 runs) crisis scenario that is integral to the design of the Disaster Response Role Playing Game (DRTRPG).

Within the DRTRPG, participants assume various roles as agents responding to an explosion in the oil and gas sector, while regional and mobile field hospitals collaborate to determine optimal response strategies and

provide care to the injured. These roles simplify elements of the ABMS simulation. The development of the game required careful consideration of game elements, roles, and simulations. Table 1 outlines the guiding principles of the DRTRPG design and serves as a three-step guide for tabletop role-playing serious game exercises with a focus on disaster response and healthcare simulation training, education, and validation.

<table><caption>Table 1: The DRTRPG exercise components, design decisions, and considerations</caption><thead><tr><td>DRTRPG elements</td><td>DRTRPG description</td><td>DRTRPG design choices</td><td>Playing considerations/ concerns</td></tr></thead><tbody><tr><td>DRTRPG storyline</td><td>A scenario to validate response interactions and outcomes in a regional and mobile field hospital simulated around an oil and gas industry disaster and modelled using an ABMS framework. Assessment tests designed to evaluate knowledge acquisition resulting from the DRTRPG exercise.</td><td>- To ensure accurate representation, the game closely follows the guidelines and parameters of the ABMS model.<br/>- Dynamically adjust the game to address player concerns as they arise in real time<br/>- The plot of the scenario is designed to explore various “what if” situations.</td><td>- The degree of accuracy with which the game replicates the dynamics of the ABMS-modelled scenario<br/>- The extent to which the game scenario and model accurately represent a potential real-world situation</td></tr></tbody></table>
<table><tr><td>DRTRPG roles</td><td>Participants assume the roles of various agents represented in the ABMS model, including positions such as emergency dispatcher, triage nurse, surgeon, and others</td><td>- Detail the tasks and decision protocols for each role within the groups<br/>- There are specific, measurable metrics available to assess performance<br/>- In the DRTRPG design, players should feel the impact of their role interactions.</td><td>- It ensures that a role is complex enough to test the model, but remains playable for participants<br/>- The impact of the roles on the model validation<br/>- The need for players to understand ABMS procedures through pre-game training</td></tr><tr><td>DRTRPG simulations</td><td>An ABMS-derived simulation sequence reflecting the oil and gas explosion incident and its aftermath, with an emphasis on collaborative response and decision making, accomplished via in-game tasks</td><td>- The simulations include scenarios and tasks that require the use of disaster management skills<br/>- Establish iterative game loops to facilitate model validation through multiple trials<br/>- Provide statistical and/or visual comparisons illustrating the impact of decisions within the DRTRPG compared to outcomes predicted by the ABMS.</td><td>- Ensure game simulation provides valuable and robust data for model validation<br/>- Achieving a balance between the complexity of the ABMS and the simplicity of the game<br/>- Effectively communicate the goals and framework of the basic ABMS model to participants and relevant stakeholders</td></tr></table>

## Experiments Set-Up

There are five essential progressive steps that make up the experiment's procedure, which is listed as follows.

(i) A summary of the objectives, rules, and plot of the DRTRPG to the players.;

(ii) Roles assignment;

(iii) Simulation activities (in-game action);

(iv) Debriefing of the players based on the game; and

(v) Gather players' and observers feedback presented in Table 2.

<table><caption>**Table 2:** DRTRPG experiment implementation steps (estimated time: 2hrs)</caption><thead><tr><td>Actions to be taken</td><td>Resource(s)</td><td>Duration (est.)</td></tr></thead><tbody><tr><td>GM/MoD explains the game's rules, plot, and objectives to all participants.</td><td>Power Point presentation</td><td>25 mins</td></tr><tr><td>The GM/MoD assigns each player their respective roles (usually pre-selected based on the background of the participants in a group).</td><td>Printed slips or via email</td><td>5 mins</td></tr><tr><td>The members (of every group) discuss the specifics of their assigned roles.</td><td>Group deliberation</td><td>5 mins</td></tr><tr><td>The simulations begin with the GM/MoD presenting the game's possible outcomes as hypothetical truths.</td><td>Power Point presentation</td><td>10 mins</td></tr><tr><td>Participants take part in simulated scenario tasks. By directing the story and generating relevant dialogue, the GM/MoD directs the game, ensuring the mechanics of the game follow the rules.</td><td>Cards, phones, PowerPoint presentation</td><td>55 mins</td></tr><tr><td>The GM/MoD initiates the debriefing process with the players and observers.</td><td>Power Point presentation of questions</td><td>10 mins</td></tr><tr><td>The participants respond to a brief multiple-choice post exercise questionnaire to assess their knowledge improvement and lesson learnt. Observers fill out the post-game performance assessment</td><td>Microsoft form designed questionnaire(s) (online)</td><td>10 mins</td></tr></tbody></table>

The GM assists players by introducing important plot points, providing hints, and initiating relevant conversations in accordance with the DRTRPG rules. Observers score the performance, both during the in-game activities and provide an overall assessment of

the game design and implementation. According to the researcher's findings, a playgroup with eight to sixteen participants would be more ideal, if the game is to produce better datasets for validating the ABMS model, further activities are required.
## The DRTRPG Guidelines

Based on traditional disaster-themed role-playing game guidelines, the DRTRPG runs according to the following rules:

* Prior to making decisions, only players on the same team are required to communicate and collaborate with their group.

* The MoD/GM gives instructions, direction, and clues for the game, and the observers observe the players and document their observations.

* Each group's discussion will be centered only on the emergency response scenario that will require prompt decision-making. There are time limits and consequences (mortality outcomes) for each in-game task, to promote deliberative decision-making and boost the game realism.

* Limited resources, such as ambulances or operating room, are incorporated into the game's design, which the players need to thoughtfully allocate to reduce mortality outcomes.

## DRTRPG Implementation Materials

Any tabletop role-playing game must choose materials

that improve the experience to succeed. The DRTRPG's resources are made up of varied materials, each of which serves a distinct function within the ecosystem of the game. The following are the exercise's basic materials.

* Maps of the disaster response scenario, depending on the in-game tasks

* PPT presentation slides

* Pencils/pens/stopwatches

* Pre/post assessment test/observer's performance evaluation forms

Figure 2 shows a representation of the in-action phases in the implemented game setup for the disaster response tabletop role-playing game. In this configuration, each group is assigned an observer, while the GM continually provides direction to the process by emphasizing the game's story and ensuring that the game mechanics are maintained. In this exercise, two teams (A and B) compete by completing a series of similar but challenging tasks consisting of 18 emergency challenges in a disaster scenario with varying patient numbers, response resources and severity index.

<figure><img src="image_4.png" /><figcaption>**Figure 2:** (a) - (d): DRTRPG players interaction and collaboration (a) the group stakeholder's tabletop setup, (b) (c) the second experiment for students and medical doctors (7 March 2024) and (d) the first experiment (6 March 2023).</figcaption></figure>

## Metrics, Data Generation and Analysis

The mortality outcome of the disaster scenario team decisions was assessed using the DRTRPG activity assessment chart (Table 3).

Although it promotes competition between groups

for efficient time management, the points awarded do not affect the outcome of the game. The estimated six minutes required to complete a task (the in-game activity), corresponds to one tick in the ABMS model.

Using this scoring table and the recorded reaction times for
<table><caption>**Table 3:** In-game activity mortality scoring metrics</caption><thead><tr><td>Decision time conditions</td><td>Estimated outcomes (mortality)</td><td>Outcome points</td></tr></thead><tbody><tr><td>Team decision taken at T ≤ 2 minute</td><td>All mild patients saved. Death's outcome ranges between d ≤ 5 from moderate and severe</td><td>10</td></tr><tr><td>Team decision taken at 2 < t ≤ 4 minutes</td><td>All mild patients saved. Death's outcome ranges between 5 < d ≤ 10 from moderate and severe patients</td><td>5</td></tr><tr><td>Team decision taken at 4 < t ≥ 6 minutes</td><td>All mild patients saved. Death's outcome d ≥ 10 from moderate and severe patients</td><td>0</td></tr></tbody></table>

each game task of each group, it was possible to estimate mortality outcomes based on the investigated ambulance policy strategies. An ambulance policy is a set of rules that control performance, patient care, ambulance dispatch, emergency response times, and other elements of an emergency response system. The two studied ambulance strategies are the “Always go to RH,” which is based on a traditional dispatching approach where ambulances are dispatched based on patient severity and availability, with only the Regional Hospital (RH) available. The second strategy (“Alternate”) involves a more dynamic approach by using a Mobile Field Hospital (MFH) to strategically support the RH in the scenario. The authors claim that shorter response times could be a sign of better decision making and team coordination. With NetLogo 6.3, large data sets was generated by modeling and simulating the complex interactions between the systems in the disaster setting. The BehaviorSpace interface, along with Python's Pandas library, gave the researchers access to a few useful libraries for working with and exploring the dataset. These tools were used for data set cleaning, information filtration and aggregation. Codes of the analysis for the ABMS and DRTRPG data is on our GitHub page at: https://github.com/Nimisingha/Ambulance_Policy_RPG/blob/main/codes_ambulance_policy_mortality_output.ipynb and https://github.com/Nimisingha/Ambulance_Policy_RPG/blob/main/codes_DRTRPG.ipynb respectively.

# RESULTS AND DISCUSSION

## Results

A total of 12 participants (divided into four groups), with 2 observers, were involved in the preliminary 2023 game test. (tabletop simulation experiment). The participants were distributed by background and experience as follows: three PhD students, one medical doctor, one emergency responder, one postdoctoral student, and nine master's degree students. 10 participants (in two groups), with 2 observers, participated in the final DRTRPG activity held in March 2024.

## Pre, Post, and Observer's Performance Evaluation

Results from the assessment tests show that the medical doctors (Group A) knowledge of disaster management and health interactions increased significantly (11.5%). However, only a slight increase in knowledge (3.1%) was observed among master's degree students. These results were generated based on correctly answered questions in the multiple-choice assessment test. According to observers' assessments, both teams also showed an elevated level of involvement and communication during the game. Still, both groups had some difficulty managing their time, suggesting they could do better. According to Figures 4a and 4b, both teams were judged to be competitive. While these results illustrate the potential and utility of TRPG design and implementation for disaster response

<figure><img src="image_5.png" /><figcaption>**Figure 3:** (a) Observers in-game performance evaluation, (b) Observers feedback performance evaluation</figcaption></figure>
class problems, further development is required to
provide a more accurate representation of the complexity
of complex systems.

ABMS Model Validation: Effects of Ambulance Policy on Mortality Outcomes

An ambulance policy is a collection of policies
that govern patient care, ambulance performance,

deployment, and other elements of emergency response
systems. The boxplot shown in Figure 5(a) represents
the distribution of data from the ABMS model, which
simulates the behavior and interactions of autonomous
agents using complex systems computational techniques.
Figure 5(b) shows the results of the teams participating
in the in-game activities based on the DRTRPG exercise,
which use a more participatory and interactive simulation

<figure><img src="image_6.png" /><figcaption>Figure 4: Effects of ambulance policy on morality outcome for (a) ABMS model, (b) DRTRPG exercise</figcaption></figure>

technique using a tabletop serious game to address
aspects of the ABMS decision protocols. This assessment
is based on two response strategies (“Always go RH” and
“Alternate”).

The “Always go to RH” ambulance policy is based on a traditional dispatching approach where ambulances are dispatched based on patient severity and availability and only the Regional Hospital (RH) is operational. The “Alternate” ambulance policy involves a more dynamic approach by deploying a Mobile Field Hospital (MFH) to strategically support the RH in the disaster response scenario. This study’s approach to comparing the results of both methods aims to provide insights into the effectiveness of using role-playing methods in the validation of agent-based models.

DISCUSSION

This study uses a different approach than the widely
used traditional methods to validate and communicate
the results of ABMS models. In this study, we compared
the simulation results of an ABMS model (visual and
statistical) with the results of a disaster response role-
playing game, rather than comparing the accuracy of
the model results with an equivalent model in the real
world. This participatory approach is intended to increase
stakeholder confidence in the model's ability to predict an
equivalent disaster situation.

As for the simplified class case of this study, which is
understandable to both the role-play participants, the
observers, and the modeler, the use of the group-based

collaborative TRPG approach to data generation and
validation of the ABM appears to provide promising
results. based on the DRTRPG findings the DRTRPG's
findings, the game can help emergency responders
improve their disaster management skills, especially
those with a medical background. Hence, the tests were
of greater benefit to people with prior knowledge of
medicine or disaster management, suggesting that the
subjects were suitable for them. Players recognized that
the game could be used as a teaching tool for disaster
management and found it helpful and educational. The
researcher emphasizes that when designing such games, it
is important to take participants' backgrounds into account
to optimize their learning outcomes. Based on observer
ratings (Figure 4) and insights into game activity, the
results support a more nuanced assessment of the team's
performance. Observers noted that both teams made
poor use of time, suggesting potential for development
in future game design. While situational awareness was
an area where Team A excelled, observers valued role
connection slightly more than overall performance, which
is why Team B received higher ratings. Overall, observers
valued different competencies within each team, with no
team emerging as leaders across all metrics.

Based on a visual comparison of the results of the two
strategies (Figure 5), the decision dynamics (the mortality
results for the policies) follow a similar trend for both
the ABMS model results and the DRTRPG results
(alternate policies led to more lives saved). However,
because the data distribution (shown in the different plot
scales) appears to be significantly different, basic statistics from the boxplot were used to examine the data in more detail. Regarding the ABMS results, the average mortality rates and Standard Deviation (SD) for the ABMS under the “Always go to RH” policy were 23.22 and 8.98, respectively, higher than under the “Alternate” policy. The smaller mean of 15.85 and standard deviation (0.79) show that the policy has consistently saved lives. There is also some variation in mortality outcomes, as evidenced by the mean and standard deviation of the “Alternate” policy, which are 19.75 and 8.27, respectively. The 25th, 75th and median percentiles, which are 20, 14 and 26 respectively, show a higher concentration of dataset points below the median.

Based on the DRTRPG data, the “Always go RH” policy has a slightly higher mean mortality (8.94) than the “Alternate” policy with an S.D. of 2.6, indicating greater variability. The lives saved, on the other hand, show good consistency, although with a mean of 29.72 and an S.D. of 2.86, above that of the “Alternate” policy (33.0, 2.11). The results of the “Alternate” policy show a S.D of 1.19, indicating lower variability and lower mean mortality (6.11) compared to the “Always go RH” policy. In terms of average number of lives saved, the “Alternate” policy also results in a higher number (33 lives saved) with an S.D. of 1.90, showing that this policy consistently saves more lives.

While the “Alternate” policy has a lower average mortality compared to the “Always go to RH” policy (suggesting that the “Alternate” policy may be a more effective strategy for reducing mortality/saving lives), the “Always go to RH” policy produces more consistent results in lives saved, as evidenced by the lower S.D. for lives saved. For both policies, DRTRPG results consistently show lower mortality and more lives saved than ABMS results, with less variability in results. This could mean that the ABMS overestimates variability or that the DRTRPG does not fully capture the range of variability encountered in the ABMS model. The maximum value for lives saved in the DRTRPG compared to the ABMS may indicate that the ABMS overestimates the lives saved.

The significant differences between the results of the ABMS model and the DRTRPG highlight the challenges in validating ABMS prediction capabilities through role-playing games. The limited number of runs

(18) in the DRTRPG compared to the ABMS model (720) may contribute to this variation and reduce the statistical robustness of the DRTRPG result. This smaller sample size may also not accurately reflect the scope of the scenario, introducing uncertainty in the validation process. Additionally, players' knowledge, experience, and biases in the DRTRPG may have influenced the results, resulting in more optimal decision-making than expected. To improve validation, the researcher suggests adapting the ABM to better fit the interactive dynamics of the DRTRPG. Since the goal is to validate the ABMS protocol using the DRTRPG, it may be necessary to consider the DRTRPG only as a “best case” scenario.

Further analysis could include comparing the models with real data (if available) to determine which strategy fits the actual results better and to examine the discrepancy between the two simulation results.

Disaster response tabletop role-playing games have been effectively utilized to validate agent-based models (De Rouck et al., 2023). These games function as simulations to assess healthcare workers' readiness for disasters and improve their understanding of disaster management (Rye & Aktas, 2022). Participatory simulation games like PREDIS have been used to validate decision support systems, demonstrating their viability in helping both experts and non-experts make quicker and better-informed decisions after a disaster (Suleman et al., 2022). Another example is the study by Ramchurn et al. (2016), who validated an ABM to enhance comprehension and assess the system's efficacy in simulating scenarios using a tabletop disaster response role-playing game. In a study conducted by Utomo et al. (2021), they examined the behavior of agents and validated their models through role-playing in an agent-based milk supply chain model. The study demonstrated the effectiveness of role-playing in validating agent-based models at both collective levels. Furthermore, Mariano and De Maria Albuquerque Alves (2020) used a simulation game focused on water management to study the actions of agents. An agent-based model was then built to simulate the results.

## CONCLUSION

The purpose of the design and implementation of the disaster response tabletop role-playing game presented in this study is to validate the predictive capability of a disaster response agent-based model and to test the impact of the game on improving participants' knowledge in the study's domain. By comparing the results between the pre- and post-test assessment for each group, the level of knowledge gained was calculated. A visual and statistical comparison between the ABM and the TRPG results provided insight into the effectiveness of the validation process. Although the combination of RPGs and ABMs is not one of the most popular ABM validation techniques, the results suggest that the approach can lead to an increase in knowledge, particularly for participants with backgrounds in medicine and disaster risk management. Visual analysis of the results revealed a similar trend in the investigated ambulance policies. However, a statistical analysis comparing the results of the different approaches showed clear differences, highlighting the challenges in validating ABM using RPGs. The game scenario therefore not only offers the opportunity to practice basic skills, but also illustrates the importance of timely and efficient coordination of various health facilities and emergency services. By providing a detailed analysis of the results of these strategies in the ABMS model and game environment, public health practitioners and policymakers can use this combined approach to develop more evidence-based and informed policies and protocols tailored to specific disaster response conditions.
Additionally, the combination of the DRTRPG and NetLogo (ABMS) approaches shows great promise as a method that can be used in evaluating contingency plans. The methodological approach of this study resulted in a realistic simulation and a platform that can be customized for staff training and decision-making. The application of these strategies demonstrated the need for improved disaster response and reduction in mortality rates. These collections of generative science and gamification techniques, can help develop a valuable framework for consideration in subsequent research.

## Limitations and Future Study

Although using TRPGs to simulate healthcare disaster situations offers stakeholders the opportunity to refine their skills and improve their decision-making ability in a safe and managed scenario, there are limitations to using RPGs to validate agent-based models. One limitation is that the game often does not have a complete representation of the scenario in the computational model. Additionally, since the study focuses on predicting mortality outcomes in simplified systems, other factors that could lead to other secondary outcomes from interaction within the model and game (e.g., coordination, clinical staff, etc.) were not fully integrated in the DRTRPG strategy. Therefore, it is suggested that future ABM models should aim to involve stakeholders in the initial model development process. In addition, a dynamic online version of the game is proposed to reduce bias among participants and provide larger data sets for more robust statistical analysis.

## Disclosure Statement

With regard to this work, the authors declare that there are no financial or personal relationships that could be construed as a conflict of interest.

## Acknowledgements

The authors thank everyone who participated in the exercises, especially the master's and doctoral degree students of the Disaster Management and Environmental Impact, Laboratoire CERIS of IMT Mines Alés, who made themselves available for both the training sessions and the actual exercises. We also thank the Petroleum Technology Development Fund (PTDF) for funding this doctoral study. Nimisingha Jacob Amakama: Conceptualization, Writing, Methodology, Implementation, Data Collection and Analysis, Visualization. Gilles Dusserre: Supervision, Reviewing, Conceptualization, Methodology Review. Axelle Cadiere: Supervision, Reviewing, Conceptualization. R. Wade Schuette: Conceptualization, Model Coding, Simulation and Analysis, Visualization, Reviewing. Gregory Zacharewicz: Conceptualization, Visualization, Reviewing.

## REFERENCES

Ademiluyi, F., Amadi, S., & Amakama, N. (2010). Adsorption and Treatment of Organic Contaminants using Activated Carbon from Waste Nigerian

Bamboo. *Journal of Applied Science & Environmental Management, 13*(3). https://doi.org/10.4314/jasem. v13i3.55351

Alejandria, L. N., Bajenting, J. M. S., Pacatan, M. a. L. D., & Diquito, T. J. A. (2023). The use of educational board game as a supplemental tool in learning periodic table of elements among senior high school students. *American Journal of Education and Technology, 2*(1), 60–67. https://doi.org/10.54536/ajet.v2i1.1292

Alonso-Fernández, C., Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2021). Improving evidence-based assessment of players using serious games. *Telematics and Informatics, 60*, 101583. https://doi. org/10.1016/j.tele.2021.101583

Alonso-Fernández, C., Martínez-Ortiz, I., Caballero, R., Freire, M., & Fernández-Manjón, B. (2019). Predicting students' knowledge after playing a serious game based on learning analytics data: A case study. *Journal of Computer Assisted Learning, 36*(3), 350–358. https://doi.org/10.1111/jcal.12405

Amakama, N. J., Dusserre, G., Cadiere, A., & Schuette, R. W. (2023a, September 5). Assessing the impact of wait times on patient mortality outcomes in a hypothetical oil and gas industry disaster scenario: An Agent-Based Modeling Approach using NetLogo. https:// imt-mines-ales.hal.science/hal-04199308

Amakama, N. J., Dusserre, G., Cadiere, A., Schuette, R. W., Zacharewicz, G., & Ahmed, N. (2023b, September 11). An Evaluation of the Effect of Ambulance Strategy on Patient Mortality Outcome from Hospital Systems: a Simulation Study in the Context of Industrial Disaster Response. https://imt-mines-ales. hal.science/hal-04208574

Anand, N., Meijer, D., Van Duin, J., Tavasszy, L., & Meijer, S. (2016). Validation of an agent-based model using a participatory simulation gaming approach: The case of city logistics. Transportation Research. Part C, Emerging Technologies, 71, 489–499. https://doi. org/10.1016/j.trc.2016.08.002

Antelmi, A., Cordasco, G., D'Ambrosio, G., De Vinco, D., & Spagnuolo, C. (2022). Experimenting with Agent-Based Model Simulation Tools. *Applied Sciences, 13*(1), 13. https://doi.org/10.3390/app13010013

Bowman, S. L. (2010). The functions of role-playing games: How participants create community, solve problems and explore identity. *McFarland*.

Cheng, P. H., Yeh, T. K., Chao, Y. K., Lin, J., & Chang, C. Y. (2020). Design ideas for an Issue-Situation-Based board game involving multirole scenarios. *Sustainability, 12*(5), 2139. https://doi.org/10.3390/ su12052139

D'Aquino, P., & Bah, A. (2014). Multi-level participatory design of land use policies in African drylands: A method to embed adaptability skills of drylands societies in a policy framework. *Journal of Environmental Management, 132*, 207–219. https://doi.org/10.1016/j. jenvman.2013.11.011

De Rouck, R., Benhassine, M., Debacker, M., Dugauquier,
C., Dhondt, E., Van Utterbeeck, F., & Hubloue, I. (2023). Creating realistic nerve agent victim profiles for computer simulation of medical CBRN disaster response. *Frontiers in Public Health, 11*. https://doi.org/10.3389/fpubh.2023.1167706

Deinkuro, N. S., Knapp, C. W., Raimi, M. O., & Nimlang, N. H. (2021). Environmental Fate of Toxic Volatile Organics from Oil Spills in the Niger Delta Region, Nigeria. *International Journal of Environment, Engineering and Education, 3*(3), 89–101. https://doi.org/10.55151/ijeedu.v3i3.64

Dubois, E., Barreteau, O., & Souchere, V. (2013). An Agent-Based model to explore game setting effects on attitude change during a role-playing game session. *JASSS, 16*(1). https://doi.org/10.18564/jasss.2065

Fleming, K., Abad, J., Booth, L., Schueller, L., Baills, A., Scolobig, A., Petrovic, B., Zuccaro, G., & Leone, M. (2020). The use of serious games in engaging stakeholders for disaster risk reduction, management and climate change adaption information elicitation. *International Journal of Disaster Risk Reduction, 49*, 101669. https://doi.org/10.1016/j.ijdrr.2020.101669

Gore, R. J., Lynch, C. J., & Kavak, H. (2016). Applying statistical debugging for enhanced trace validation of agent-based models. *Simulation, 93*(4), 273–284. https://doi.org/10.1177/0037549716659707

Guiamalodin, N. M., Culele, R. G. A., Silguerra, J., & Diquito, T. J. A. (2024). Unfolding the Experiences and Preparedness Mechanism of Senior Citizens in the Wake of Disaster: Towards a Disaster Resilient Community. *American Journal of Society and Law, 3*(1), 1–13. https://doi.org/10.54536/ajsl.v3i1.2299

Guyot, P., Drogoul, A., & Honiden, S. (2007). Multi-Agent participatory simulations between experimental economics and Role-Playing games. In *Agent-based social systems* (pp. 11–19). https://doi.org/10.1007/978-4-431-71307-4_2

Guzdial, M., Acharya, D., Kreminski, M., Cook, M., Eladhari, M., Liapis, A., & Sullivan, A. (2020). Tabletop Roleplaying Games as Procedural Content Generators. *International Conference on the Foundations of Digital Games*. https://doi.org/10.1145/3402942.3409605

Hosseini, M. M., Hosseini, T. M., Qayumi, K., & Baeradeh, N. (2022). Game-based vs. Case-based Training for Increasing Knowledge and Behavioral Fluency of Nurse Students Regarding Crisis and Disaster Management; a Quasi- Experimental Study. *DOAJ (DOAJ: Directory of Open Access Journals)*, 10(1), e77. https://doi.org/10.22037/aaem.v10i1.1739

Husna, C., Kamil, H., Yahya, M., Tahlil, T., & Darmawati, D. (2020). Does tabletop exercise enhance knowledge and attitude in preparing disaster drills? *Nurse Media: Journal of Nursing/Nurse Media Journal of Nursing, 10*(2), 182–190. https://doi.org/10.14710/nmjn.v10i2.29117

Introduction to Agent-Based modeling. (2019). In *SAGE Publications Ltd eBooks* (pp. 14–34). https://doi.org/10.4135/9781529793543.n2

Kwadwo, W., & Morkla, T. D. (2024). Occupational health and safety challenges faced national disaster management personnel. *American Journal of Arts and Human Science, 3*(1), 19–23. https://doi.org/10.54536/ajahs.v3i1.2406

Ligtenberg, A., Van Lammeren, R. J., Bregt, A. K., & Beulens, A. J. (2010). Validation of an agent-based model for spatial planning: A role-playing approach. *Computers, Environment and Urban Systems, 34*(5), 424–434. https://doi.org/10.1016/j.compenvurbsys.2010.04.005

Linehan, C., Lawson, S., & Doughty, M. (2009). Tabletop prototyping of serious games for “Soft skills” training. *2009 Conference in Games and Virtual Worlds for Serious Applications, 182–185*. https://doi.org/10.1109/vs-games.2009.9

Liu, C., Zhu, E., Zhang, Q., & Wei, X. (2021). Exploring the effects of computational costs in extensive games via modeling and simulation. *International Journal of Intelligent Systems, 36*(8), 4065–4087. https://doi.org/10.1002/int.22450

Liu, Y., Fang, A., Moriarty, G., Kraut, R., & Zhu, H. (2023). Agent-based simulation for online mental health matching. *arXiv (Cornell University)*. https://doi.org/10.48550/arxiv.2303.11272

Macal, C. M. (2016). Everything you need to know about agent-based modelling and simulation. *Journal of Simulation, 10*(2), 144–156. https://doi.org/10.1057/jos.2016.7

Mariano, D. J. K., & De Maria Albuquerque Alves, C. (2020). The application of role-playing games and agent-based modelling to the collaborative water management in peri-urban communities. *RBRH, 25*. https://doi.org/10.1590/2318-0331.252020190100

Marks, R. E. (2007). Validating Simulation Models: A general framework and four applied examples. *Computational Economics, 30*(3), 265–290. https://doi.org/10.1007/s10614-007-9101-7

McCourt, E., & Watson, K. (2023). Creating a disaster ready pharmacy workforce: Evaluation of a Disaster tabletop exercise. *Prehospital and Disaster Medicine, 38*(S1), s106–s107. https://doi.org/10.1017/s1049023x2300290x

McDonald, G. W., & Osgood, N. D. (2023). Agent-Based Modeling and its Tradeoffs: An Introduction & Examples. *arXiv (Cornell University)*. https://doi.org/10.48550/arxiv.2304.08497

Meijer, S., Hofstede, G. J., Beers, G., & Omta, S. W. F. (2006). Trust and Tracing game: learning about transactions and embeddedness in a trade network. *Production Planning & Control, 17*(6), 569–583. https://doi.org/10.1080/09537280600866629

Ogie, R., O'Brien, S., & Federici, F. (2022). Towards using agent-based modelling for collaborative translation of crisis information: A systematic literature review to identify the underlying attributes, behaviours, interactions, and environment of agents. *International Journal of Disaster Risk Reduction, 68*, 102717. https://doi.org/10.1016/j.ijdrr.2022.1167706
doi.org/10.1016/j.ijdrr.2021.102717

Olalekan, M. R., Albert, O., Iyingiala, A. A., Sanchez, D. N., Telu, M. (2022). An environmental/scientific report into the crude oil spillage incidence in Tein community, Biseni, Bayelsa state Nigeria. *J Environ Chem Toxicol. 2022; 6*(4), 01-06., doi:10.37532/ pulject.2022.6(4);01-06.

Olalekan, R. M., Omidiji, A. O., Nimisngha, D., Odipe, O. E., & Olalekan, A. S. (2018). Health Risk Assessment on Heavy Metals Ingestion through Groundwater Drinking Pathway for Residents in an Oil and Gas Producing Area of Rivers State, Nigeria. *Open Journal of Yangtze Oil and Gas, 03*(03), 191-206. https://doi.org/10.4236/ojogas.2018.33017

Onggo, B. S., & Karatas, M. (2016). Test-driven simulation modelling: A case study using agent-based maritime search- operation simulation. *European Journal of Operational Research, 254*(2), 517-531. https://doi.org/10.1016/j.ejor.2016.03.050

Pasinetti, N. J., Alvarado, Y., Fernández, J., Guerrero, R. A., & Rodríguez, G. (2015). A serious game about recycling rules. XXI Congreso Argentino De Ciencias De La Computación (Junín, 2015). http://sedici.unlp.edu.ar/bitstream/handle/10915/50227/Documento_completo.pdf?sequence=1

Pira, M. L., Marcucci, E., & Gatta, V. (2017). Role-playing games as a mean to validate agent-based models: an application to stakeholder-driven urban freight transport policy-making. *Transportation Research Procedia, 27*, 404-411. https://doi.org/10.1016/j.trpro.2017.12.060

Poděbradská, M., Noel, M., Bathke, D. J., Haigh, T. R., & Hayes, M. J. (2020). Ready for drought? A Community Resilience Role-Playing game. *Water, 12*(9), 2490. https://doi.org/10.3390/w12092490

Raimi, M. O., Iyingiala, A., Sawyerr, O. H., Saliu, A. O., Ebuete, A. W., Emberru, R. E., Sanchez, N. D., & Osungbemiro,

Raimi, M. O., Iyingiala, A. A., Sawyerr, O. H., Saliu, A. O., Ebuete, A. W., Emberru, R. E., ... & Osungbemiro, W. B. (2022). Leaving no one behind: impact of soil pollution on biodiversity in the global south: a global call for action. In *Biodiversity in Africa: potentials, threats and conservation (pp. 205-237)*. Singapore: Springer Nature Singapore.

Raimi, M. O., Adindu, I. B., Udensi, E. O., Funmilayo, A. A., Opufou, T., Deinkuro, N. S., Adekunle, A. P., & Adeniji,

Raimi, M. O., Adindu, I. B., Udensi, E. O., Funmilayo, A. A., Opufou, T., Deinkuro, N. S., ... & Adeniji, A. O. (2020). Health impact assessment: expanding public policy tools for promoting sustainable development goals (SDGs) in Nigeria. *EC Emergency Medicine and Critical Care*.

Raimi, M. O., Ayibatonbira, A. A., Anu, B., Odipe, O. E., & Deinkuro, N. S. (2019). ‘Digging Deeper’Evidence on Water Crisis and Its Solution in Nigeria for Bayelsa State: A Study of Current Scenario. *Int J Hydro, 3*(4),

244-257.

Ramchurn, S. D., Huynh, T. D., Wu, F., Ikuno, Y., Flann, J., Moreau, L., Fischer, J. E., Jiang, W., Rodden, T., Simpson, E., Reece, S., Roberts, S., & Jennings, N. R. (2016). A Disaster Response System based on Human-Agent Collectives. *Journal of Artificial Intelligence Research, 57*, 661-708. https://doi.org/10.1613/jair.5098

Robinson, D. T., Brown, D. G., Parker, D. C., Schreinemachers, P., Janssen, M. A., Huigen, M., Wittmer, H., Gotts, N., Promburom, P., Irwin, E., Berger, T., Gatzweiler, F., & Barnaud, C. (2007). Comparison of empirical methods for building agent-based models in land use science. *Journal of Land Use Science, 2*(1), 31-55. https://doi.org/10.1080/17474230701201349

Rye, S., & Aktas, E. (2022). Serious Games as a validation tool for PREDIS: a decision support system for disaster management. *International Journal of Environmental Research and Public Health/International Journal of Environmental Research and Public Health, 19*(24), 16584. https://doi.org/10.3390/ijerph192416584

Sakai, K., Shimizu, H., Toyoda, Y., & Kanegae, H. (2019). A study on gaming of participatory Evacuation planning in Tourist areas using agent simulation. In *Translational systems sciences* (pp. 247-256). https://doi.org/10.1007/978-981-13-8039-6_23

Sanchez, D. N., Knapp, C. W., Olalekan, R. M., & Nanalok, N. H. (2021). Oil spills in the Niger Delta Region, Nigeria: Environmental fate of toxic volatile organics. *Research Square (Research Square). https://doi.org/10.21203/rs.3.rs-654453/v1

Simões-Marques, M., Moreno, D., & Correia, A. (2020). Developing a serious game to support disaster management preparedness - a design thinking approach. In *Advances in intelligent systems and computing* (pp. 108-115). https://doi.org/10.1007/978-3-030-51369-6_15

Squazzoni, F., Polhill, J. G., Edmonds, B., Ahrweiler, P., Antosz, P., Scholz, G., Chappin, É. J. L., Borit, M., Verhagen, H., Giardini, F., & Gilbert, N. (2020). Computational models that matter during a global pandemic outbreak: a call to action. *JASSS, 23*(2). https://doi.org/10.18564/jasss.4298

Suguitan, L., & Natividad, E. (2022). Localized Game-Based Learning Activity Sheets in Mathematics 7. *American Journal of Multidisciplinary Research and Innovation, 1*(4), 210-227. https://doi.org/10.54536/ajmri.v1i4.723

Suleman, I., Pomalango, Z. B., & Slamet, H. (2022). Disaster exercise tabletop media improves knowledge of health personnel about disaster management. *Jambura Journal of Health Sciences and Research, 5*(1), 90-99. https://doi.org/10.35971/jjhsr.v5i1.16633

Tanguid, S. B. J., & Tanguid, R. V. (2023). Disaster resilience and capability in the maintenance of public order during disaster of Calamity-Prone municipalities in Oriental Mindoro, Philippines. *American Journal of Multidisciplinary Research and Innovation, 2*(5), 83-88.
https://doi.org/10.54536/ajmri.v2i5.2058

Takadama, K., Kawai, T., & Koyama, Y. (2008). Micro-and macro-level validation in agent-based simulation: Reproduction of human-like behaviors and thinking in a sequential bargaining game. *Journal of Artificial Societies and Social Simulation, **11**(2), 9.

Terti, G., Ruin, I., Kalas, M., Láng, I., Alonso, A. C. I., Sabbatini, T., & Lorini, V. (2019). ANYCaRE: a role-playing game to investigate crisis decision-making and communication challenges in weather-related hazards. *Natural Hazards and Earth System Sciences, **19**(3), 507–533. https://doi.org/10.5194/nhess-19-507-2019

Utomo, D. S., Onggo, B. S. S., Eldridge, S., Daud, A. R., & Tejaningsih, S. (2021). Eliciting agents’ behaviour and model validation using role playing game in agent-based dairy supply chain model. *Journal of the Operational Research Society, **73*(12), 2670–2693. https://doi.org/10.1080/01605682.2021.2013137

Villamor, G. B., Van Noordwijk, M., & Troitzsch, K. G. (2023). Triangulating agent-based models, role-playing games, and a stakeholder-centric approach to change scenarios. *Current Opinion in Environmental Sustainability, **64*, 101323. https://doi.org/10.1016/j.cosust.2023.101323