# A MultiAgent Architecture for Collaborative Serious Game applied to Crisis Management Training: Improving Adaptability of Non Player Characters

Erwan Tranvouez, M’hammed Ali Oulhaci, Sébastien Fournier, Bernard Espinasse

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Erwan Tranvouez, M’hammed Ali Oulhaci, Sébastien Fournier, Bernard Espinasse. A MultiAgent Architecture for Collaborative Serious Game applied to Crisis Management Training: Improving Adaptability of Non Player Characters. EAI Endorsed Transactions on Serious Games, 2014, 1 (2), pp.e7. 10.4108/sg.1.2.e7. [hal-03568478](https://hal.science/hal-03568478v1)

HAL Id: hal-03568478

https://hal.science/hal-03568478v1

Submitted on 31 Oct 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A MultiAgent Architecture for Collaborative Serious Game applied to Crisis Management Training: Improving Adaptability of Non Player Characters

M’hammed Ali Oulhaci *,†, Erwan Tranvouez*, Sébastien Fournier*, Bernard Espinasse*

## Abstract

Serious Games (SG) are more and more used for training, as in the crisis management domain, where several hundred stakeholders can be involved, causing various organizational difficulties on field exercises. SGs specific benefits include player immersion and detailed players’ actions tracking during a virtual exercise. Moreover, Non Player Characters (NPC) can adapt the crisis management exercise perimeter to the available stakeholders or to specific training objectives. In this paper we present a Multi-Agent System architecture supporting behavioural simulation as well as monitoring and assessment of human players. A NPC is enacted by a Game Agent which reproduces the behaviour of a human actor, based on a deliberative model (Belief Desire Intention). To facilitate the scenario design, an Agent editor allows a designer to configure agents’ behaviours. The behaviour simulation was implemented within the pre-existing SIMFOR project, a serious game for training in crisis management.

Received on 30 December 2013; accepted on 31 March 2014; published on 22 May 2014

**Keywords:** Serious Game, Multi-agents system, Multi-agent Simulation, Crisis Management.

Copyright © 2014 A. Oulhaci et al., licensed to ICST. This is an open access article distributed under the terms of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/), which permits unlimited use, distribution and reproduction in any medium so long as the original work is properly cited.

doi:10.4108/sg.1.2.e7

## 1. Introduction

Serious Games (SG) for training is growing in number as well as in application domains, and particularly for crisis management. Crisis management objectives is to deal with major events jeopardizing a socio-technical or environmental system stability with potential human, material or ecological costs. One answer can be to train the stakeholders to react accordingly to the crisis gravity (based on predefined plans for example) in order to minimize the crisis consequences. Such training can gather several hundred stakeholders, which can present various difficulties when organizing field exercises.

Serious Game constitutes a more practical alternative with specific benefits. Within a computer environment, the players’ actions during a simulated exercise

can be timely traced (in real time for instant assessment or replayed for debriefing). Moreover, Non Player Characters (NPC) can be used to adapt the crisis management exercise perimeter to the available stakeholders as well as to specific training objectives. Either absent or unnecessary, a stakeholder can be replaced with a simulation allowing the training to go on. The work presented in this paper focuses on adding NPC capabilities to the preexisting SIMFOR Serious Game dedicated to training actors (with various level of expertise) involved in a Crisis Management situation. NPC capabilities implies being able to simulate humans’ behaviours with whom human players interact.

This paper addresses the modelling and architectural requirements supporting these objectives. A general Multi-Agent System (MAS) architecture has thus been proposed enabling the behavioural simulation as well as the monitoring and assessment of human players. To each NPC is associated a so called Game Agent designed to reproduce the behaviour of the actor simulated.

<footnote>* Aix Marseille Université, CNRS, LSIS UMR 7296, 13397 Marseille, FRANCE</footnote>

<footnote>† SII Group, IT Consulting and Engineering Company, Aix-en-Provence, FRANCE</footnote>

<footnote>*ali.oulhaci@lsis.org, erwan.tranvouez@lsis.org
sebastien.fournier@lsis.org, bernard.espinasse@lsis.org</footnote>

<footnote>†aoulhaci@sii.fr</footnote>
The Game Agents are based on a deliberative model (Belief Desire Intention) which is quite usual in MAS for complex behaviour modelling (and simulation). This feature can be summarized by describing an agent as an autonomous entity pursuing multiple goals (possibly with different priorities), which can be attained by plans composed of sequences of actions either applied in the virtual world (3D environment) or resulting in interactions between agents/human players (with a self-evolutionary response). To facilitate the scenario design, we have implemented an agent editor which allows a designer to configure agents' behaviours (as well as dialogues) for a SG scenario, applied here to crisis management training.

The next section presents the SIMFOR project, a serious game for training crisis management. Section 3 discusses the NPC issue and relates to different works in the field of multi-agent systems and behaviours simulation. In section 4 we define our Game Agent model for the SIMFOR project and in section 5, we present a short game scenario in the SIMFOR project to illustrate how the implementation was conducted. Finally we conclude and present future works for the SIMFOR project.

## 2. Serious games and crisis management

In this section, we present the problematic of crisis management and how are Serious Games (SG) involved in this field. We then briefly present the SIMFOR project, a serious game for crisis management as well as the general architecture combining Intelligent Tutoring System (ITS) and SG elements.

### 2.1. Crisis management problematic

In France, crisis management is based on the ORSEC plan (standing for Organization of Civil Security Response). The plan is designed to mobilize and to coordinate under the authority of the prefect, the actors of civil security beyond the prevalent level of response or daily services. The aim is to develop the preparedness of all actors, public or private, who may be involved in population protection. Each actor must seize the tasks within its competence and transcribe them in its internal organization. Moreover, the only way to test these plans is to make exercises in real conditions, which can become very heavy in terms of organization and very expensive. To reduce the cost and saving time, computer tools are solicited among which SG, adding a fun way to learn. With the digital age, many schools and organizations are using SG for training.

Of course, SGs do not replace an ORSEC exercise because there is no pressure and no physical tools manipulation (especially for firefighters). However, SGs help to get used the ORSEC procedures as well as enhancing teamwork. Indeed, crisis management is first

and foremost a collaborative process that aims to restore a normal situation. Thus the objective of the use of SGs is above all to learn to communicate (between stakeholders in crisis management).

Several works have been undertaken in the field of crisis management, such as the PEGASE [1] system (Pedagogical Generic and Adaptive System), a training tool for crisis management on aircraft carrier. Amokrane [2] proposes a training system for SEVESO sites management (ie with high industrial risk) called HERA (Helpful agent for safEty leaRning in Virtual Environment). Buche and colleagues [3] propose the MASCARET model which has been applied to SÉCURÉVI, a computer tool for firefighters training. There are also competition like The RoboCup Rescue project and conferences [4]. This project proposes to compare research works of AI, computer science/engineering, and robotics through international competitions to contribute to serious disaster management.

### 2.2. Serious games needs

Serious Games (and other systems) need some tools to reach their goals. Simulation is one of those needs, which can be used in different ways: to simulate natural disaster [5], fire propagation [6] or to simulate human behaviour as in the RoboCup Rescue project. Such human behaviours can be complex, like decision making or problem-solving processes [7], or simple, like crowd simulation [8] (where reactive behaviours are used for emergency evacuation).

Another need, as SG aims to learn something, is evaluation of learners (players) in order to validate (or not) their training. This implies some form of representation of the knowledge to be learned: in training systems there are many ways to represent knowledge [9], either simply (through data structure), or requiring more complex models such as an ontology [10] or a metamodel [11]. Among existing knowledge representation models, we can cite the SKR model [12] (Skill, Knowledge, Rule), the CTT model [13] (Concur Task Tree), and Tardif [14] model based on declarative knowledge, procedural knowledge, and conditional knowledge, stemming from the TEL domain (Technology Enhanced Learning). Depending on the training objectives, the choice of a representation model will participate in the assessment capability of the training system.

To obtain a snapshot of a learner knowledge state, we must collect information from the learning environment. In the literature this process is named "student analysis". The student analysis aims to extract data and knowledge to improve learning [15]. The tutor (in a broad sense) exploits these traces to determine whether the proposed activity is successful, or to
identify where are the difficulties of the learners. In serious game (and similar tools) student analysis provides support to the learner and enables him to visualize his own activity during training.

In this paper, we will focus on behaviours simulation especially the adaptive aspect of the NPCs. We will show how adaptive NPCs can improve learning in the virtual environment.

## 2.3. SIMFOR context

SIMFOR (figure 1) is a SG developed by SII<fnref n="1" /> company in partnership with Pixxim<fnref n="2" /> company, in response to serious gaming call for project launched by the French Secretary of State for Forward Planning and Development of the digital economy. SIMFOR provides a fun and original approach for learning crisis management as a serious game. SIMFOR is adapted to actors' needs and enables learners to train for major crisis management by integrating multi-stakeholder aspect (ie heterogeneous learning profiles). The project objective is to create a training environment that immerses users in a crisis management situation in real-time context and realistic in terms of environment, self-evolving scenarios and actors.

<figure><img src="image_3.png" /><figcaption>Figure 1. Screenshot from SIMFOR project</figcaption></figure>

SIMFOR is a multi-player game and allows different people to learn skill (shared or specific) in the same game. This is possible because SIMFOR does not target only the specialists in the field of crisis management, but rather the non-professional. Managing a major crisis can mobilize several hundred stakeholders, from the regional Prefect in his office to the firefighter in the field. These stakeholders are required to communicate and work together in order to restore a normal situation. However, SIMFOR at the beginning lacked actors' simulation and assessment capabilities, which our research work intends to complete.

## 2.4. The general architecture of the system

The SIMFOR architecture combines elements from the Intelligent Tutoring System and Serious Game domains (figure 2, see [16] for a detailed presentation). Our goal is to associate the playful learning of SG and the different modules of an ITS (domain model, learner model, pedagogical model) to get the optimal learning environment. The SIMFOR architecture is composed of the following components:

<figure><img src="image_4.png" /><figcaption>Figure 2. General architecture of the system.</figcaption></figure>

* **The SG module (SIMFOR):** this module includes the 3D models, user interface (as a communication channel between the learner and the system), simulation module (for natural phenomena such as fire propagation), and data models. This module constitutes the former "perimeter" of the Simfor SG to which behaviour simulation and user assessment capabilities are added.

* **The Behaviours Simulation module:** which allows simulating humans behaviours to replace absent players with "artificial" actors (Game Agent).

* **The Evaluation module:** the evaluation module provides skills assessment of players in real time to the pedagogical module.

* **The Pedagogical module:** which plays the role of a virtual tutor accompanying the learners by providing support and help during (and after) their training.

* **Knowledge representation module:** All knowledge used or produced by the previous modules of our proposed architecture are stored as an ontology in the domain model and the learner model. The ontology illustrated in figure 3 describe the general domain of crisis management (adapted to the SIMFOR context). The domain model and learner model are described as follows:

<footnote n="1">¹http://www.groupe-sii.com</footnote>

<footnote n="2">²http://www.pixxim.fr</footnote>
- **The Domain model:** the domain model represent the general concepts of crisis management and is segmented into parts representing a role or a skill to learn.

- **The learner model:** for each learner or agent, a learner model is associated, which represents its mental state at a time t.

As this paper focuses on the behaviours simulation module which deals with the non player characters (NPC), the following section exposes the scientific issue of NPCs in the SIMFOR project and some relevant work in this field.

## 3. Adaptive NPC for SIMFOR

To evolve into an full training system, the SIMFOR project faces two issues:

* The simulation of human behaviour of NPC players.

* The monitoring and evaluation of learners during their training.

The learner assessment was discussed in [17] and [16]. To deal with the heterogeneous aspect of the learner assessment (assess different skills and trades), we have proposed the concept of the "Evaluation Space". The guiding idea is to consider a SG scenario through different view, each corresponding to a particular evaluation objective. An "Evaluation Space" thus gathers (homogeneous) information and primitives to manipulate these informations in order to produce assessments, such as a Behavioural Space (for evaluating procedural knowledge) or Social Space (for evaluating actors interaction during a game scenario).

Adding NPC capabilities to the SIMFOR SG implies being able to simulate actors' behaviours with whom human players interact (in the best case scenario without knowing the virtual nature or not of other players). This step requires extracting from domain experts nominal behaviours which players are expected to follow, and expressing them in suitable format. The challenge of behaviours simulation is how to transform an expert domain nominal behaviour to a SG NPC behaviour ? Given the number of stakeholders' as well as their skill heterogeneity, designing a scenario to establish crisis management exercises is a complex task. We present a scenario example in section 3.1 to discuss this issue.

### 3.1. Illustration of NPC and complex behaviour simulation needs

As a SG, SIMFOR aims at immersing players in a virtual world enabling them to pretend acting as they would (and should) do in a real emergency situation. Knowledge and skills involved in such situation are

various in nature as well as in terms of evaluation means, but nonetheless must be all assessed in order to certify (or not) that players know their part of the job on which many lives may depend. To better understand the heterogeneous aspect of the behaviours simulation as well as the assessment needs, let's consider a simplified example of emerging situation scenario.

This scenario starts with a TDM (Transport of Dangerous Material) truck overturned after a traffic accident. The tank is damaged and hydrocarbon is spreading over the road. A witness to the accident gives the alarm by calling the CODIS (Departmental Center for Operational Fire and Rescue Services in french) which in turn must perform four missions consequently to the alert. First, CODIS has to send a Firefighter on the scene to retrieve information about the accident ("send firefighter"). Once information on the accident is received (transmitted by the Firefighter in the ground), confirming a TDM accident has occurred, the CODIS must secondly gives instructions to an Officer (firefighter) on the measures to be taken. In a TDM accident the Officer must give the intervention order (send another Firefighter with a fire truck). Then thirdly, the CODIS must complete an information sheet on the disaster that passes later through a fax to the Mayor, Prefect and the Sub-Prefect (sending order is not important). Finally the last mission is to inform the OCP officer (Operational Command Post) once it is sent by the Prefect.

This scenario excerpt illustrates the needs of the domain for behaviours simulation (as it defines nominal agent behaviour), as well as for the assessment process (if the CODIS is played by a human). Actors can play the same role (Firefighter), but enact different behaviours (one collect disaster information, and the other should intervene on the disaster). Moreover, we have trades which do not belong to a sole organization (given the large number of stakeholders in crisis management), like School Principal, Mayor, media, etc. In addition, the actors behaviours may differ from a scenario to another (depending of the disaster nature ie fire, earthquake...). Therefore, the scenario designer (domain expert) must specify the actors involved, their associated behaviours, disasters consequences, etc, for each scenario exercise. In the next section we present some works in the field of behaviour simulation and multi-agents architecture and how our work relates to these works.

### 3.2. Related work

Among the technologies used in SGs, Artificial Intelligence (AI) has always been present in video games, more or less elaborated (depending on the video game objective) [18]. SGs borrow much from classic video games, but a SG is more than a game, a SG is a game that is used to learn something. The tools and
<figure><img src="image_5.png" /><figcaption>Figure 3. An ontology describing the general domain of SIMFOR.</figcaption></figure>

mechanisms borrowed to the video game must be more elaborated, like AI capabilities.

AI is used to simulate human behaviours or even natural phenomena to obtain a realistic virtual world for training [19]. In [3], Buche and colleagues propose MASCARET, a pedagogical multi-agent system for virtual environment for training. The MASCARET model organizes the interactions between agents and endows them with reactive, cognitive and social abilities to simulate the physical and social environment. The physical environment represents, in a realistic way, natural phenomena. The social environment is simulated by agents executing collaborative and adaptive tasks. The MASCARET model was applied to SÉCURÉVI, an application for fire-fighters training. This model allows to design a complex organisation (by defining role, organisation, behaviours, ...), but is difficult to transpose in situations where many and heterogeneous organisations interact. For each organisation must be defined roles and procedure etc, which is not adapted for larger scale crisis management exercises. In addition, the SÉCURÉVI project does not include any assessment solution.

More specifically on multiagent organisational modelling, the MOISE+ model [20] (Model of Organisation for multi-agent SystEmS) considers organisational structure and dynamics of a MultiAgent System or MAS (for example for simulation purpose). This model adds an explicit deontic relation (to structure and dynamics)

to make the (artificial) agents able to reason on the fulfilment of their obligations or not. In these models, obligations and permissions are entitled to roles (such as a firefighter has to extinguish a fire and may use a fire hose which a bystander may not). As training may require actors to detect wrong behaviours (during a collaborative task between a simulated actor and human player), we would also need to allow (voluntary) erroneous behaviours which is not covered by MOISE+.

To simulate NPCs in the SIMFOR project, we have opted for a BDI architecture. The BDI (Beliefs, Desires, Intentions) model is an agent modelling standard in the field of agent behaviour modeling, inspired from the human reasoning process [21], and has been widely applied. A BDI model is based on the notion of capacity, skills, beliefs, purpose, desire, and intention-plan. Agents aims at achieving their goals by executing plans depending of their current knowledge (beliefs). These concepts allows designing and programming agents with complex behaviours. Our goal in the SIMFOR project, is to provide an agent architecture helping the domain expert to design NPCs for a specific scenario. These NPCs can have a nominal behaviours (perform the expected behaviours) or intentionally erroneous behaviours. The NPCs must also adapt their behaviours related to other players (learners) actions and interaction (social abilities) as well as events from the environment (3D world) that can occur during the game.
## 4. A BDI architecture for SIMFOR

The BDI-like proposed model is composed of a set of agent, actions and facts. Each SIMFOR scenario is associated with several agents model that reflect the NPCs behaviours. An agent model is represented as follows:

$$Model_{(role)} = \{\text{Goals}, \text{Plans}, \text{Facts}, \text{Dialogues}\} \quad (1)$$

A Game Agent (GA) will play a role in a scenario, and as such tries to achieve Goals (activated ie evaluated as reachable, by the context) by enacting its associated plans depending on its knowledge of the situation (defined as a list of declarative Facts). Each plan is composed of actions either directed toward the environment or other agents/actors (causing different type of "effects" on the scenario as defined in the ontology in figure 3). In the former case, interactions between Game Agent and human actors (or between GAs to fully simulate a scenario) are codified by adaptive Dialogues as a set of Sentences (see section 4.2). Figure 4 synthesizes the general structure of the GA model with an UML metamodel.

<figure><img src="image_6.png" /><figcaption>Figure 4. The SIMFOR Game Agent metamodel</figcaption></figure>

On a architectural or software level, an agent Engine is defined in order to enliven the agents (ie enact the agents life cycle). A Game Agent (GA) begins to update its Facts Base through its perception of the environment (events) and messages from other agents (or players), it then selects the appropriate goal according to the situation (role, ...) and the intention (plan) that will achieve this goal, and finally execute the plan. For goal selection, the GA identifies first realizable goals by studying the plans feasibility for each goal. Once the realizable goals known, the GA chooses the highest priority goal. If there are several goals with the same priority, the choice is randomly made. This process

enables an agent to adapt its behaviour to the state of the environment in a broad sense (ie information about the virtual world and the other actors whether incarnated by agents or not).

The objective of this model is to provide adaptive NPC behaviours, but also to help the scenario designer(s) with a user-friendly and efficient tool to configure these NPCs behaviours. Thus, a graphical editor tool has been developed as illustrated in section 5.1 applied to the risk management case study.

### 4.1. Action modelling

Actions characterize what an actor can do during a crisis management situation. These actions thus guide the design of agents' behaviours as well as provide assessment data when comparing what is done to what should have been done by human actors. Regarding the Game Agent, an action has *Preconditions* (expressed by a set of facts supposed to be present in the agent Facts Base) and *Effects* (see metamodel in fig. 4). An action can be performed in several ways: one shot (action undertaken only once), cyclic (repetitive action such as "check fire progression") or performed at a time *t*. For each action the designer can specify the number of attempts (if the number is reached without success, the agent goes to the next action).

Actions influence the environment through three kind of *Effects*:

* **Physical effects (PE):** such action influences/impacts the SIMFOR 3D environment (as would a human actor through its user interface).

* **Knowledge modification effect (KE):** direct consequence of an action are facts modification (ie knowledge update) which in turn can abort a goal or validate other goals or actions preconditions.

* **Message Effect (ME):** such effect reflects the social nature of agent and actors as in the SIMFOR SG, they communicate in order to carry out the collaborative task of managing a crisis situation.

Table 4.2 illustrates actions with various effect as defined in the SIMFOR SG. Some actions may only carried out by GA (NPCs). These actions help to enrich the simulation and make it more realistic to human actors. For example, a (virtual) mayor assistant may prepare a room for a press conference. This action will take some time during which the (human) mayor cannot start the press conference. These kind of actions can also trigger some physical change in the SIMFOR 3D environment, for example "activate an emergency siren", or "put in place a foam pad" (firefighter action). These virtual actions are defined by the scenario
<table><caption>**Table 1.** Example of Crisis Management Actions and their effects.</caption><thead><tr><td>Action</td><td>Effect type</td><td>Description</td></tr></thead><tbody><tr><td>Phone</td><td>KE, ME</td><td>This action allows to join a player by phone. If both interlocutors are human players, the communication will be oral via VOIP (Voice over IP). If one of the interlocutors is a GA, the communication will be done as a dialogue with textual phrases exchange (see section 5.2).</td></tr><tr><td>Fax</td><td>KE, ME</td><td>This action allows to send a fax to one or more recipients. Faxes are represented by preformatted HTML documents related to crisis management (to be filled in with the right informations).</td></tr><tr><td>Radio</td><td>KE, ME</td><td>This action allows to join a players by radio. The player must then select a channel and press the talk button to communicate with all actors listening to this channel. To add a radio action into a plan, the designer must specify the channel and the id of the dialogue that will be used (by GA).</td></tr><tr><td>Talk</td><td>KE, ME</td><td>This action allows to talk with the nearest players (with a defined perimeter).</td></tr><tr><td>Move</td><td>PE</td><td>This action allows to move in the virtual environment. The players can move using the mouse or automatically by selecting a chosen address in address book (either on foot or using a vehicle).</td></tr><tr><td>Daybook</td><td>KE</td><td>the daybook simulates a web portal that allows stakeholders to relay information on the disaster. The player writes the information to be shared and will be available to all stakeholders.</td></tr></tbody></table>

designer detailing their execution time, preconditions and effects.

## 4.2. Dialogue modelling

There are several ways to interact in SIMFOR (phone, fax, radio...) but if one of the interlocutors is a GA, the communication is done in the form of textual dialogue. Partial automated interaction may result in fixed interaction lacking flexibility to reflect the various situations actors and agent may face. In order to avoid that, we have defined a dialogue process as a set of possible *Sentences* (see fig. 4), each one characterized with a context pertaining to the agents perception of their environment. A dialogue is thus designed as a sentence tree, where each node is represented by a sentence. A sentence is characterized by a list of properties described in table 2.

In section 5.2, we will see a dialogue example and how the dialogue can be adaptive and interactive.

## 5. Implementation

In this section we present how was carried out the agent model and the corresponding editor tool, on a software level.

### 5.1. The agent editor

The agent editor allows the designer to set up the NPCs behaviours. The SIMFOR scenario specifies the actors, means and disasters involved during the game as well as the associated events (fire accident for example). For each SIMFOR actors (ie roles), the designer must associate an NPC. If the actor is not played by a human participant, it will be simulated by a GA, and conversely the GA will be disabled. Moreover, if the game begins with a NPC and a learner want to join in an ongoing game and play the role of the NPC, the GA will be disabled. Also, if a learner plays a role and for some reason leaves the game, the NPC takes over and plays his role.

<figure><img src="image_7.png" /><figcaption>**Figure 5.** The Game Agent edition.</figcaption></figure>

For each SIMFOR scenario, the designer (domain expert) uses the agent editor to specify facts, actions for each agent (GA) involved in the scenario (figure 5). Facts are used for actions' preconditions and effects, for goal preconditions and for dialogues. The Facts base represents the facts known by the GA at the beginning of the game. Action(s) represent the actions that can be realized by the GA (these actions will be used to define plans). The trigger events are facts which can make goals realizable (cf. agent engine in section 4). A goal is defined by its name, a priority, preconditions, previous goal (goals that must be previously completed) and finally a set of plans (permitting to achieve this goal). Each plan is represented by a set of (ordered) actions. We will see in more details the dialogues between human players and NPCs in section 5.2.

### 5.2. The dialogue editor

In crisis management scenario, the different stakeholders must collaborate to restore the normal situation. To do this, there are several interaction processes between actors during the game. In accordance with the Dialogue and Sentence concepts defined in section 4.2, the designer can make rich and interactive dialogue with adaptive response. Within the agent editor, a dialogue editor helps the dialogue and sentence design (figure
<table><caption>**Table 2.** Structure of a Sentence.</caption><thead><tr><th>Sentence Attribute</th><th>Description</th></tr></thead><tbody><tr><td>Id</td><td>identifies the sentence in the Dialogue tree</td></tr><tr><td>Parent</td><td>list of (possibly) preceding sentences (parent nodes in the Dialogue tree)</td></tr><tr><td>Children</td><td>list of (possibly) following sentences (children nodes)</td></tr><tr><td>Content</td><td>represents the displayed text in the dialogue box</td></tr><tr><td>Preconditions</td><td>contains the preconditions (set of facts) required for the GA to answer this sentence.</td></tr><tr><td>Transmitted information</td><td>represents the fact transmitted when the sentence is answered (by GA or human player)</td></tr><tr><td>Display type</td><td>defines if the phrase will be displayed for all roles, only roles in the role list or all roles except the roles in the role list</td></tr><tr><td>Roles list</td><td>a list of roles involved in the sentence, used for display</td></tr></tbody></table>

6) and saves it in a XML format. We can also import dialogues (of phrase) to create and reuse more complex dialogues.

Figure 7 presents an example of dialogue. This dialogue represents an interaction between the actor *Codis* and the actor *Fireman1*. The *Codis* must inform the *fireman1* of the accident, and the *fireman1* can answer with two choice: *Ask for the road closures or not*. The response of *Codis* actor will depend on its fact base (precondition *road blocked*). In figure 6 we can see the tree structure of the dialogue as well as the different sentences properties (id, content, precondition etc as described in table 2).

## 5.3. The agent/environment interaction

The Agent model designed for behaviours simulation is implemented as a library and is completely generic. As agent actions may influence/modify the 3D environment, a communication interface with the SIMFOR 3D environment is required. This communication is based on *Commands*, which drive the behaviours simulation model (GA), but are also exploited as learner traces for the learner support and skill assessment.

For example, when the GA wants to perform a move action, the Agent engine sends a *Move* command with the necessary parameters (destination, means of transport used). SIMFOR processes the request: if the 3D avatar of the GA is near the vehicle and the vehicle is available, SIMFOR carries out the trip (as a 3D animation), otherwise, SIMFOR sends an error command. This process can relate to the MASQ (Multi-Agent Systems based on Quadrants) model [22]. The MASQ model separates the agent mind (decisions) and the agent body (actions). A mind corresponds to the internal structure of an agent or to the decision-making component. The body, either physical or social are parts of the environment and are connected to minds. As with SIMFOR, the mind of the agent is represented by the GA, and the body of the agent is represented by the 3D avatar in the virtual environment.

In the next section, based on the example presented in section 3.1, we illustrate how GA behaviours as well as interactions between players and GA can be edited.

## 5.4. Case study

To illustrate the behaviours simulation, we present a scenario example defined by a domain expert, which describes the interaction between GA and a human player. For this, we resume the scenario presented in section 3.1.

Considering the CODIS role, figure 8 shows an UML activity diagram of the GA. With the given scenario (defined in section 3.1), we have defined and attributed five goals to the GA that play the role of the CODIS.

The first goal "send firefighter" is triggered when the CODIS receives the alert (from a witness of the accident), who then sends a firefighter at the disaster scene. This goal is reached by calling the firefighter and ordering him to go to the disaster scene (dialogue process).

The second goal "give instruction to the officer" consists in informing the officer of the initial actions to undertake concerning the TDM. The CODIS calls the officer and with a dialogue process gives the procedures to follow.

The third goal "Follow up the fire-fighter for disaster information" will be triggered if the firefighter is slow to transmit information. This goal is achieved by a cyclic action repeated until the disaster information is received.

The fourth goal "warn municipal officials" consists in warning municipal officials by filling in the information sheet of the disaster and faxing it to the Prefect, Mayor and Sub-prefect. If the CODIS is played by a human player, the information sheet will be filled in manually by the player, if not, the GA will use a pre-filled sheet based on its Facts base content (disaster information).

Finally the last goal "inform officer for PCO place", aims at informing the officer of the PCO location, by selecting the right address (found in the Facts base) and calling the officer. This Behaviour model can be used for behaviours simulation, but also for learner assessment,
<figure><img src="image_8.png" /><figcaption>Figure 6. The dialogue and phrase edition.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 7. A dialogue example from the TDM scenario.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Figure 8. Codis activity diagram</figcaption></figure>

as it is part of the domain model [23] and is used as a reference (overlay model) for the learner model [24] (learner's actions and knowledge).

During the game, we can follow each GA actions, its current goal, current plan and the content of its facts base. We can also modify the GA behaviour through the interface, for example we can reset a goal, add new fact
(to trigger some goal), etc. This flexibility in the control of the NPCs is very useful because crisis management is a collaborative process, and the GA behaviours can influence the learner performance. In [17], we have presented different kinds of learner assessment, for example the collective assessment assesses the global performance of all the stakeholders. This performance takes into account the human players (learner) as well as the NPCs.

## 6. conclusion

With the growing interest in SG for training purpose, the behaviours simulation of NPCs is increasingly relevant. In this paper, we have presented behaviours simulation could challenge SG into better training simulation. Consequently, in order to address this challenge, we have proposed a BDI-like Game Agent architecture to simulate the NPCs.

The goal of this implementation is to cover all trades of crisis management stakeholders and facilitate agent programming for better design of crisis management scenario. This Game Agent model is integrated into SIMFOR project, a serious game for crisis management. The GAs interact with the SIMFOR environment through command system as well as human players through a dialogues system. This integration is used for behaviours simulation but also plays a role in the learners assessment. The crisis management is a collaborative process, and the learners player and GA must collaborate to restore the situation. The GAs behaviours can influence learners performance and the GAs behaviours can be intentionally erroneous to evaluate the learners' behaviour in reaction to these errors. Our immediate work in the SIMFOR project is to focus on the collaborative aspects in the field of crisis management, based on an analysis of the interaction graph permitting real-time interpretation for better pedagogical support. On a more medium-term perspective, primary feedback on our conceptual and architectural proposition reveals sufficient genericity to consider applying our approach to other SG training situation.

## References

[1] BUCHE, C., BOSSARD, C., QUERREC, R. and CHEVAILLIER, P. (2010) Pegase: A generic and adaptable intelligent system for virtual reality learning environments. *International Journal of Virtual Reality* 9(2): 73.

[2] AMOKRANE, K., LOURDEAUX, D. and BURKHARDT, J.M. (2008) Hera: Learner tracking in a virtual environment. *IJVR* 7(3): 23–30.

[3] BUCHE, C., QUERREC, R., DE LOOR, P. and CHEVAILLIER, P. (2003) Mascaret: pedagogical multi-agents systems for virtual environment for training. In *Cyberworlds, 2003. Proceedings. 2003 International Conference on* (IEEE): 423–430.

[4] KITANO, H., TADOKORO, S., NODA, I., MATSUBARA, H., TAKAHASHI, T., SHINJOU, A. and SHIMADA, S. (1999) Robocup rescue: Search and rescue in large-scale disasters as a domain for autonomous agents research. In *Systems, Man, and Cybernetics, 1999. IEEE SMC'99 Conference Proceedings*. 1999 IEEE International Conference on (IEEE), 6: 739–743.

[5] KOMATITSCH, D., TSUBOI, S., JI, C. and TROMP, J. (2003) A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte earthquake simulation on the earth simulator. In *Proceedings of the 2003 ACM/IEEE conference on Supercomputing (ACM)*: 4.

[6] OHGAI, A., GOHNAI, Y. and WATANABE, K. (2007) Cellular automata modeling of fire spread in built-up areas. *AIAA Tool to aid community-based planning for disaster mitigation*. *Computers, environment and urban systems* 31(4): 441–460.

[7] OULHACI, M.A., TRANVOUEZ, E., FOURNIER, S. and ESPINASSE, B. (2013) A multiagent architecture for collaborative serious game applied to crisis management training: improving adaptability of non played characters. In *7th European Conference on Games Based Learning* (acpi): 465–474.

[8] HELBING, D., FARKAS, I.J., MOLNAR, P. and VICSEK, T. (2002) Simulation of pedestrian crowds in normal and evacuation situations. *Pedestrian and evacuation dynamics* 21: 21–58.

[9] TCHÉTAGNI, J. and NKAMBOU, R. (2002) Hierarchical representation and evaluation of the student in an intelligent tutoring system. In *Intelligent Tutoring Systems* (Springer): 708–717.

[10] PROVITOLO, D., MÜLLER, J. and DUBOS-PAILLARD, E. (2009) Validation of an ontology of risk and disaster through a case study of the 1923 great kanto earthquake. *ICCSA Proceedings Special Sessions*: 140–150.

[11] BUCHE, C. (2005) *Un système tutoriel intelligent et adaptatif pour l’apprentissage de compétences en environnement virtuel de formation*. URL http://books.google.fr/books?id=4Q_h0wAACAAJ.

[12] RASMUSSEN, J. (1983) Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. *Systems, Man and Cybernetics, IEEE Transactions on* (3): 257–266.

[13] MORI, G., PATERNÒ, F. and SANTORO, C. (2002) Ctte: support for developing and analyzing task models for interactive system design. *Software Engineering, IEEE Transactions on* 28(8): 797–813.

[14] TARDIF, J. (1993) L’analyse dans le paradigme constructiviste. *Hivon R. (Dir.) L’analyse des apprentissages. Réflexions, nouvelles tendances et formation*. Sherbrooke: Editions du CRP (Faculté d’éducation, Université de Sherbrooke): 27–56.

[15] BAUDOIN, C., BENEY, M., CHEVAILLIER, P. and LE PALLEC, A. (2007) Analysis of virtual helps usage in a virtual environment for learning. In *Proceedings of the 4th International INTUITION Conference*: 52–60.

[16] OULHACI, A., TRANVOUEZ, E., FOURNIER, S. and ESPINASSE, B. (2013) Intelligent tutoring systems and serious game for crisis management: a multi-agents integration architecture. In *IEEE International conference on state-of-the-art research in enabling technologies for collaboration in*
CT2CM Track (IEEE).

[17] OULHACI, A., TRANVOUEZ, E., FOURNIER, S. and ESPINASSE, B. (2013) A multi-agent system for learner assessment in serious games: Application to learning processes in crisis management. In *Seventh IEEE International Conference on Research Challenges in Information Science (IEEE).*

[18] BAKKES, S., SPRONCK, P. and VAN DEN HERIK, J. (2009) Rapid and reliable adaptation of video game ai. *Computational Intelligence and AI in Games, IEEE Transactions on* **1**(2): 93–104.

[19] ZYDA, M. (2005) From visual simulation to virtual reality to games. *Computer* **38**(9): 25–32.

[20] HÜBNER, J.F., SICHMAN, J.S. and BOISSIER, O. (2002) Moise+: towards a structural, functional, and deontic model for mas organization. In *Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 1 (ACM)*: 501–502.

[21] RAO, A., GEORGEFF, M. et al. (1995) Bdi agents, from theory to practice. In *Proceedings of the first international conference on multi-agent systems (ICMAS-95)* (San Francisco): 312–319.

[22] STRATULAT, T., FERBER, J. and TRANIER, J. (2009) Masq: towards an integral approach to interaction. In *Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 2, AAMAS '09 (Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems)*: 813–820. URL http://dl.acm.org/citation.cfm?id=1558109.1558124.

[23] ANDERSON, J. (1988) The expert module. *Foundations of intelligent tutoring systems* : 21–53.

[24] VANLEHN, K. (1988) Student modeling. *Foundations of intelligent tutoring systems* : 55–78.