# ImpactMap: A Collaborative Environment to Support Impact Projection of Complex Decision

Juliana Santos França, André Viana Tardelli, Raffael Souza, Angélica Silva Dias, Marcos Silva Borges

<figure><img src="image_1.png" /></figure>

## To cite this version:

Juliana Santos França, André Viana Tardelli, Raffael Souza, Angélica Silva Dias, Marcos Silva Borges. ImpactMap: A Collaborative Environment to Support Impact Projection of Complex Decision. 4th International Conference on Information Technology in Disaster Risk Reduction (ITDRR), Oct 2019, Kyiv, Ukraine. pp.119-134, 10.1007/978-3-030-48939-7_11. hal-03374231

HAL Id: hal-03374231

https://inria.hal.science/hal-03374231

Submitted on 12 Oct 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# ImpactMap: A Collaborative Environment to Support Impact Projection of Complex Decision

Juliana Baptista dos Santos França<fnref n="1" />, André Viana Tardelli<fnref n="2" />, Raffael Siqueira de Souza<fnref n="2" />, Angélica Fonseca da Silva Dias<fnref n="2" />, Marcos Roberto da Silva Borges<fnref n="2" />,³

³TECNUN, University of Navarra, Donostia/San Sebastián, Spain
julibsf@gmail.com, {andretardelli, raffael.siqueira94}@gmail.com
{angelica.dias, mborges}@ppgi.ufrj.br

**Abstract.** In emergency domain, specialists must make complex decisions to solve problems. Complex decisions are characterized as a complex dynamic system composed of interrelated variables. Complex decisions are made up of actions, and their complexity arises from the surrounding environment, including the context and the behaviors of the individuals involved. It is difficult to isolate the elements that influence such a decision. These decisions lead to unpredictable impacts, causing the need to deal with impacts mitigation in the earlier phases of the decision process and the emergency management cycle. To reach a consensus and solve complex decision problems, collaborative strategies are being used. The general knowledge and different experiences, acquired from the experts through collaborative approaches, is crucial to identify and discuss the impacts (consequences) of decisions actions in a broader way. This paper presents ImpactMap approach, a collaborative environment that allows exchange of ideas and perspectives to discuss and project impacts of complex decisions. This research was evaluated in an emergency simulation and the results achieved showed that this research approach is able to create an interactive environment and supports the impact projection needs of a decision team.

**Keywords:** Complex Decisions, Emergency Management, Impact Projection, Collaborative Decision-Making.

## 1 Introduction

Emergency management is a discipline that deals with risk. Risk represents a broad range of issues and includes a diverse set of players and the necessity to discuss the impacts of their actions on the environment. Emergence management is concerned to the security of everyone and should be consider in everyday situations, not only in disasters situation [1]. According to [2], the emergency management is characterized as the entire rescue planning and intervention process to reduce the impact of emergencies, as well as response and recovery measures to mitigate significant social, economic and environmental consequences for the community.

<footnote n="1">¹ Federal Rural University of Rio de Janeiro (UFRRJ), Brazil</footnote>

<footnote n="2">² Federal University of Rio de Janeiro (UFRJ), Brazil</footnote>
Emergency management should be comprehensive, progressive and not just reactive. Its process should include risk and impact analysis, besides need to consider all phases of the emergency cycle, prioritize actions to minimize present and future impacts on the scenario [3]. Good emergency management should always consider the impact of decisions made on the environment and on the lives reached or attainable.

Within the emergency cycle, there will always be decision making ranging from a high-level individual to a field responder who needs to decide within seconds due to the present risks. In the occurrence of a disaster, specialists, volunteers and government agencies do their best to supply quick and effective responses to the dwelling problem that immediately arises [4]. In such scenario complex decisions are present, and in this research, the definition of complex decision is based on Naturalistic Decision-Making [5] because it describes how decision-makers work in real life problems [6], [7]. Besides it, collaborative aspects are also present and act as resources to support decision-making during emergency management through communication, cooperation and coordination [8] actions.

Decisions with high complexity can be understood as a complex dynamic system [9]. Complex decisions are composed of actions, and those deal with uncertainty [10]. Their impacts (consequences) are interdependent, and the environment in which they exist generates constant change in the decision [11].

The analysis of impact projection of complex decisions is a way to minimize un-expected consequences [3], [12] inside an emergency management, after the decision taken. To enable a broader share of exchanged information, it is necessary to establish a collaborative interaction between decision-makers and specialists in an environment suited for it.

To support decision-makers to analyze complex decisions and project its futures impacts, a research was conducted for the development of an approach materialized into ImpactMap technological environment. Based on fundamentals discussed in [3], [12], [13], this environment is the first version of a collaborative and virtual tool where decision-makers can be part of a group and discuss, analyze and project impacts still in the planning phase of a decision making process. According to [14], collaborative initiatives support decision making process during an emergency response situation.

The first version of ImpactMap proposes also a different visualizations of projection information such as maps, textual descriptions and graphics. Therefore, this tool allows the development of collaborative maps along with communication resources such as instant messages and videoconferences. This research main goal is an approach to support decision makers project complex decision impacts in the environment, especially considering emergency domains. This goal will attend the phases mitigation and preparedness of emergency management cycle.

In literature, it is hard to find works that discuss the impact of a complex decision before the decision's execution in an emergency domain. Based on these arguments, this research will provide a technological and interactive environment able to share ideas, points of view and experiences, supporting decision-makers groups to structure their thoughts.

If a collaborative environment that supports impact projection of complex decision is available, decision-makers will be able to project impact maps, making their thoughts
and experiences externalized in a structured way. Furthermore, it is expected that this
environment promotes further interactions between decision-makers. To evaluate this
work, a simulation was made based on a real-life emergency problem, and groups of
experienced decision-makers conducted the impact projection using the ImpactMap en-
vironment.

This paper is structured through five sections. Section II presents related concepts and work. Section III presents the ImpactMap approach with its fundamentals, functions and characteristics. Following it, section IV presents the research evaluation and section V presents and discusses the results achieved. To finish this paper, section VI presents our conclusions highlights the goals achieved and the limitations of this research.

## 2 Related Concepts and Works

### 2.1 Decision Impacts Investigation

The word Impact is understood as the measure of tangible and intangible consequences
of something upon another. According to Hammond [15], to achieve intelligent choices,
it is necessary to compare the merits of the known alternatives, assessing how well each
one satisfies the decision's fundamental goal, and analyze the consequences of each
decision executed. Some authors defend the possibility to analyze the impact of a de-
cision implemented through its process monitoring. How-ever, it is necessary to wait
on the occurrence of the decisions actions' impact to introduce solutions for the damage
or improvements on the environment and its components.

It is possible to find on literature other works that support an impact analysis be-
yond inference and quantitative results. Others discuss that the decision impact is still
at the beginning of decision analyses – like those on the decision-planning phase [16].
However, most research in this area does not highlights details about how effectively
the impact is projected in practice, not showing how to systematize these projections
[17], [18], [19]. Most of the authors deal with impacts projected in a subjective way.
That is why complex domain had shown difficulty in anticipating the secondary effect
of decision actions. As was discussed in this section, project impact of complex deci-
sions is an action performed by decision makers to minimize unexpected consequences
after the decision implementation.

Regarding emergency domain, tools to support projected impacts usually use geo-
graphic information systems to combine the relevant data and overlay the impact of the
disaster [20]. Regarding this combination, it is possible to identify population, infra-
structure and resources affected by the disaster.

### 2.2 Collaborative Decision Making

The concept of collaboration considers that two or more individuals working together
can reach an equilibrium situation [21]. In it, ideas can be exchanged between the par-
ticipants of the group, generating new knowledge fruits of the collective work. In con-
trast to the task-sharing model, the collaboration aims for teamwork establishment, fo-
cusing on synchronous activities of continuous efforts that help maintain a shared con-
ception of the problem.

Implementing collaboration in an environment allows for increased processing power of information. In a collaborative environment, more participants deal with problem-solving. This promotes the sharing of different points of view, intuition and experiences about the same problem.

In collaborative decision making, the definition of decision groups is an activity to be considered. According to Sommers [22], there is a greater sharing of knowledge in decision making when the group is formed by members of different characteristics. Factors such as the diversity of personalities, values, and cognitive abilities lead to a greater use of the information obtained inside the team.

It must also be considered that not every collaborative decision-making process is
beneficial. One of the major challenges of group activities is interpersonal conflicts
[20]. Often discussions involving conflict of opinions, inability to obtain consensus, or
shyness among participants leads to unsuccessful discussions to resolve problems.
Therefore, it is necessary to analyze groups that have greater compatibility of inter-
personal communication and complementation of cognitive and emotional attrib-utes,
thus guaranteeing better performance in the results of the tasks [23].

Considering the concepts and works discussed in this section, this paper propos-es the development of a collaborative environment, capable to promote impacts projection of a complex decision based on interactions between decision-makers. In this environ- ment, decision-makers are encouraged to construct projection models based on their shared knowledge and experiences. Consequently, projected impacts may support new decision-making groups and individual decision-makers in complex decision-making.

3 ImpactMap Approach

The four phases model of emergency management encompasses mitigation, preparedness, response and recovery. Mitigation involves deciding what to do where a risk to the health or safety has been identified. Mitigation is a sustained action to reduce or eliminate risk to people or hazards and their effects. Preparedness is a continuous cycle of planning, organizing, training, evaluating the actions to be applied to solve an emergency decision problem. Response is the management of resources including personnel, equipment, and supplies. The response phase is a reaction to the occurrence of a catastrophic disaster. To finish, recovery involve activities focus on restoring critical functions to stabilize operations. This phase goal is to bring the affected area back to some degree of normality as soon as possible [24]. The approach proposed by the technological tool (ImpactMap) will be able to support the mitigation and preparedness phases.

In each phase of an emergency management cycle, complex decisions can be made.
This research is concerned in supporting decision makers team, to project impacts of
complex decisions in the environment. The tasks involved in the decision-making pro-
cess are not trivial, especially when decisions are complex. This happens because deci-
sion-makers need to consider the lack of clarity in defining the decision's problems and
objectives and analyze the external influences of the environment. Besides, find a way
to work collaboratively in a scenario where specialists need decision’s information de-
tails and orientation about what they must discuss.

Information and communication technologies can assist a group decision-making
process through tools that promote collaboration and interaction among participants.
ImpactMap is a collaborative web-based decision support tool. It has as main objective
to provide a collaborative environment that is able to stimulate the exchange of experi-
ences and knowledge between decision-makers participating in the session. Because of
these interactions, ImpactMap aims to support the mapping of impact projections be-
fore executing an alternative decision.

Kirikihira and Shimada [19] proposed a tool for supporting consensus-building be-
yond a discuss map development, called “Discussion Map with Assistant (DMA)”. This
tool considers in its consensus map two main constructors that are: alternatives and
criteria. Although this proposal supports a decision question discussion as ImpactMap,
it does not promote the complex decision impacts discovery and analyses in a collabo-
rative way.

The main language implemented on ImpactMap’s development was Javascript,
along with base libraries such as JQuery and Bootstrap to create the interface, with
NodeJS and MongoDB to store the maps and send requests to the server. This tool
presents three basic structures: Model structures, Collaborative strategies
(collaborative environment), Projections management (reuse, cloud, and store), and
Descriptions (Fig. 1).

<figure><img src="image_3.png" /><figcaption>Fig. 1. ImpactMap structure.</figcaption></figure>

Table 1 shows the set of functionalities provided by ImpactMap. Each one is asso-
ciated with a structure previously mentioned.

ImpactMap is an environment that provides impact projection maps development in
a collaborative session and stored by different URL's. This tool allows data persistence
and the URL associated with the collaborative projection session is sent as an invitation
to other decision-makers to attend the session. All maps and constructs' descriptions in
ImpactMap are stored in the cloud and can be viewed and edited by those with an access
profile to their decisions. The ImpactMap environment allows knowledge sharing, joint
analysis and impact projections still in the planning phase of the decision-making process.

<table><caption>**Table 1.** ImpactMap functions</caption><thead><tr><td>ImpactMap Structure</td><td>Function</td></tr></thead><tbody><tr><td>Models</td><td>impact model, alternative model, graphics</td></tr><tr><td>Collaborative strategies</td><td>chat, videoconference, collaborative map construction, shared actions in real time by socket models</td></tr><tr><td>Description</td><td>Descriptions, export description as PDF, invite to collaborate, export history chat, tutorial screen</td></tr><tr><td>Projection management</td><td>cloud environment</td></tr></tbody></table>

The impact and alternative models are the core maps of ImpactMap. This tool is based on the development of impact maps that are like mental maps [25]. The use of mental maps is constantly diffused as a solution for the resolution of complex decisions, since it is seen as a cognitive facilitator of decision-makers. The main difference between common mental models and the models proposed by ImpactMap is the definition of its constructs and the hierarchy of these elements for the composition of impacts projection. The maps proposed by ImpactMap allows the conduction of an orientation process to perform impact projections.

ImpactMap is based on the theory proposed by França [13] that deals with the evolution of [3] and [12]. In this theory, a **decision** is made up of one or more scenarios. **Scenarios** aim to represent the possible effects of variables that affect the decision and to classify the areas of action of the decision in a domain. They are an integral part of the hierarchy to build the situation analysis. Each scenario can be made up of one or more **alternatives**, and they are associated with one or more impacts. At the end, the **impact** represents the consequences of an alternative, since it has not yet been implemented. To develop the projection maps according to the predicted structure, the tool presents a tutorial that teaches the structure to design it (Fig. 2).

Fig. 2 presents ImpactMap tutorial highlighting the existing functionalities and a glossary with the constructs of the impact and alternative model also provided by the tool. Fig. 3 presents a draft hierarchy of an impact and alternative models.

The alternative model view is activated by selecting an alternative previously presented in the impact model. In the on-screen expansion of the alternative model, the alternative characterization elements created and evaluated by the decision-maker group are presented. The values imputed in each characterizer can be analyzed in a radar chart.

The graphs created for each alternative allow the analysis of the decision not only by the projection group, but also by new groups or individual decision-makers who will reuse this information. To support the impact projections, some collaboration resources were introduced in ImpactMap, like communication (via videoconferences and chat – Fig. 4), cooperation and coordination strategies.
<figure><img src="image_4.png" /><figcaption>Fig. 2. Tutorial ImpactMap.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 3. Impact Model and Alternative Model Vision.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 4. Communication resource.</figcaption></figure>
All actions taken on impact and alternative models are monitored in real time by all components of the projection section. When an element is updated or created, these actions appear to the other participants of the session, maintaining the new changes throughout the entire group. In next section, we present our approach evaluation, highlighting the participating groups, the domain applied and the influence of this research in an emergency management.

# 4 Research Evaluation

ImpactMap was evaluated having in mind this research main goal stated as an approach to support decision makers project complex decision impacts in the environment, especially considering emergency domains. In order to provide a simple and easy tool that encourages collaboration between decision-makers, this study was divided into two phases: a pilot experiment (an exploratory study) and a remote experiment (Fig. 5).

<figure><img src="image_7.png" /><figcaption>**Fig. 5.** Research evaluation phases.</figcaption></figure>

Each phase was planned considering the simulation of an emergency domain, considering the mitigation phase of the emergency management cycle. To conduct this evaluation, it was necessary to form decision-makers groups. The following Figure (Fig. 6) presents the process that oriented this evaluation. This process was used in both phases.

However, Phase 1 was conducted considering groups that interacted in person (but using ImpactMap environment – each participant in one computer, but at the same virtual session), while phase 2 considered remote interaction with group members geographically dispersed.

Phase 1 had three groups with two, three and four decision-makers, respectively. All of them are decision maker and specialist in software usability, trained as emergency responder in their enterprise. Phase 2 had two groups with the same characteristics as phase 1. All groups projected impacts using ImpactMap tool simulating: (i) Phase 1: the building where they work was invaded by an immense shooting and they must save their live and as many other lives as possible. (ii) Phase 2: the building where they work caught fire and they must save his life and as many other lives as possible.

All groups attended the tutorial session, where the tool and features were presented. After it, all groups had a period of time to finish their impacts projections. During the
evaluation, all groups were observed, and these data were collected by notes. In Phase 1 the observation occurred in person, while in Phase 2, occurred inside the ImpactMap session (virtual session).

<figure><img src="image_8.png" /><figcaption>Fig. 6. Process evaluation.</figcaption></figure>

Another way to investigate the applicability of this technological environment to project impact of complex decisions attending the mitigation and preparedness phases of emergency management cycle were the answers collected beyond a questionnaire. These answers presented the specialists view about the real applicability of this proposal simulated in this study. This instrument was prepared considering two main aspects: (a) capability to project decisions impacts in a collaborative way in the previous phases of decision making process and emergency management cycle, and (b) tool usability (utility, efficiency, effectiveness, learnability, and satisfaction) to verify if the technological problems could be the reasons to evaluate the tool as unsuitable to discuss a decision problem and project its impacts in a collaborative way [26]. The questionnaire was applied in both phases, however, Phase 2 had specific questions related to the communicative functions of ImpactMap. The next section presents the results achieved by both phases of experiments and discuss it considering the main goals of this research.

# 5 Results Presentation and Discussion

All groups projected impacts of the simulation proposed (i and ii). They developed impact and alternative models, besides each elements' description.
Fig. 7 shows the impact map produced by group III during the experiment. All groups produced a similar structure. In Phase 1, we have three groups. Table 2, 3 and 4 present a summary of these groups' experiences and report some technical problem faced during the experiment.

<figure><img src="image_9.png" /><figcaption>**Fig. 7.** Impact projection using Impact and alternative model. Example produced by Group III - Phase 2.</figcaption></figure>

<table><caption>**Table 2.** Group I participation – phase 1</caption><thead><tr><td>Aspects</td><td>Description</td></tr></thead><tbody><tr><td>Participants number</td><td>2</td></tr><tr><td>Duration</td><td>13 minutes</td></tr><tr><td>Technical problems faced</td><td>After deleting a node, the model presented problems in the hierarchy.</td></tr></tbody></table>

<table><caption>**Table 3.** Group II participation – phase 1</caption><thead><tr><td>Aspects</td><td>Description</td></tr></thead><tbody><tr><td>Participants number</td><td>3</td></tr><tr><td>Duration</td><td>15 minutes</td></tr><tr><td>Technical problems faced</td><td>Alternative characterizers were not intuitive.</td></tr></tbody></table>

<table><caption>**Table 4.** Group III participation – phase 1</caption><thead><tr><td>Aspects</td><td>Description</td></tr></thead><tbody><tr><td>Participants number</td><td>4</td></tr><tr><td>Duration</td><td>45 minutes</td></tr><tr><td>Technical problems faced</td><td>Node description doesn't allow editing<br>Absence of a function to undo actions.<br>Removal of compound alternative to return to common alternative unstable.</td></tr></tbody></table>
In phase 2 the experiment was conducted considering a new version of ImpactMap, with the technological problems faced in phase 1 solved. The solution's evolution is relevant for this study because we can isolate the technical influences and analyze the main strategy to support decision making and emergency management.

Phase 2 main goal focus on the investigation of the same aspects discussed in phase 1, but now the participants considered another simulation domain and analyzed the communication resources made available by ImpactMap.

<table><caption>**Table 5.** Group I participation – phase 2</caption><thead><tr><td>Aspects</td><td>Description</td></tr></thead><tbody><tr><td>Participants number</td><td>4</td></tr><tr><td>Duration</td><td>17 minutes</td></tr><tr><td>Technical problems faced</td><td>Video conferencing functionality does not work for those who need to use the microphone without a camera.</td></tr></tbody></table>

<table><caption>**Table 6.** Group II participation – phase 2</caption><thead><tr><td>Aspects</td><td>Description</td></tr></thead><tbody><tr><td>Participants number</td><td>2</td></tr><tr><td>Duration</td><td>24 minutes</td></tr><tr><td>Technical problems faced</td><td>Alert messages appear to all users in the session.</td></tr></tbody></table>

Tables 5 and 6 summarizes the participation in Phase 2. Some problems were faced but the collaborative environment (ImpactMap) was enough to investigate the potentials of the approach proposed (as will be discussed below considering the usability aspects - utility, efficiency, effectiveness, learnability, and satisfaction). The groups activity observations and the answers collected by the questionnaire led us to some conclusions.

In both phases, groups with more participants developed impact projections in more detailed but required more time to do so. As was observed, larger groups can promote conflicts in peer interactions.

ImpactMap and the approach related can provide detailed information, however it could be a problem if an emergency demands a short period of time as few minutes or seconds for the specialist project and analyze the decisions impacts in the early stages of an emergency management cycle. In this case, this approach is not recommended. But it is able to support decisions team in their impact's projection and decision analysis in emergency situations considering geographically disperse specialists.

During phase 1, all participants informed that it is possible to play more than one collaboration pattern using ImpactMap to project impacts. Fig. 8 shows the numbers considering: (a) Member which sought consensus in conflict situations; (b) Member more focused on organization and description of model elements; (c) Member which led the discussions between the group; and (d) Which proposed creative ideas for solving situations. Phase 2 showed a similar result in all the usability aspects.
<figure><img src="image_10.png" /><figcaption>Fig. 8. Collaboration pattern applied during the impact projection section.</figcaption></figure>

For more than 70% of participants in phase 1, ImpactMap supported the impacts projection and its discussion (Fig. 9), and more than 65% were satisfied with the impact projection results. These results were the same in Phase 2 and shows that ImpactMap is useful. So, **utility** and **satisfaction** aspect are observed in this tool.

<figure><img src="image_11.png" /><figcaption>Fig. 9. Utility aspect result – Phase 1.</figcaption></figure>

The **Efficiency** aspect was inconclusive. There are different kind of variables that influence this aspect. The complexity of the decision and the numbers of the participant in each group affect the time of impact projections.

<figure><img src="image_12.png" /><figcaption>Fig. 10. Effectiveness results – Phase 1.</figcaption></figure>
For all the experiment participants, ImpactMap supported decision-makers, showing the concepts that must be analyzed in an impact projection. For that, ImpactMap strongly shows **effectiveness** in its functions (Fig. 10).

**Learnability** was a usability aspect that generated controversy. Fig. 11 shows that 11,1% answered that the functionalities are not intuitive, while 33,3% defined as neutral, 33,3% intuitive, and 22,2% strongly intuitive. This difference occurred because some participants tried to evaluate functions that are useful in another step of the decision-making process, like the radar graphics. These graphics bring a different and summarized visualization to support new decision-makers with details about a complex decision already analyzed. The functions related to support impact map construction and description were evaluated as easy to learn.

The tool's functionalities were very easy and intuitive to use.

9 answers

<figure><img src="image_13.png" /><figcaption>Fig. 11. Learning results – Phase 1.</figcaption></figure>

These results show that ImpactMap is an environment prepared to support decision-makers and specialists analyze a decision problem and project its impact in a collaborative way. This tool allows specialists work together even they are not present in the same place, i.e. dispersed geographically.

As were discussed in this paper, the phases mitigation and preparedness can be supported by ImpactMap. Mitigation phases is concerned deciding what to do in a problem identify. So, the approach delivered by ImpactMap tool is capable to orient specialists analyze the problem and project the impacts before the solution proposed by the team be implemented. Preparedness is worried to plan, organize, train and evaluate the actions selected to solve an emergency problem. Know the actions' (alternative construct) impacts will give more resources to decision-makers team to analyze the influences of this choice in the environment.

# 6 Conclusion

This paper presented the ImpactMap approach. It provided an environment to support decision-maker projecting impacts of complex decision in a collaborative way. Another important quality of ImpactMap is its capability to orient decision-makers in how to externalize and share tacit knowledge.
The main goal of this paper was to provide an approach through the ImpactMap tool able to support decision makers project complex decision impacts in the envi-ronment, especially considering emergency domains. This goal will attend the phases mitigation and preparedness of emergency management cycle. This tool provided an environment able to share ideas and experiences, in order to support decision-makers groups struc-ture their knowledge into impacts map. This research presented an envi-ronment that encourage collaboration between decision-makers disperse geograph-ically. This re-search argued that if a collaborative environment to support impact projection of com-plex decision is available, decision-makers will be able to project impact maps, making their thoughts and experiences externalized in a structured way.

This research was evaluated in a study divided into two phases, both using Im-pact-Map environment. The first one was conducted in person and the second was conducted remotely. This evaluation showed that ImpactMap is a technological resource able to support decision makers analyze a complex decision and project its impacts. During this evaluation some technical limitations were faced (concerned to the tool develop-ment), but all of them did not influenced the main result of this re-search. However, the time available to analyze and discuss a decision question is important to decide for the use of ImpactMap.

The next steps include: (a) run more tests with different group sizes; (b) Evolve the tool considering the aspects observed during the evaluation; (c) Make the tool avail-able in mobiles devices; (d) Run a study to investigate the influence of ImpactMap in the whole emergency management cycle; and (e) Conduct a study about the influ-ence of decision-makers personal characteristics inside a group responsible to project-ing impacts of complex decisions.

## Acknowledgements

This research was supported by National Council for Scientific and Technological De-velopment (CNPQ) in Brazil. Marcos R. S. Borges was partially supported by the Na-tional Council for Scientific and Technological Development (CNPQ) under grant # 308149/2015-7, and by Rio de Janeiro Research Support Foundation (FAPERJ) un-der grant # E-26/202.876/2018.

## References

1. Haddow, G. D., Bullock, J. A.: Introduction to emergency management. Newton, MA: But-terworthHeinemann (2003).

2. Nimpuno, K., Hilman, R., United Nations Centre for Human Settlements., & Technische Hogeschool Delft: Disaster management glossary. Nairobi, Kenya: United Nations Centre for Human Settlements (Habitat), (1998).

3. França, J.B.S., Neiva, F.W., Dias, A.F.S., Borges, M.R.S.: Towards Projected Impacts on Emergency Domains Through a Conceptual Framework. In: International Conference on Information Systems for Crisis Response and Management, Albi, France. XIV International Conference on Information Systems for Crisis Response and Management, (2017). http://idl.iscram.org/files/julianabsfranca/2017/1467_JulianaB.S.Franca_etal2017.pdf
4. Lederman E., Chimenz L.: Supporting Decision Making in Disasters: The DiMas Tool. In: Murayama Y., Velev D., Zlateva P. (eds) Information Technology in Disaster Risk Reduction. ITDRR 2017. IFIP Advances in Information and Communication Technology, vol 516. Springer, Cham (2019).

5. Klein, G.A., Orasanu, J., Caldewood, R., Zsambok, E. (Eds.): Decision Making in action: Models and Methods. New Jersey: Ablex Publishing Corporation Norwood, P. 138–147, (1993).

6. Klein, G. A., Calderwood, R.: Decision models: Some lessons from the field. IEEE Transactions on systems, man, and cybernetics, 21, 1018-1026, (1991).

7. Orasanu, J., Connolly, T.: The Reinvention of Decision Making. In: G.A. Klein, J. Orasanu, R. Caldwood, E. Zsambok, (Eds). Decision Making in action: Models and Methods. New Jersey: Ablex Publishing Corporation Norwood, p. 3-20, (1993).

8. Fuks, H., Raposo, A., Gerosa, M. A., Pimentel, M., Lucena, C. J. P.: The 3C Collaboration Model. The Encyclopedia of E-Collaboration, Ned Kock (org), ISBN 978-1-59904-000-4, pp. 637–644, (2007).

9. Qudrat-Ullah, H.: Better Decision Making in Complex, Dynamic Tasks. Training with Human-Facilitated. Interative Learning Environments. School of Administrative Studies. York University, Toronto, Canada (2015).

10. Yang, L., Xu, Y.: Decision making with uncertainty information based on lattice-valued fuzzy concept lattice. Journal of Universal Computer Science, vol. 16, no. 1, pp. 159–177, (2010).

11. Doyle, J. K., Radzicki, M.J., Trees, W.S.: Measuring Change in Mental Models of Complex Dynamic Systems. In: Qudrat-Ullah, H., Spector J.M., Davidsen, P.I., Eds. Complex Decision Making. Theory and Practice. Cambridge/Massachusetts, pp. 269 – 294 (2008).

12. França, J.B.S., Neiva, F.W., Dias, A.F.S., Borges, M.R.S.: Toward Impact Projection Characterization of Complex Decisions. In: IEEE International Conference on Systems, Man, and Cybernetics, Banff. IEEE International Conference on Systems, Man, and Cybernetics, V1, (2017), https://ieeexplore.ieee.org/document/8123003/.

13. França, J.B.S.: EPIDRÓ: An Approach to Guide a Collaborative Impacts Projection of Complex Decisions. Post-Graduation Program in informatics, Mathematics Institute, Tércio Pacitti Institute of applications and Computational Research, Federal University of Rio de Janeiro. Rio de Janeiro, Brazil, (2018). <in portuguese>

14. Padilha, R. P., Borges, R.M.S., Gomes, J.O., Canós, J.H.: The design of collaboration support between command and operation teams during emergency response. The 2010 14th International Conference on Computer Supported Cooperative Work in Design. In: 2010 14th International conference on computer supported cooperative work in design (CSCWD). Shanghai, China: IEEE, abr. (2010). http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5471873>

15. Hammond, J. S., Keeney, R. L., Raiffa, H.: Smart Choices: A practical guide to making better life decisions. New York: Broadway Books, (2002).

16. Shattuck, L. G., Miller, N.L.: Extending Naturalistic Decision Making to Complex Organizations: A Dynamic Model of situated cognition. Organization Studies, 27, 989-1009, (2006).

17. Aldea, A., Bañares-Alcántara, R., Skrzypczak, S.: Managing Information to Support the Decision Making Process. Journal of Information & Knowledge Management 11 (3) (2012).

18. Ayyub, B. M.: Risk Analysis in Engineering and Economics. Second Edition, Chapman & Hall/CRC, ISBN 1-58488-395-2, (2014).

19. Kirikihira, R., Shimada, K.: Discussion Map with an Assistant Function for Decision-Making: A Tool for Supporting Consensus-Building. Collaboration Technologies and Social
Computing. 10th International Conference, CollabTech 2018, Costa de Caparica, Portugal,
September 5-7, (2018).

20. De Dreu, C. K. W., Weingart, L. R.: Task versus relationship conflict, team performance, and team member satisfaction: A metaanalysis. Research of Applied Psychology, 88(4), pp. 741-749, (2003).

21. Dillenbourg, P., Baker, M., Blaye, A., O'Malley, C.: The evolution of research on collaborative learning. In E. Spada & P. Reiman, Eds. Learning in Humans and Machine: Towards an interdisciplinary learning science, pp. 189-211. Oxford: Elsevier, (1996).

22. Sommers, S. R.: On racial diversity and group decision making: Identifying multiple effects of racial composition on jury deliberations, Journal of Personality and Social Psychology, 90(4), pp. 597-612, (2006). http://dx.doi.org/10.1037/0022-3514.90.4.597

23. Van Knippenberg, D., Schippers, M.C.: Work group diversity. Annual Review of Psychology, (2007). https://doi.org/10.1146/annurev.psych.58.110405.085546.

24. Sylves, R.: Disaster Policy and Politics: Emergency Management and Homeland Security (Washington, DC: CQ Press, 2008).

25. Qudrat-Ullah, H., Spector, J.M., Davidsen, P.I. (Eds.): Complex decision making. New York, NY: Springer, (2007).

26. Jeff, R.: Handbook of usability testing. 2a (ed.), (2008).