# A multi-criteria framework for critical infrastructure systems resilience

Zhuyu Yang, Bruno Barroca, Katia Laffréchine, Alexandre Weppe, Aurélia Bony-Dandrieux, Nicolas Daclin

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Zhuyu Yang, Bruno Barroca, Katia Laffréchine, Alexandre Weppe, Aurélia Bony-Dandrieux, et al. A multi-criteria framework for critical infrastructure systems resilience. International journal of critical infrastructure protection, 2023, 42, pp.100616. 10.1016/j.ijcip.2023.100616. hal-04135558

HAL Id: hal-04135558

https://imt-mines-ales.hal.science/hal-04135558

Submitted on 21 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

# A multi-criteria framework for critical infrastructure systems resilience

<figure><img src="image_4.png" /></figure>

Check for
updates

Zhuyu Yang<sup>a,b,*</sup>, Bruno Barroca<sup>a</sup>, Katia Laffréchine<sup>a</sup>, Alexandre Weppe<sup>c</sup>, Aurélia Bony-Dandrieux<sup>c</sup>, Nicolas Daclin<sup>c</sup>

## ARTICLE INFO

### Keywords:

Critical infrastructure systems

Multi-criteria

Resilience

Disaster management

Resilience assessment

## ABSTRACT

Critical infrastructure systems (CISs) play an essential role in modern society, as they are important for maintaining critical social functions, economic organisation, and national defence. Recently, CISs resilience has gained popularity in both academic and policy filed facing increased natural or technological disasters. Resilience assessments have become convenient and common tools for disaster management, as assessment results provide useful information to CIS managers. However, CISs resilience assessment is facing challenges of being practical to use in operational risk management.

Although there are many existing assessments for CISs resilience, some shortcomings relating to assessment criteria, which cannot turn resilience useful in practical operation, are frequent in their assessment process. Existing assessments are based on different definitions, which makes criteria generalization difficult. Besides, these assessments are not comprehensive enough. Especially, few assessments address both the cost, effectiveness, and safety of optimisation actions. Moreover, most of the suggested criteria are not specific enough for being used for practical CISs risk management in real cases.

This article develops therefore a multi-criteria framework (MCF) for CISs resilience, consisting of general criteria and a guide for defining specific sub-criteria. In this MCF, the side effects, cascading effects and cost-benefit in resilience scenarios are considered indispensable for CISs resilience assessment. The paper also presents an example of the application of the developed guide through two detailed scenarios, one on a single infrastructural system affected by a natural disaster, and the other addressing the interdependence of this infrastructural system and an urban healthcare system. The designed MCF contributes to the operationalisation and comprehensiveness of CISs resilience assessments.

## 1. Introduction

Critical Infrastructures (CIs), which provide vital services for people and communities, are those physical and information technology facilities, networks, services, and assets that, if disrupted or destroyed, would have a serious impact on the health, safety, security, or economic well-being of citizens or the effective functioning of governments [1]. The United Nations Office for Disaster Risk Reduction (UNDRR) [2] emphasises the importance of investment in resilient infrastructure for a

better future in the context of increasing extreme weather events and threats. Modern societies are becoming increasingly dependant on interconnected technological systems that could be called critical infrastructure systems (CISs) [3]. Resilience in the engineering domain includes technical systems designed by engineers that interact with humans and technology [4]. A CIS consists of human or non-human, physical or mental components involving its management [5], as it is formed when, engineered systems and socio-ecological context are integrated [6]. These CISs play a fundamental role in delivering

**Abbreviations:** CIs, Critical Infrastructures; CIS(s), Critical Infrastructure System(s); UNDRR, United Nations Office for Disaster Risk Reduction; MCF, multi-criteria framework; C&I, Criteria & Indicators; CBE, consequence-based engineering; PrES, pre-event stage; DES, during event stage; PoES, post-event stage; NEPS, next event preparation stage; BB model, "Behind the Barriers" model; NRR system, Nantes Ring Road system; EMS system, Emergency Medical Service system; F-NRR-PAR, flood - Nantes Ring Road - planning alternative roads; DNRR-EMS-UAR, dysfunction of Nantes Ring Road - Emergency Medical Service - using alternative roads; MCDM, Multi-Criteria Decision Making; MCDA, Multi-Criteria Decision Analysis.

<footnote><sup>a</sup> Lab'urba, Université Gustave Eiffel, Champs-sur-Marne 77420, France</footnote>

<footnote><sup>b</sup> LATTS, UMR CNRS 8134 Université Gustave Eiffel/Ecole des Ponts ParisTech, Marne la Vallee, France</footnote>

<footnote><sup>c</sup> Laboratoire des Sciences des Risques (LSR), IMT mines Ales, Alès, France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: zhuyu.yang@univ-eiffel.fr (Z. Yang).</footnote>
commodities that are essential to various functions in urban systems [7]. However, the difficulty in commonly defining “resilience” in the field of infrastructure raises the challenges of finding an agreement to assess CISs resilience. Resilience, introduced by Holling in 1973 [8], originally means a persistent ability to absorb change and disturbance and still maintain the same state variables. CISs resilience today still does not have a broadly accepted definition, but it is highly related to engineering or socio-technical science.

Resilience assessments have become key aspects of disasters management. According to Resilience Alliance [9], an efficient resilience assessment could integrate a set of key concepts and provide alternative ways of thinking about and practicing resource management. One of the important objectives of resilience assessment is to support the decision-making process, which in practical cases requires a consideration of various crucial criteria. "Resilience" is an inherent but abstract property of a system [10]. In an assessment process, criteria are characters or signs that can be used to distinguish an abstract property (such as resilience), in order to make a judgement of appreciation.

Although the approaches built for CISs resilience assessment are diverse and multidisciplinary [11–16], most existing assessments of CISs resilience have several common limitations. Firstly, in modern society, a catastrophic event often causes cascading effects while optimisation measures could lead to side effects. The current lack of thinking about spatial and temporal interactions across urban systems prevents designing beneficial actions and suppressing dangerous ones. In addition, the vagueness existing recently in CISs resilience definition makes it difficult to develop generalizable indicators or criteria for resilience assessment. Besides, each CISs disaster event, in real cases, has uniqueness, but few existing criteria are specific enough to correspond to concrete situations aimed by different CISs stakeholders. It results that most resilience assessments for CISs cannot make the ‘resilience’ concept usefulness at the operational level of risk management.

The operationalisation of CISs resilience could be through a framework of multi-criteria that correspond to the needs of managers in practical operation. Operationalization refers to making a theory have practical and operational significance, transforming a theory into an object of practical value, regarding in the broader sense of ‘using’ a theory for different purposes [17,18]. As a tool for conflict management, multi-criteria evaluation is a very efficient tool to implement a multi/inter-disciplinary approach [19]. Scientists have demonstrated the usefulness of multi-criteria assessments in many sustainability options and management problems [20–23]. Therefore, the aim of this article is, based on the definitions of CISs resilience that are potentially useful for practical operation, to develop a multi-criteria framework (MCF), which contributes to defining both general and specific criteria. This objective MCF aims additionally at systems’ interdependencies, and the cost, effectiveness, and safety of optimisation actions.

One of the keys of operationalisation refers to the practice application by local managers. Therefore, this study will provide an application example with the participation of an infrastructure management organisation-Direction interdépartementale des routes Ouest (DIRO) in charge of the Ring Road of Nantes City in France. Considering continuous events following a single hazard event, this example consists of two detailed scenarios, one involving Nantes Ring Road (NRR) system affected by flood events and the other involving Nantes Emergency Service (EMS) system affected by the dysfunction of NRR.

The design of the objective MCF presents in Section 3, after an introduction of the objectives, methods and structure of this study in Section 2, which describes: why a MCF is needed to assess CISs resilience; which elements a MCF should contain; and the methods for designing a MCF. Section 4, on a sound theoretical and material basis, describes the process to define criteria, and to develop a guide for defining specific sub-criteria. Section 4 applies this guide to the mentioned example to explain its practical use for defining specific sub-criteria.

<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Hierarchical structure in MCA approaches for C&I-based assessment, adjusted by authors, sources: Van Bueren and Blom, Prabhu et al., and Mendoza and Prabhu [27–29].</figcaption></figure>

## 2. Multi-Criteria framework: why, what and how

### 2.1. Why is a multi-criteria needed?

Some review works on CISs resilience assessment [11–16] show the different criteria, dimensions, and aspects of focus or assessment methods currently used in scientific studies. However, in the current studies for CIS resilience, there are several common phenomena relating to assessment criteria after a pre-analysis.

Firstly, many resilience assessments do not discuss “criteria”, even though their focused dimensions or perspectives, such as capacities, abilities and characteristics, could be further developed and translated to ‘criteria’. During assessment processes, a target criterion is the desired direction of selected objective information, i.e. an indicator that is used to monitor the evolution of a specific aspect of the issue dealt with. Without assessment criteria, practical operators and managers have no envisaged positive outcomes of assessment results. Existing CISs resilience assessment studies usually focus on one single or a couple of aspect of CISs resilience, such as performance, function and vulnerability, which are the most frequently presented [12]. However, most assessments focus mainly on the functionality of infrastructure from a technical perspective and do not consider disaster management from a socio-organisational perspective, resulting in inadequate results to help overall urban resilience. For instance, only from an efficacy perspective, building a new city in an area without flood risks is one of the most effective ways to prevent flooding. Obviously, from a social, economic, and ecological perspective, it is not a sustainable and cost-effective solution. Being in a complex modern society, CISs managers should keep holistic thinking that balances the various advantages and disadvantages. In particular, much of the research focuses on the abstract capabilities associated with resilience but overlooks the fact that it is vital for every CISs manager to discuss effective actions that can be implemented without excessive cost or negative impact. For example, Øien and Bodsberg [24] highlight four key attributes to assessing the resilience of smart critical infrastructures (“Understand risks”, “Anticipate/prepare”, “Absorb/withstand”, “Respond/recover” and “Adapt/learn”), but this study did not address concrete operational actions. Trucco et al. [25] pre-defined five cycles, four dimensions, four capacities, and two organisational levels for resilience assessments. This study [25] takes into account the efforts of implementing actions, but not considers the negative side and cascading effects caused after a single scenario. The multi-criteria analysis involving actions investigation is therefore necessary for making the assessment applicable in practice by managers, and make it possible to consider different alternatives and the multidimensionality of the real world, to address different realities in the infrastructure assessment [20].
<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Position of the target criteria in the indicator-based resilience assessment approach, and in the hierarchical structure in MCA, adjusted by authors, source: Yang et al. [12].</figcaption></figure>

Overall, this study aims at a multi-criteria analysis for CISs resilience. One of the prospects of this study is contributing to CISs resilience assessments that could be useful for practical operations. Assessments by Criteria & Indicators (C&I) could rely on a conceptual hierarchical structure, which is originally developed for sustainable forest management (see Fig. 1) [26-29] In a hierarchical structure, a higher-level “goal” is divided into aspects or themes, which are in turn divided into criteria each with a number of indicators [26]. Van Bueren and Blom [27] believe that “the hierarchical framework facilitates the development of consistent and coherent standards that described what should be accomplished and enables an assessment if, or to what extent, accomplishment is realised”. The selection criteria adapted to the specific needs of stakeholders play a crucial role in the operational process. An investigation for multi-criteria helps makes the CISs resilience assessment by C&I complete. Assessments consisting of criteria and indicators provide a commonly agreed framework for articulating and defining expectations (including targets), developing management methods, best practices and performance elements, and are then used in monitoring and evaluating progress towards those expectations and targets [30].

## 2.2. What the objective MCF is and how to design it?

To understand what a criterion is, it is important to understand what C&I-based assessments is. Indicators are frequently used for CISs resilience assessments, but 'criteria' and 'indicators' are usually misused [12]. Acquiring knowledge like the resilience of CISs, which can be used to make decisions, set policies, requires information systems to transform data into information [31]. The methods for transforming from data, to information and knowledge are various, such as estimation, evaluation, assessment, etc. amongst them, assessment is frequently used and defined as a process by which information is obtained relative to some known objective or goal. In addition, during the process, the chosen objective information intended to observe the evolution (and/or status) of targeted knowledge is called "indicator". Thus, the process of indicator-based assessment, targeting an objective or a goal, has two phases [12] (see, Fig. 2):

* Goal assessment: a process in which goal values are obtained by usable indicators;

* Indicator assessment: a process in which indicator values are obtained by reliable data.

This study, which aims at 'criteria' is situated in the first phase of Fig 2. In this phase, the "goal" in this study is resilience, while the "aspects" refer to the principles that are essential priorities [29] to overall CIs resilience. An indicator is a sign or signal that shows something exists or is true, or that makes something clear. An indicator is associated with a criterion, while a criterion is associated with a number of indicators. Criteria are characters or signs, which make it possible to distinguish a thing, or a concept, to make a judgement of appreciation. Criteria could be considered as the points to which the objective information provided by indicators can be integrated and where an interpretable assessment crystallises [32]. For a C&I-based assessment, aspects are essential and

<figure><img src="image_7.png" /><figcaption>**Fig. 3.** Methods, objectives, and structure, created by authors.</figcaption></figure>

general, while indicators are specific to practical situations. Criteria play a role considered a bridge linking general aspects and specific indicators. In the field of management, there is an ongoing debate about whether criteria should be universally applicable or specific. Nevertheless, if a criteria framework is desired for operational management, it should allow it to be defined and modified by managers and decision-making groups in specific situations. This study therefore believes the necessary to design a multi-critical framework (MCF) consisting of various criteria and this MCF should have both genericity and specificity.

Developing generalizable criteria for resilience assessment is a current challenge to turn resilience into operational tools, because the existing definitions of resilience are multitude and different [33,34]. Thus, the first crucial contents for the objective MCF are general criteria, which could be defined based on some selected significant aspects.

Eurostat [35] emphasises that the important aspect allows the setting up of the definition of the desired evolution of indicators, i.e. the criteria of each indicator. The identification of an important "aspect" is based on an analysis of the assessment target and goal. Maggino [26] believes that an investigation of studied phenomena allows for defining the phenomenon, its domains, and general aspects, even though it is a complex stage requiring the identification and definition of theoretical constructs. This means that an analysis of the definition and phenomena of CISs resilience should be performed firstly, based on which, secondly, an investigation of the essential priorities helps define important aspects. The designing of the objective MCF relies on the results of preparation works involving these two steps (see Fig. 3). Additionally, to overcome the existing limitations mentioned above that produce the difficulty of resilience operationalisation, the used definitions and phenomena of CISs resilience should address a socio-organisational perspective. The general criteria in the objective MCF should be unable to adapt to all
<figure><img src="image_8.png" /><figcaption>**Fig. 4.** A conceptual scenario of resilience, adjusted by authors, source: Yang et al. [41].</figcaption></figure>

types of CISs.

Moreover, the objective MCF should address sub-level specific criteria under each general criterion. Many existing theories or models for CISs resilience assessment only predefine general perspectives or dimensions for resilience assessment [11]. Although they are valuable, this study believes that, for making resilience theory become practice, it is necessary to consider the uniqueness of each specific situation. To address resilience operationalisation, governments and practitioners need descriptive support and guidance about operational process [36, 37]. Rather than predefining criteria for all potential resilience scenarios of CISs, a practical MCF should provide a guide for consulting potential users to define specific sub-criteria, because each user has different concrete situations. Just as teaching a man to fish, rather than simply giving him fish. A guide for defining specific sub-criteria is thus desired for the objective MCF. This guide needs to be detailed and operational, equally combined with available theories for CISs resilience operationalisation.

Overall, in the objective MCF, the criteria linking directly to important ‘aspects’ are general criteria, while the sub-criteria associating potential indicators are specific. The objectives, structure and prospect of this study are shown in Fig. 3. Indeed, the fact that the definition of CISs has no orthodoxy in its conceptualization and application would make the MCF design difficult. However, this study tries as far as possible to find solutions based on the current highly accepted studies of CISs resilience.

## 3. Designing MCF

This section presents the establishment of a MCF for CISs resilience, and is divided into three sub-sections (see Fig. 3):

1. Preparation works that address an analysis of the definition, phenomena and important aspects of CISs resilience.

2. The definition of general criteria that, in this study, are considered applicable to all CISs

3. The development of a guide that helps define specific sub-criteria in terms of each individual case. The definition of the specific sub-criteria requires a deep knowledge of each studied case.

### 3.1. Preparation works

#### 3.1.1. Definition and phenomena of CISs resilience

EXCIMAP [38] defines resilience as the ability of a system, community or society exposed to hazards to resist, absorb, accommodate, adapt to, transform and recover from the effects of a hazard in a timely and efficient manner. Vugrin et al. [39] and Vugrin et al. [40] argue that a system's resilience measurement involves two components: systemic impact, which is changed performance following a disruptive event; and total recovery effort, which refers to the number of resources expended following the disruption. Based on these opinions, Yang et al. [41]

present a conceptual scenario of resilience (see. Fig. 4), which illustrates a resilience-based phenomenon that “a hazard affects a system”, and highlights two conditions under which this phenomenon established:

1. the hazard causes negative consequences in the affected system and its components;

2. the affected system applies the actions to resist, absorb, accommodate, adapt to, transform recover and learn from hazards.

These descriptions for resilience are potentially significant for resilience operationalisation. The "consequence" or "impact" can be linked in detail and specifically to the affected internal components of the infrastructure system in the event of a disaster. The concretization of "action" and "effort" can be further interpreted in terms of costs, benefits, damages, side effects, etc., which have to be taken into account in the practical management of the operation for enhancing effective actions and suppressing dangerous ones.

The term "consequence" could be both positive and negative. Since the definition of resilience emphasises the ability to cope with the negative effects of a hazard, all "consequence" mentioned in this study is negative. Etymologically, resilience comes from the Latin, resilio, resilire, which means a return and the ability to resume. Therefore, this study highlights that only the CISs damaged by hazards and not in a normal state could be the objects of resilience studies. Studies that only focus on the ability to avoid consequences from hazards without the consideration of recovery and adaptation, address resistance, vulnerability, and robustness, not the concept of resilience. Resilient CISs accept and adapt to hazards. The adaptive capacity of CISs resilience, one of the key factors of resilience, could be presented through the level of consequence and the rate of deterioration. In addition, the function of CISs is supported directly or directly by all internal components [5]. Shavelson [42] argue that a complete assessment measures the different components of a system. An assessment that regards only the main function or performance is not comprehensive for decision-makers. For example, the physical injury caused by a terrorist attack does not directly reduce the transport service of a rail infrastructure system, but it is a critical criterion for rail managers. However, it is recognised that not every component will have significant effects. In the guide for sub-criteria definition, the steps of how to identify meaningful negative consequences or damage by analysing important components will be presented.

A resilient CIS should have different capabilities and involve actions to improve its capabilities [43], including adaptive capacity. More and more researchers argue that the actions for improving resilience need to be designed for responding to short-term challenges, and meanwhile to long-term strategies [44,45]. Thus, the implementing action mentioned in this study refers to all possible operations could be taken for optimising the resilience of CIs, like programmes, strategies, projects, measures or practices for both temporary (at short-term) and permanent preventive (for long-term) management. Irrespective of the moment when the action is implemented, the effect of the action can be reflected both in short term (one single scenario) and in long term (on future scenarios). Many studies [46-48] believe that a resilient system should have the ability of learning and improvement from experience. Implementing actions with long-term positive effects makes CISs more resilient in the face of new shocks than they were before. In addition, due to the interaction of internal components, the actions of one CIS potentially bring unexpected side effects on itself [14,49-51]. In medicine, a side effect is commonly described as unintended adverse effects. In this study, the side effects described in particular the unintended damage caused to infrastructures by the actions implemented to increase its resilience to disasters.

More and more CISs resilience studies discuss the connections and interdependencies, which are primarily between CISs [52-55]. As CISs are not isolated and may be physically, geographically, cyber and logically dependent and interdependent, every single CIS can be affected by
<figure><img src="image_9.png" /><figcaption>Fig. 5. Four types of cascading scenarios resulting from the continued evolution of hazard events, created by authors.</figcaption></figure>

disruptions in other systems through different types of dependency relations [7,56]. Moreover, external interdependency of a CIS goes beyond itself and refers to its interactions with all other human-environment components [5,57,58]. Human-environment includes physical, atmospheric, biological, social, economic and political components, conditions, and factors that influence the state, condition, and quality of living conditions, employment, and health [59]. Spatial and temporal interactions across networks and within CISs or cities, is of paramount concern for the resilience of human-related systems. Some historical events show that catastrophic impacts of CISs origins (disruption, construction, action, etc.) have already occurred on systems beyond the origins themselves [60–64]. The trend of interdependency comes from the awareness of the cascading effects due to interconnected urban components in the development community. Increasing hazards require the urban system to cope with potential cascading effects after consequences on CISs [65]. Furthermore, from a consequence-based approach, the negative effects caused by used resources and measures should also be taken into account for the analysis of interdependencies [5,66]. That is, the damage caused by a catastrophic event to one infrastructures, and the actions implemented on it, can cause secondary serious damages to the externally associated system. A resilient infrastructural system should have the ability to manage multiple equilibriums with other urban systems.

Overall, the phenomena of CISs resilience today should address the potentially cascading effects that could occur on itself or its connected urban components. Whether the effects are inside or outside that infrastructural system, positive or negative, they involve four types of cascading scenarios resulting from the continued evolution of hazard events (Fig. 5, A.1, A.2, B.1, B.2), following the initial scenario illustrated in Fig. 5 (adjusted from Fig 4).

For an affected CIS, positive effects refer to the effectiveness of implemented actions that optimise this CIS to better face future hazards. The following scenarios, relating to positive effects, are that future hazards affected this optimised CIS. Meanwhile, the implemented action might also harm this CIS, so they become the source of side effects in continuous scenarios. The continuous scenarios concerning effects in this CIS refer to:

* Effectiveness of action: the optimisation of target CIS resulting from actions presents in a new resilience scenario, which consists of another hazard arriving after the initial scenario, the improved CIS, the consequence of the new hazard on this CIS, and the actions of the improved CIS (Fig. 5, A.1);

* Side effects of action: the implemented action causes side effects on the initially affected CIS. It produces a new resilience scenario, which consists of this action as the source of a side effect, the same CIS
<figure><img src="image_10.png" /><figcaption>Fig. 6. Event tree produced by an initial scenario, adjusted from Zuccaro et al. [67].</figcaption></figure>

affected initially, and the damage of implemented action to this CIS, and the actions of this CIS (Fig. 5, A.2).

Moreover, the implemented actions are considered the source of cascading effects when they affect the external system. Similarly, the ‘consequence’ in the initial scenario (in Fig. 5) that leads to more damage, becomes the source of cascading effects in continuous scenarios. The continuous scenarios, concerning the cascading effects in external urban systems, refer to:

* Cascading effects of action: the implemented action negatively affects external urban systems. It produces a new resilience scenario, which consists of this action as the source of a cascading effect, an indirectly affected CIS or urban system, their consequence and action (Fig 5, B.1).

* Cascading effects of consequence: the consequence of initially affected CIS cause cascading effects on external urban systems. It produces a new resilience scenario, which consists of the consequence as the source of a cascading effect, an indirectly affected CIS or urban system, and similarly, their consequence and action (Fig. 5, B.2).

The ‘actions’ and ‘consequence, which make negative effects in these continuous scenarios (Fig. 5, A.1, A.2, B.1, B.2), plays the same role as the ‘hazard’ in the initial scenario in Fig. 5.

Consequently, the studied phenomena identified refer to all scenarios illustrated in Fig. 5, which provides four basic types of cascading

scenarios resulting from the continued evolution of hazard events. To identify all continuous scenarios resulting from initial scenarios, Event Tree Method (EMT) currently used by recent studies for analysing domino effects could be suggested [67]. As shown in Fig. 6, the form of an event tree allows for establishing causal chains from the initiating scenario to the scenarios upon, and each chain of scenarios is represented by a path in the event tree [68]. While these successive scenarios can lead to an infinite number of further continuous scenarios, scenarios closer to the original scenario have a higher probability of occurring and are more important. This study thus tends to focus on scenarios at level A, and one of the scenarios at level A will present in Section 4.

Due to the complex interaction between urban systems, in practical terms, it is difficult to distinguish between direct and indirect damages [41]. For instance, when flooding occurs, it directly affects road infrastructure and agricultural land. The damage to these two systems in turn affects each other. The investigation of initial and scenario scenarios is conceptual and for interpreting the importance of the consideration on cascading events.

### 3.1.2. Important aspects of CISs resilience

The conceptual scenario of resilience shown in Fig. 4 presents already two main aspects of resilience, i.e. “consequence” and “action”. A MCF for CISs resilience, therefore, needs to provide information about these two aspects.

Many studies [69–72] believe consequence-driven approaches should be considered as a key to hazard risk management. Based on a paradigm of consequence-based engineering (CBE) created by Abrams
<figure><img src="image_11.png" /><figcaption>Fig. 7. Stages of resilience scenario, source: Yang et al. [12].</figcaption></figure>

[73] aiming at performance-drive assessment, Wen and Ellingwood [74] and Wen et al. [75] refined and detailed uncertainty and vulnerability analysis through the consideration of demand, capacity, repair costs and losses of engineering systems. Furthermore, Ellingwood and Wen [69] apply the CBE for analysing the earthquake risk of buildings and transportation infrastructure. Indeed, in the field of CISs resilience, consequence assessment has been applied in several current studies [76–79]. Kabir et al. [80] and Heinimann and Hatfield [81] argue that, for a society exposed to high-consequence events, such as earthquakes, tsunamis, and floods, a consequence-based decision-making framework needs to be previously proposed for different applications. In addition, potential action analysis helps identify decisions that should be taken to reconcile objectives and constraints in the best possible manner [82].

In the field of risk management, consequence-based and action-based strategies cannot be separated. The aim of implementing actions is to reduce consequences, and consequence assessment can be used for designing actions. Consequences in a given resilience scenario can be used as experience to enhance future actions. In addition, the effectiveness of some actions can be observed by whether the consequences are reduced in future resilience scenarios. Consequence-action-drive assessment is an endless and continuous process.

CISs resilience is always described through the properties and capabilities of infrastructures or catastrophic event stages. A combined assessment of action and consequences also makes it possible to take account into all capabilities and properties of CISs, as well as all catastrophic event stages. The capabilities of a system could be capacities, characteristics, abilities, resources, and knowledge [83–87]. Catastrophic event stages could be divided into [12] (see Fig. 7):

* Pre-event stage (PrES): from the occurrence of a hazard to the beginning degradation of the function of CISs,

* During-event stage (DES): from the beginning of the degradation to the maximum degradation of the function of CISs,

* Post-event stage (PoES): from the maximum degradation of the function of CISs to the function returning to the level of the pre-event stage or recovering to an ideal state, structure, or property (but still lower than the original state), and

* Next event preparation stage (NEPS): from the function returning to the level of the pre-event stage to the occurrence of the next shock on CISs. This stage emphasises the ability to learn and improve from experience. The actions made in this stage refer to the scenario relating to actions' effectiveness in the long-term (Fig 5. A.1).

The implementing actions are required content of a CIS, and all actions are implemented for improving or changing the capabilities, properties, or status at event stages of CISs resilience [5,12,43]. The ultimate aim of these improvements and changes is to optimise the resilience of CISs and their connected systems. Therefore, differing from the studies that marked the target capabilities, properties, or event stages, this study suggests the ‘action’ aspect, which allows the objective MCF to involve all these potential targets.

## 3.2. General criteria

The establishment of criteria should be founded on the defined aspects [35], i.e. consequence and action in this study, and on thinking about the factors that should be observed during practice assessment [26].

Concerning the “consequence” aspect, this study considers the negative consequence of all components and suggests the first general criterion, which has been highly used in resilience assessment approaches according to current review works [12,88–90]: “Damage to internal components”. All components of a system interact, inter-support, and inter-influence for the existence, function, and development of this system [5]. Thus, this criterion consists of numerous parts corresponding to each internal component. The definition of specific sub-criteria of “Damage to internal components” requires the identification of significant damages, in the company of the manager of the specific case.

For the “action” aspect, the definition of general criteria requires a point of view of organisational management [91]. The positive or
<figure><img src="image_12.png" /><figcaption>Fig. 8. General criteria relating to a single scenario, created by authors.</figcaption></figure>

negative impact of an action is related to issues of efficiency and safety, which determine the degree of satisfaction of actions [51,92,93].

The consideration of cost-benefit or cost-effectiveness of actions has been discussed in some resilience studies [11,94] concerning action efficiency. These analyses are used to compare the positive or negative, advantages and disadvantages of a potential decision [95]. In addition, they are commonly emphasised in decision-making approaches [96–98], as they estimate or evaluate the benefits, effectiveness, or profits against the cost of a decision, project, or policy [99]. For the resilience assessment, cost-benefit or cost-effectiveness analysis could become one tool that can provide such information about the prioritization of risk management and adaptation options [100]. Therefore, two factors are suggested: effectiveness of action referring to actions’ positive benefits or profits, and efforts for actions referring to the efforts that need to be paid for action implementation. High cost-benefit means less cost of action with high effectiveness. Therefore, two general criteria are suggested: “Effectiveness of actions”, and “Effort for actions”.

On the other hand, another criterion “Damage of action” is suggested for assessing the safety and security of implementing actions. The idea of "safety of actions" comes from “drug safety”, which heavily focuses on adverse drug reactions. Side effects are defined as any response to a drug that is noxious and unintended, including lack of efficacy [101]. The “Damage of action” criterion aims at assessing the negative effects of unintended adverse events resulting from implementing actions in one single CIS. Whether the effects are inside or outside that infrastructure, they could create negative cascading effects on the indirectly affected object (compared with the CIS directly affected by the original hazard). Therefore, the assessment for "Damage of action" could be considered a new resilience assessment of indirectly affected objects.

Some costs of actions could have harmful points, which make costs misunderstood as consequences of actions. In this study, costs are needed in the course of implementing actions, consumed by internal components, and are mainly foreseeable, even if the final consumption amounts were not initially foreseen, they should be known and supported by the decision-make. On the other hand, damage from unintended adverse events is not expected and is not intentional. Damage of action is not necessarily linked to internal components and can refer to damage to external systems (Fig. 8).

Even though the defined aspects and general criteria of the designed MCF are not directly involved in the highly discussed capabilities, properties, or event stages mentioned in Section 3.1, the criterion "Effectiveness of actions" could be linked to all of them. All desirable optimisations of a CIS depend on the actions undertaken on it or on its internal components of the system. A CIS that has been artificially constructed cannot be resilient without human-provided actions. The

assessment of the effectiveness of actions could therefore be regarded as an assessment of one or more targeted capabilities, properties, or event stages that the implementing actions focus on. The integral combination of these targets is resilience.

### 3.3. A guide for specific sub-criteria definition

As mentioned in Section 2, a practical MCF should provide a guide for consulting potential users to create sub-criteria corresponding to different concrete situations. In this study, the sub-criteria that correspond to the general criterion “damage to internal components” relating to the aspect “consequence”, refers to functional, mental, or physical damages of critical components that are considered significant by CIS managers. This means that a sub-criteria is selected based on its significance in practical situations, as not all damage to internal components is of value to the resilience assessment in real cases. Similarly, the actual stakeholders taking into account the significance of the following elements determine the sub-criteria corresponding to the ‘action’ aspect:

* for the general criterion “effectiveness of action”: ideal outcomes because of implementing actions;

* for the general criterion “effort for action”: significant economic, functional, environmental, and resource costs produced by implementing actions;

* for the general criterion “damage of action”: significant functional, mental or physical damage caused by implementing actions to target CISs or other interconnected systems.

Traditional risk approaches would seek to identify the range of possible scenarios and events [102]. To find suitable criteria applicable to the target practical situation, it is necessary to predefine a studying resilience scenario. The pre-mentioned conceptual scenario (see Fig. 4) is also used in this step for helping users define the studied scenario. Four indispensable factors need to be clarified: hazard, affected system, consequence, and action:

* The “affected system” is therefore a target CIS;

* The “hazard” is one or more potential catastrophes of the target CIS;

* The “consequence” refers to the damages caused by the hazard on target CIS;

* The “action” could be one or several implementing actions for improving the resilience of the target CIS.

The definition of all sub-criteria depends on the knowledge and information of the studied scenario obtained by each user. The former two refer to the study object of each case and are therefore easier to be identified than the latter two. Therefore, the definition of “affected CIS” and “hazard” is placed first. The method and process of the identification of the significant factors relating to “consequence” and “action” are shown separately in “Form 1” and “Form 2”. When a scenario consists of more hazards and actions, it requires taking into account the effect of their superimposition. During the use of forms, infrastructures managers with different competencies work together, exchange and discuss their joint view of all issues for making collective decisions.

#### 3.3.1. “Consequence” aspect

The identification of significant damages is based on the use of ‘Form 1’, which is a mission to carry out the specific sub-criteria for the criteria “damages to internal components”. According to Yang et al. [5], the internal components of each CIS could be categorised into four main types: human components are categorised as individual or collective, while non-human components are divided into physical structures and non-physical existents. For CISs resilience studies, the last of these could be referred to as the “main function of a target CIS” [5]. This leads to the
<figure><img src="image_13.png" /><figcaption>Fig. 9. Form 1 for defining damage-related sub-criteria, created by authors.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 10. Form 2 for defining sub-criteria of “Effectiveness of action” and “Effort for action”, created by authors.</figcaption></figure>

first part of the consequences: “Damage of main functions”. As the function of a CIS is based on the function of all components of this CIS, the damage to components’ function is one of the essential factors to be considered for CISs’ damage. As a result, the second part could be defined as “Functional damage to components. Furthermore, amongst all internal components, physical structures are physical material objects, whose physical damage is common and needs to be assessed additionally. Besides, in considering human injury, the individual human actor has both physical and mental damage, i.e. physical injury and mental injury [103,104]. On the other hand, the collective human actor which generally refers to a public or private organisation, unit, or company is a virtual object or concept created for human society. Therefore, only their functional damage should be taken into account. All significant functional, physical, and mental damage could be translated to the sub-criteria for defined scenarios if they are considered related to the assessment goal.

### 3.3.2. “Action” aspect

For the sub-criteria related to the “action” aspect, not only implementing actions should be identified, the ideal outcome and costs of actions need to be clarified. The process is presented in ‘Form 2’.

The choice of implementation actions is various and multidimensional. Recently, some frameworks are created for CIs stakeholders to design implementation actions for improving CISs resilience [47,85,102, 105]. Among these theories and methods, we highlight the “Behind the Barriers” model (BB model), developed by Barroca and Serre [43], which allows effective and comprehensive development of infrastructural system resilience by considering the interdependencies in various urban scales. BB model argues that the actions for improving capabilities could be described in four dimensions:

* A cognitive dimension refers to knowledge, awareness, and the identification of resilience by the persons concerned. Cognitive actions refer to all processes relating to knowledge, thus the
<figure><img src="image_15.png" /><figcaption>Fig. 11. Hierarchical structure relating to continuous scenarios, created by authors.</figcaption></figure>

processes of identification, acquisition, and processing of information that is produced by population or managers for risk identification and resilience evaluation.

* A functional dimension specific to material objects and technical urban systems forming the territory. Functional actions are implemented by working on reliability (by overprotecting infrastructures), increased redundancy (by finding different pathways or using several action modes for the same service), and risk-related stock management (by creating temporary or permanent storage facilities close to the place of use).

* A correlative dimension that recognizes that service and utilisation form a whole whose different sections are interconnected together. Correlative actions aim at balancing need and service capacity in the targeted internal infrastructural system.

* An organisational dimension that raises the question of the persons involved (public and private players, populations, etc.) and the strategies that contribute to improving resilience. Territorial actions depend on the capacity of the organisation and management beyond infrastructure itself or local conditions, thus designing general measures at larger scales.

The applications of the “Behind the Barriers” model to resilience analysis are wide and various, as it is a multidisciplinary, transversal resilience theory aiming at adaptation capacity [5,10,106,107]

The criterion “Effectiveness of action” depends on the content of the action, including its objective, obtained result, and efforts [108]. The sub-criteria designed based on the objective of actions show the performance of an action by the information of ideal outcomes of actions corresponding to the objective. The sub-criteria, under the “Effort for action” criterion, refer to the consumption of the measures implemented and has four dimensions: environment, function, economic, and human or material resource. Economic cost and human-material resources are more common. Environmental costs could be described as planned consumption or effects of the factor in an energy-natural system. The energy-natural system includes a natural environment composed of air, water, soil, radiation, land, forest, wildlife, flora and fauna, etc. [109], and materials produced or consumed, such as food, waste, water, etc. [33]. Functional costs address the planned functional interruption or reduction of CISs components. The functional costs could refer to the functional unavailability of a component of a CIS. For example, the action of repairing railway tracks necessarily requires the suspension of the use of the railway lines where the repair point is located. In this case, the functional cost of railway track maintenance is the interruption of the function of the relevant railway lines. Functional costs could also involve a reduction in use despite the functionality of the component

being available. For instance, in the example just used, the maintenance does not reduce the service capacity of the train station located on the suspended railway line. However, the suspension of the railway line reduces the number of trains in and out of the station, so the functionality of this train station is also reduced. Thus, functional cost refers equally to the service reduction of the train station, not only service capacity reduction.

### 3.4. Damage of action and continuous scenarios

In summary, the sub-criteria, under “Damage to internal components”, “Effectiveness of action” and “Effort for action” in a defined scenario, are based on the information obtained about the affected components, their failure modes, the objectives of the selected actions, the costs of the actions, etc. (see Figs. 9 and 10). However, the sub-criteria for the "Damage of action" criterion are not discussed in the above sections. The "Damage of action" criterion requires the investigation of continuous resilience scenarios as shown in Fig. 5 (A.2, B.1). The key is the internal components and external systems that have suffered indirect functional, mental, or physical damage. These cascading damages imply the emergence of new resilience scenarios. The criterion "Damage to internal components" in these two continuous scenarios (Fig. 5, A.2, B.1) is therefore equal to the criterion "Damage of action" in the initial scenario (see Fig. 11). Form 1 in Fig. 9 is also adapted to defining significant damage of action in potential continuous scenarios.

Moreover, continuous scenarios also involve:

* the cascading effects of damage that has already occurred (see Fig. 5, B2);

* and the positive effects of actions in the long term (see Fig. 5, A1).

As continuous scenarios are endless, they allow assessing CIS resilience to be continuous over time. Nevertheless, as mentioned in Section 3.1.1, this study tends to focus on the scenarios closer to the original scenario that have a higher probability of occurring and are more important. The resilience assessment of a CIS should rely also on significant scenarios listed in Fig. 5. All continuous scenarios studied in this paper are all based on the conceptual scenario of resilience (see Fig. 4). The components of all studies scenarios are therefore highly similar. Thus, the important aspects, the defined criteria, and the guide designed for the scenario could apply to all scenarios.

This study highlights its prospect that is summarised in Fig. 11: contribution to the development of a hierarchical structure for CISs resilience assessment.
<figure><img src="image_16.png" /><figcaption>**Fig. 12.** Initial and continuous scenarios of presented example, created by authors.</figcaption></figure>

### 3.5. Developed mcf summary

The design of MCF is based on the collected knowledge about CISs resilience, especially investigations of “definition and phenomena” and “important aspects”. The general criteria in the designed MCF could be adapted to all CISs, while the specific sub-criteria are applicable only to relevant target scenarios. The guide for defining sub-criteria begins with the definition of scenarios (initial scenarios and continuous scenarios). After that, the identification of significant damages, actions' outcomes, and costs could follow “form 1” and “form 2”. The resilience, identified aspects, criteria, and sub-criteria produce a part of a prospective hierarchical structure for CIS resilience assessment. All contents of developed MCF are interconnected because the process of the establishment relies on reliable logical development.

Since the definition of sub-criteria highly corresponds to specific cases, an example is shown in Section 4 for presenting how to use the guide.

## 4. Example application of the designed guide

Combining the presentation above for the designed MCF, for each user, defining the ‘hazard’ and ‘affected system’ in a predefined initial scenario (like in Fig. 4) is the first essential step. Defining the continuous scenarios relating to the initial one, and repeating the first step (defining the ‘hazard’ and ‘affected system’), are follow-on works. This example targets a specific occurrence, in which Nantes Ring Road (NRR) system is affected by a flood, as an original scenario. Section 4.1 focuses on identifying the sub-criteria related to the initial scenario, while Section 4.2 focuses on one of the possible continuous scenarios because the resilience scenarios are continuous and numerous due to the interconnections between various urban systems and components. The selected continuous scenario fits type B.2 in Fig. 5, and in this example, is defined to be concerning the dysfunction of the NRR system affecting the Emergency Medical System (EMS) in Nantes (see Fig. 12). Section 4.3 presents a discussion about the “Damage of action” criterion in this example without detailed sub-criteria.

With a length of 42 kms, the NRR system has services extending beyond the local level and is attractive in the region and even in the nation [110]. However, the section between the "Porte de la Chapelle" and the "Porte de la Beaujoire" is frequently closed due to the flooding of the Gesvres River [111]. NRR is selected in terms of its interesting situation for studying cascading effects. The flood hazard in Nantes city causes less direct damage to other urban components except for NRR, in comparison with other cities in southern France. This fact lets this study focus firstly on the consequence of NRR caused by flood, and secondly on the cascading effects caused after the first scenario. The direct and indirect consequences would be difficult to be distinguish if flood hazards affect directly a large number of urban components. The SMUR in Nantes was chosen for its urgent and crucial service, whose delay of one minute minute's delay can cause significant damage to victims.

<caption>**Table 1**</caption>

Significant damages for the defined initial scenario F-NRR-PAR, created by authors.

<table><thead><tr><th>Components</th><th>Functional damage</th><th>Physical damage</th><th>Mental damage</th></tr></thead><tbody><tr><td>Transport function</td><td>X</td><td></td><td></td></tr><tr><td>Managers</td><td>X</td><td></td><td></td></tr><tr><td>Individual users</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Maintenance and intervention centre</td><td></td><td>X</td><td></td></tr><tr><td>Vehicles</td><td></td><td>X</td><td></td></tr><tr><td>Road structures</td><td></td><td>X</td><td></td></tr></tbody></table>

### 4.1. Nantes ring road in flooding (Initial Scenario)

The initial scenario is called F-NRR-PAR (flood - Nantes Ring Road - planning alternative roads) in which a flood is defined as the hazard while NRR is defined as the studied system affected by the flood (see Fig. 12). This study uses the research findings of Yang et al. [5] about the identification of internal components of NRR (including the main function of NRR) and their functions (see Appendix 1). Following the process of “from 1”, a scientific team, involving staff of Cerema Ouest that participates in the management of NRR, identifies 8 significant damages to 6 components (Table 1).

Based on the internal documents of Cerema Ouest and BB theory [43], to increase the resilience of NRR in such situations, stakeholders could develop various actions in considering cognitive, functional, correlative, and organisational dimensions (see Appendix 2). The actions listed in Appendix 2 are intended as a reference only and do not represent all the actions that could be implemented. amongst these actions, the action “Planning Alternative Roads” is frequently applied and is used here to give an example for identifying outcomes and costs of actions. During the closure of the section between the "Porte de la Chapelle" and the "Porte de la Beaujoire", local road management [112] suggests the alternative roads shown in Fig. 13. The implementation of this measure has one principal objective: allowing users using alternative roads. The ideal outcome of implementing the action would be the increased transport function of the alternative roads. The completion of the action relies directly on two relevant internal components, the “Managers” that plan it and the “Individual users” who use it. The material and economic costs are then dominated by their costs. For example, the economic cost of individual users is referred to a study from Cordier [113], in which the average cost of an individual French vehicle is 33.5 cents per km (ignoring tolls and parking, which are linked to particular roads, motorways, tunnels, certain urban roads, etc.). The environmental cost refers to the component that causes harm to the environment during the implementation of the action. For instance, it could be the air or noise pollution from "Vehicles" in this scenario.

Based on all investigations above, the sub-criteria for the studied initial scenario F-NRR-PAR are summarised in Table 2.

### 4.2. Cascading effects on emergency medical system (Continuous Scenario)

The second part of this case study aims at a continuous scenario referring to the cascading effect caused by dysfunction of the inundated NNR system. In the paper by Yang et al. [5], it is shown that the dysfunction of the NRR affects the services of the city's Emergency Medical Service (EMS) system. Then this study takes the EMS system in Nantes as the second affected system to be studied. Furthermore, Yang et al. [5] consider the use of alternative roads (the same as shown in Fig. 13) for ambulances as a measure for EMS to face the closure of NRR. This means that, in the new resilience scenario, the hazard is the "NRR dysfunction", the affected urban system is Nantes EMS (see Fig. 12, part 2). This scenario is called DNRR-EMS-UAR (dysfunction of Nantes Ring Road - Emergency Medical Service - using alternative roads). The
<figure><img src="image_17.png" /><figcaption>Fig. 13. Suggested alternative roads in flooding events, source: Yang et al. [5].</figcaption></figure>

function of all components is listed in Appendix 3.

This study tries to apply the designed guide for CISs resilience to the Nantes EMS system, and also to test its suitability for urban socio-economic systems. In this scenario, the identification of significant damages is based on an interview with an expert having working experience in French ambulance (see Table 3). The action in this scenario is closely similar to the action in the initial scenario, the only

elements that have changed are the two internal components that
complete the action, which have changed from "Manager" and "Indi-
vidual users" of NRR to "Ambulance drivers" of Nantes EMS. At the same
time, the objective becomes: allowing "Land vectors" to go through
alternative roads and rebooting the mission of EMS "emergency medical
services to patients", stopped earlier due to the NRR dysfunction. The
economic and environmental costs are not significant in this scenario
<table><caption>**Table 2**
Sub-criteria for the studied initial scenario F-NRR-PAR, created by authors.</caption><thead><tr><th>Criteria</th><th>Sub-criteria</th></tr></thead><tbody><tr><td rowspan="7">Damage to internal components</td><td>Damage to transport function</td></tr><tr><td>Functional damage to managers</td></tr><tr><td>Functional damage to individual users</td></tr><tr><td>Physical damage to individual users</td></tr><tr><td>Mental damage to individual users</td></tr><tr><td>Physical damage to maintenance and intervention centre</td></tr><tr><td>Physical damage to Vehicles</td></tr><tr><td>Physical damage to road structures</td></tr><tr><td>Effectiveness of action</td><td>Increased transport function of alternative roads</td></tr><tr><td rowspan="4">Effort for action</td><td>Resource costs of managers</td></tr><tr><td>Time costs of individual users</td></tr><tr><td>Economic costs of individual users</td></tr><tr><td>Environmental costs from vehicles</td></tr></tbody></table>

<table><caption>**Table 3**
Sub-criteria for the defined continuous scenario DNRR-EMS-UAR, created by authors.</caption><thead><tr><th>Criteria</th><th>Sub-criteria</th></tr></thead><tbody><tr><td rowspan="5">Damage to internal components</td><td>Damage to medical function</td></tr><tr><td>Physical damage to patients</td></tr><tr><td>Mental damage to patients</td></tr><tr><td>Functional damage to staff</td></tr><tr><td>Mental damage to staff</td></tr><tr><td>Effectiveness of action</td><td>Restored medical function</td></tr><tr><td>Effort for action</td><td>Increased time costs of ambulance drivers</td></tr></tbody></table>

that focus on citizen healthy. The identification of sub-criteria is based on the designed guide presented in Section 3.3 and the results are summarised in Table 3.

### 4.3. Scenarios involved “Damage of action”

The sub-criteria identified above for both scenarios are only able to assess "Damage to internal components", "Effectiveness of action" and "Effort for action". As mentioned in Fig. 11 the sub-criteria to assess the criterion of "damage of action" must be based on the analysis of the continuous scenarios in relation to the side effects of implemented action. The resilience of CISs is highly correlated with the safety and security of the actions carried out. An action with low safety and security can lead to more serious negative effects on the infrastructural system and even on the other urban systems associated with it. Therefore, the damage assessment of action must be linked to the consequences in continuous scenarios, both on that infrastructural system itself and on other urban systems.

"Damage of action" is easily confused with the costs of actions, as the latter may also be considered as a negative impact of actions. Thus, this study makes it clear that the costs of action refer to the foreseeable negative effects (condition 1), consumed by international components (condition 2) that arise in the course of action implementation (condition 3). Damage of action does not arise during its execution and its development is unpredictable. For example, in the continuous scenario of this case, if the implementation of alternative roads continues for years and years, the public would doubt the EMS capacity of Nantes. Even if the Nantes ring road reopen, the trust of the public in the health system would be still impacted. Therefore, the public distrust for EMS, even though it is foreseeable, is not an effort but a side damage because it occurs after the implemented action. For managers, the decision about which action should be implemented, for how long, and the extent of the effort should be based on an assessment of the damage of actions.

## 5. Discussion

### 5.1. Practical applications of the designed guide

The example does not list all the relevant continuous scenarios based on the initial scenario but demonstrates how sub-criteria for one single scenario can be identified based on the developed user guide. The sub-criteria definition for the above two scenarios can be used as a reference for other scenarios that have not been analysed. From a theoretical perspective, the case demonstrates that there is an endless occurrence of continuous scenarios and therefore endless resilience assessment and sub-criteria definition. Thus, for practice management, significant scenarios for resilience assessment should be selected based on the Event Tree Method (EMT), and depends on the circumstances of each specific case.

The results of criteria identification could help define optimisation actions for both two analysed systems, the NRR and EMS. For example, according to the identified sub-criteria, additional "Time costs of individual users", DIRO could reduce additional time by some changes of road equipment, like removing speed limits, minimising barriers and eliminating one-way streets. Meanwhile, every ambulance driver could be considered an individual user. The managers of EMS could also reduce additional time by improving ambulance performance or changing transport sectors. However, the two urban systems need to be monitored and assessed through continuous supervision of their internal structure, as well as their connections with other urban systems, in a context of continuous environmental and social change. Sub-criteria for assessing the resilience of urban CISs cannot be set in stone with the challenge of increasing unexpected disasters. For example, each country experienced an unprecedented epidemic disaster in 2020. This context produces a scenario contrary to the example given: the dysfunction of Nantes EMS affects the performance and efficacy of the NRR transport security service [5]. The sub-criteria, also relating to NRR and EMS systems, therefore should be re-identified.

Many existing theories or models for CIS resilience assessment are valuable, although they differ in the definitions and perspectives of this study. Nevertheless, this study insists that for resilience theory to become practical, it is necessary to consider not only the cost-effectiveness and negative effects of the operation, but also the uniqueness of each case. Just as teaching a man to fish, rather than simply giving him fish. Rather than predefining criteria for all potential resilience scenarios of CISs, the MCF provides a step-by-step guide that helps identify specific sub-criteria based on concrete situations. The methodology, therefore, allows a wide margin of autonomy for managers and policymakers who have the responsibility for building CISs resilience and need support and guidance to operationalize the resilience-building process. MCF includes a continuous multidimensional assessment of positive and negative aspects, which can better help infrastructure managers to make decisions that are more profitable.

At the same time, in the presented example, this study tries to apply this designed guide to the selected socio-economic system, the Nantes EMS system. The results prove the possibility of applying the designed MCF to other urban systems. Some theories or models used in this study, like C&I-based assessment, and cost-benefit/effectiveness, as well as the interpretations for resilience, are universally suitable for a variety of disciplines. Then, it is considerable for the definition of general criteria and sub-criteria in this study whether they can be used in other human-related systems.

### 5.2. Prospects, limitations and future works

This study has consistently emphasised that the designed MCF forms part of a hierarchical system for multi-criteria analysis in relation to indicators. The defined general criteria and identifiable specific criteria, in combination with aspects and indicators, can produce a hierarchical
structure for continuous assessment of CISs resilience. On the other hand, how to use the created indicators, identified sub-criteria, and defined criteria to assess CISs resilience is also worth more future work. The available methods for resilience, damage or cost-benefit assessment are various, like scoring, modelling, and could be quantitative, qualitative, or semi-quantitative. For example, Mebarki et al. [114] propose a probabilistic framework for assessing the resilience-related damages to buildings caused by flooding, regarding both infrastructural structural damage and socio-economic damage. Chen and Elise Miller-Hooks [115] present equations for measuring specified recovery costs (budgetary, temporal, and physical) for intermodal freight transports. Chang [116] demonstrates a life cycle cost analysis in considering: planned costs incurred by the lifeline agency, unplanned costs from seismic hazards incurred by the agency, and their associated societal costs. Which existing methods or theories are more appropriate for the designed multi-criteria framework and more suitable for CISs resilience assessment deserves further discussion. In addition, the assessment of each indicator requires a reference that needs to be adapted to the practical case, and not universal. In short, the designed MCF is still in its infancy and it is hoped that more research work will be done in the future to complete it and put it into practice.

Furthermore, the hierarchical structure relating to C&I (see Fig. 1) contributes frequently to Multiple-Criteria Decision-making (MCDM), which is particularly in psychological and management research. MCDM, named also Multi-criteria decision analysis (MCDA), is originally a decision-making tool used in environmental sustainability to evaluate a problem by giving an order of preference for multiple alternatives based on several criteria [117]. MCDM now become a tool developed for complex multi-criteria problems and participative allowing direct involvement of multiple experts, interest groups, or stakeholders and presentations of demands that are relevant to their interests [118]. From a perspective of organisational management, to make the concept of resilience into operational practice, MCDM could be integrated into CIS resilience assessments. Furthermore, the C&I involved in CISs resilience assessment needs to be weighted, as one of the greatest steps of MCDM is the subjectivity of the weighting with dimensions decided by stakeholders [117]. For example, the contributions of different damage to total consequence could be identified by weighting the sub-criteria relating to internal components damages.

A sufficient autonomy for users, i.e. infrastructure managers, in defining scenarios and sub-criteria is an advantage of the created MCF. At the same time, however, it can also be interpreted as a weakness. Managers' experience or knowledge may be so limited that they overlook invisible factors. From a holistic perspective, a collaborative multi-stakeholder exchange can reduce this shortcoming. But, a significant investment of human resources at the same time that may reduce the cost-benefit of collaborative management. Research in the field of management is therefore needed for a better application of the MCF.

Besides, around this study, more topics could be raised for complementing C&I-based resilience assessment. For instance, this study suggests overlaying the effects of more actions for the final decision-making process. However, in practical situations, possible actions could be inter-reinforcing or inter-constraining. The method for integrating the interactions of actions in a scenario into the created MCF could be therefore mentioned. Similarly, many infrastructural systems are facing the challenge of multi-hazards. It is also necessary to analyse the interactions or interdependencies of two or more hazards that exist in the same given scenario.

## 6. Conclusion

Focusing on the resilience assessment of Critical Infrastructure Systems, this study develops a multi-criteria framework (MCF), consisting of four general criteria, “damage to internal components”, “effectiveness of action”, “efforts for action”, “damage of action”, and a guide for defining specific sub-criteria. This study presents also the application of

## Appendix 1

All components of NRR and their principal functions, source: Yang et al. [5].

<table><thead><tr><th>Categories</th><th>Components</th><th>Principal Functions</th></tr></thead><tbody><tr><td rowspan="4">Human collective components</td><td>Managers</td><td>Ensure the daily operation of NRR, providing comfort and safety to users, through the management and maintenance of roads</td></tr><tr><td>Project managers</td><td>Project management of investment operations (public or private) and management of the noise observatory of the NRR and of the flood-warning project for the eastern part of Highway Infrastructure</td></tr><tr><td>State partners</td><td>Define and fund projects</td></tr><tr><td>Safety observation</td><td>Produce and disseminate information on road safety</td></tr><tr><td rowspan="2">Human individual components</td><td>Collective users</td><td>Organize mobilisation for different activities (posters, couriers, travellers, merchandise, health emergency services, etc.)</td></tr><tr><td>Individual users</td><td>Mobilise different activities (posters, couriers, travellers, merchandise, health emergency services, etc.)</td></tr><tr><td rowspan="7">Physical structures</td><td>Individual staff</td><td>Work for affiliated institutions to ensure system functions</td></tr><tr><td>Rest areas</td><td>Supply energy and fuel to vehicles and provide material and spiritual needs to users in dedicated service areas</td></tr><tr><td>Counting regulation</td><td>Provide information on road traffic</td></tr><tr><td>Access regulation</td><td>Improve traffic flow on the Highway Infrastructure by controlling the injection of vehicles</td></tr><tr><td>Green spaces</td><td>Protect water resources and enhance ecological transparency</td></tr><tr><td>Maintenance and intervention centre</td><td>Provide support to state institutions (such as the police), cleaning, ordinary and extraordinary maintenance (road signs, lighting, localised damage, etc.)</td></tr><tr><td>Drainage system</td><td>Remove surface water from the roads as quickly as possible (drainage) to ensure safety with minimum nuisance to users, implement effective subsurface drainage to maximise the lifecycle of infrastructures, minimise the impact of run-off on the external environment in terms of flood risk and water quality</td></tr><tr><td rowspan="3">Functions</td><td>Road structures</td><td>Enable mobility by the construction of horizontal structures or structures in elevation or in excavation</td></tr><tr><td>Vehicles</td><td>Transport passengers and goods on the ground</td></tr><tr><td>Transport function</td><td>Serve individual and collective users in mobility: passenger, freight, postal, or auxiliary transport services (including medical services)</td></tr></tbody></table>

this guide through a specific example based on a road infrastructure system in France. The MCF is designed based on the definition, phenomena, and important aspects of CISs resilience relating to damages and optimisation actions. The designed MCF is adapted to all types of CISs and continuously changing situations as it takes into account the fact that the development of events is uninterrupted and that requires the assessment of resilience to be continuous. The results of the resilience assessments with the multi-criteria defined through MCF could help CISs managers during decision-making process, as it is a multiple-criteria approach developed for allowing consideration of various interests of stakeholders. Overall, the designed MCF, therefore, makes it possible to define multi valuable criteria for practical operation in CISs disaster management.

## Funding

This work was supported by the UrbaRiskLab (URL) project (https://
## Appendix 2

Designed actions based on BB model, created by authors.

<table><thead><tr><th>Dimension</th><th>Target factor</th><th>Actions</th></tr></thead><tbody><tr><td rowspan="4">Cognitive</td><td>Knowledge on CI</td><td>Regular diagnosis of NRR conditions<br>Advice to users on driving on NRR during flood seasons</td></tr><tr><td>Risk prevention</td><td>Flooding information/communication alert<br>Development of flood management for NRR<br>Pre-trip flooding information (co-modal information event)</td></tr><tr><td>Level of knowledge of actors concerning</td><td>Flooding Risk education for NRR managers and staff<br>Public education to develop population awareness</td></tr><tr><td>Reliability</td><td>Creating flood dams for NRR</td></tr><tr><td rowspan="4">Functional</td><td>Stock management</td><td>Lane management with field intervention: neutralization of lanes, use of emergency or collective transport lanes for temporary events<br>Construction of temporary bridge for inundated sections</td></tr><tr><td>Increased redundancy</td><td>Replacement of permeable pavement for reducing surface water<br>Increase green space on NRR to increase water absorption<br>Increase in underground storage capacity</td></tr><tr><td>Needs control</td><td>Spreading traffic over time: deferred or stop advice<br>Distribute traffic in space: closure with mandatory exit, diversion bridge in case of flooding</td></tr><tr><td>Services substitution</td><td>Planning alternative roads during flood events</td></tr><tr><td rowspan="3">Organisational</td><td>Promote absorption</td><td>Regional rainwater management projects</td></tr><tr><td>Crisis adaptation</td><td>Developing teleworking for avoiding using NRR<br>Prohibit school transport on NRR<br>Ensure an efficient and attractive public transport offer</td></tr><tr><td>Post-crisis recovery</td><td>Regional flooding waste management</td></tr></tbody></table>

urbarisklab.org/fr/) and the RESIIST project (ANR-18-CE39-0018, https://research.gi.mines-albi.fr/display/resiist/RESIIST+Home (in French)) that is jointly funded by the French National Research Agency (ANR) and the General Secretary of Defense and National Security (SGDSN). The authors acknowledge these organizations for their support that helped improve the paper.

### CRediT authorship contribution statement

**Zhuyu Yang:** Conceptualization, Methodology, Investigation, Writing – original draft, Visualization. **Bruno Barroca:** Conceptualization, Methodology, Writing – review & editing, Supervision. **Katia Laffréchine:** Writing – review & editing, Supervision. **Alexandre Weppe:** Writing – review & editing. **Aurélia Bony-Dandrieux:** Writing – review & editing, Supervision. **Nicolas Daclin:** Writing – review & editing, Supervision.

### Declaration of Competing Interest

The author(s) declares no conflict of interest.

## Appendix 1

## Appendix 3

All components of Nantes EMS and their principal functions, source: Yang et al. [5].

<table><thead><tr><th>Categories</th><th>Components</th><th>Principal functions</th></tr></thead><tbody><tr><td rowspan="2">Human collective components</td><td>Manager structure</td><td>Organise the first emergency care of patients and that after transport to the nearest indicated health structure, or organize inter-hospital transport</td></tr><tr><td>Doctors</td><td>Manage patients for the development the first diagnosis</td></tr><tr><td rowspan="3">Human individual components</td><td>Nurses</td><td>Provide first aid to patients in case of medical emergencies in a public or private context, stabilise patients, and enable transport</td></tr><tr><td>Ambulance drivers</td><td>Provide transport for medical teams and patients, ensure logistical activities during missions, monitor vehicle maintenance</td></tr><tr><td>Other service staff</td><td>Work for affiliated institutions to ensure system functions</td></tr><tr><td rowspan="2">Physical structures</td><td>Patients</td><td>Receive medical treatment</td></tr><tr><td>Land vectors</td><td>Provide mobility and loading of the SMUR teams (including the necessary equipment for urgent medical operation) and of patients from their location to the hospital, or also inter-hospital transfers</td></tr><tr><td rowspan="2">Functions</td><td>Aerial vectors</td><td>Provide mobility and loading of the SMUR teams (including the necessary equipment for urgent medical operation) and of patients from their location to the hospital, or also inter-hospital transfers</td></tr><tr><td>Medical equipment</td><td>Provide the equipment recommended for pre-hospital monitoring and operation</td></tr><tr><td></td><td>Primary and transfer missions</td><td>Offer emergency medical services to patients</td></tr></tbody></table>

## Appendix 2

## Appendix 3

## References

[1] EUR-Lex, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=legissum%3A133259, 2010. (accessed 11 May 2022).

[2] UNDRR, https://www.undrr.org/news/investing-resilient-infrastructure-better-future, 2022. (accessed 18 October 2022).

[3] M. Ouyang, Review on modeling and simulation of interdependent critical infrastructure systems, Reliab. Eng. Syst. Saf. 121 (2014) 43–60.

[4] S. Hosseini, K. Barker, J.E. Ramirez-Marquez, A review of definitions and measures of system resilience, Reliab. Eng. Syst. Saf. 145 (2016) 47–61.

[5] Z. Yang, M.F. Clemente, K. Laffréchine, C. Heinzlef, D. Serre, B. Barroca, Resilience of social-infrastructure systems: functional interdependencies analysis, Sustainability 14 (2) (2022) 606, https://doi.org/10.3390/su14020606.

[6] A. Mottahedi, F. Sereshki, M. Ataei, A. Nouri Qarhasanlou, A. Barabadi, The resilience of critical infrastructure systems: a systematic literature review, Energies 14 (6) (2021) 1571, https://doi.org/10.3390/en14061571.

[7] Q. Mao, N. Li, Assessment of the impact of interdependencies on the resilience of networked critical infrastructure systems, Natural hazards 93 (2018) 315–337.

[8] C.S. Holling, Resilience and stability of ecological systems. Annual Review of Ecology and Systematics, 1973, pp. 1–23. https://www.jstor.org/stable/2096802.

[9] Resilience Alliance, https://www.resalliance.org/resilience-assessment. (accessed 18 October 2022).

[10] B. Barroca, M.F. Clemente, Z. Yang, Application of “Behind the Barriers” Model at Neighbourhood Scale to Improve Water Management under Multi-Risks Scenarios: a Case Study in Lyon, France, Int. J. Environ. Res. Public Health 20 (3) (2023) 2587.

[11] A. Sharifi, Y. Yamagata, Urban resilience assessment: multiple dimensions, criteria, and indicators. Urban Resilience, Springer, Cham, 2016, pp. 259–276, https://doi.org/10.1007/978-3-319-39812-9_13.

[12] Z. Yang, B. Barroca, A. Weppe, A. Bony-Dandrieux, K. Laffréchine, N. Daclin, V. Chapurlat, Indicator-based resilience assessment for critical infrastructures–A review, Saf. Sci. 160 (2023), 106049.
[13] H. Cai, N.S. Lam, Y. Qiang, L. Zou, R.M. Correll, V. Mihunov, A synthesis of disaster resilience measurement methods and indices, International journal of disaster risk reduction 31 (2018) 844–855, https://doi.org/10.1016/j.ijdrr.2018.07.015.

[14] D. Guo, M. Shan, E.K. Owusu, Resilience Assessment Frameworks of Critical Infrastructures: state-of-the-Art Review, Buildings 11 (10) (2021) 464, https://doi.org/10.3390/buildings11100464.

[15] R. Osei-Kyei, L.M. Almeida, G. Ampratwum, V. Tam, Systematic Review of Critical Infrastructure Resilience Indicators, Construction Innovation, (ahead-of-print), 2022, pp. 1471–4175. ISSN.

[16] C. Curt, J.M. Tacnet, Resilience of critical infrastructures: review and analysis of current approaches, Risk Anal. 38 (11) (2018) 2441–2458, https://doi.org/10.1111/risa.13166.

[17] E. Chiappero-Martinetti, J.M. Roche, Operationalization of the Capability approach, from Theory to practice: A Review of Techniques and Empirical Applications, Debating Global Society: Reach and Limits of the Capability Approach, 2009, pp. 157–203.

[18] F. Comim, Operationalizing Sen's capability approach, in: Justice and Poverty: Examining Sen's Capability Approach Conference, 2001, pp. 1–16.

[19] G. Munda, Multiple Criteria Decision Analysis and Sustainable Development, Multiple Criteria Decision Analysis: State of The Art Surveys, 2016, pp. 1235–1267.

[20] L.A. Sierra, V. Yepes, E. Pellicer, A review of multi-criteria assessment of the social sustainability of infrastructures, J. Clean. Prod. 187 (2018) 496–513.

[21] W. Ho, X. Xu, P.K. Dey, Multi-criteria decision making approaches for supplier evaluation and selection: a literature review, Eur. J. Oper. Res. 202 (1) (2010) 16–24.

[22] K. Govindan, S. Rajendran, J. Sarkis, P. Murugesan, Multi criteria decision making approaches for green supplier evaluation and selection: a literature review, J. Clean. Prod. 98 (2015) 66–83.

[23] M. Herva, E. Roca, Review of combined approaches and multi-criteria analysis for corporate environmental evaluation, J. Clean. Prod. 39 (2013) 355–371.

[24] K. Øien, L. Bodsberg, A. Jovanović, Resilience assessment of smart critical infrastructures based on indicators. Safety and Reliability-Safe Societies in a Changing World, CRC Press, 2018, pp. 1269–1277.

[25] P. Trucco, B. Petrenj, C.D. Mauro, Resilience capacities assessment for critical infrastructures disruption: READ pilot applications (part 2), Int. J. Crit. Infrastruct. 14 (3) (2018) 221–247.

[26] F. Maggino, Developing Indicators and Managing the Complexity. Complexity in society: From indicators Construction to Their Synthesis, Springer, Cham, 2017, pp. 115–137, https://doi.org/10.1007/978-3-319-60595-1_4.

[27] E.M.L. Van Bueren, E.M. Blom, Hierarchical Framework For the Formulation of Sustainable Forest Management Standards, Tropenbos Foundation, Wageningen, 1997.

[28] R. Prabhu, C.J.P. Colfer, R.G. Dudley, Guidelines For developing, Testing and Selecting Criteria and Indicators For Sustainable Forest Management: a C&I Developer's Reference, 1, Cifor, 1999.

[29] G.A. Mendoza, R. Prabhu, Multiple criteria decision making approaches to assessing forest sustainability using criteria and indicators: a case study, For. Ecol. Manage. 131 (1–3) (2000) 107–126.

[30] S. Linser, P O'Hara, Guidelines For the Development of a Criteria and Indicator Set For Sustainable Forest Management, UN, 2019.

[31] D. Bourgeois, Information Systems For Business and Beyond, The Saylor Foundation, 2014. https://lib.hpu.edu.vn/handle/123456789/21478.

[32] R. Prabhu, C.P. Colfer, P. Venkateswarlu, L.C. Tan, R. Soekarmadi, E. Wollenberg, Testing Criteria and Indicators For the Sustainable Management of forests: Phase 1: Final Report, Cifor, 1996.

[33] S. Meerow, J.P. Newell, M. Stults, Defining urban resilience: a review, Landsc. Urban Plan 147 (2016) 38–49, https://doi.org/10.1016/j.landurbplan.2015.11.011.

[34] R.J. Klein, R.J. Nicholls, F. Thomalla, Resilience to natural hazards: how useful is this concept? Glob. Environ. Change Part B: Environ. Hazards 5 (1) (2003) 35–45.

[35] Eurostat, Getting messages across using indicators - A handbook based on experiences from assessing Sustainable Development Indicators, https://ec.europa.eu/eurostat/web/products-manuals-and-guidelines/-/ks-gq-12-001, 2014 (accessed 11 May 2022).

[36] J. Hernantes, P. Maraña, R. Gimenez, J.M. Sarriegi, L. Labaka, Towards resilient cities: a maturity model for operationalizing resilience, Cities 84 (2019) 96–103.

[37] J. Weichselgartner, I. Kelman, Geographies of resilience: challenges and opportunities of a descriptive concept, Prog. Hum. Geogr. 39 (3) (2015) 249–267.

[38] EXCIMAP, 2007. Handbook on good practices for flood mapping in Europe.

[39] E.D. Vugrin, D.E. Warren, M.A. Ehlen, R.C. Camphouse, A framework for assessing the resilience of infrastructure and economic systems. Sustainable and Resilient Critical Infrastructure Systems, Springer, Berlin, Heidelberg, 2010, pp. 77–116.

[40] E.D. Vugrin, D.E. Warren, M.A. Ehlen, A resilience assessment framework for infrastructure and economic systems: quantitative and qualitative resilience analysis of petrochemical supply chains to a hurricane, Process Saf. Prog. 30 (3) (2011) 280–290, https://doi.org/10.1002/prs.

[41] Z. Yang, K. Laffréchine, A. Bony-Dandrieux, H. Dolidon, B. Barroca, Practical indicators for road infrastructure resilience to flood risks in France, case study of Nantes Ring Road network, in: FLOODrisk 2020-4th European Conference on Flood Risk Management, Budapest University of Technology and Economics, 2021, https://doi.org/10.3311/FloodRisk2020.11.21.

[42] R. Shavelson, Indicator Systems For Monitoring Mathematics and Science Education, Publications Department, The RAND Corporation, 1700 Main Street, Santa Monica, CA, 1987, pp. 90406–92138. ISBN-0-8330-0804-8.

[43] B. Barroca, D. Serre, Behind the barriers: A resilience Conceptual Model, SAPI EN. S. Surveys and Perspectives Integrating Environment and Society, 2013 (6.1).

[44] A.M. Madni, S. Jackson, Towards a conceptual framework for resilience engineering, IEEE Syst. J. 3 (2) (2009) 181–191, https://doi.org/10.1109/JSYST.2009.2017397.

[45] L. Pearson, P. Newton, P. Roberts, Resilient sustainable cities, A future. Abingdon: Routledge 10 (2014), 9780203593066.

[46] S.A. Argyroudis, S.A. Mitoulis, E. Chatzi, J.W. Baker, I. Brilakis, K. Gkoumas, I. Linkov, Digital technologies can enhance climate resilience of critical infrastructure, Climate Risk Manag. 35 (2022), 100387, https://doi.org/10.1016/j.crm.2021.100387.

[47] Cerema, 2020. La boussole de la résilience: repères pour la résilience territorial.

[48] Z. Yang, B. Barroca, A. Bony-Dandrieux, H. Dolidon, Resilience indicator of urban transport infrastructure: a review on current approaches, Infrastructures 7 (3) (2022) 33, https://doi.org/10.3390/infrastructures7030033.

[49] N. Leveson, N. Dulac, D. Zipkin, J. Cutcher-Gershenfeld, J. Carroll, B. Barrett, Engineering resilience into safety-critical systems. Resilience Engineering, CRC Press, 2017, pp. 95–123.

[50] M.F. Costella, T.A. Saurin, L.B de Macedo Guimarães, A method for assessing health and safety management systems from the resilience engineering perspective, Saf. Sci. 47 (8) (2009) 1056–1067, https://doi.org/10.1016/j.ssci.2008.11.006.

[51] E. Hollnagel, D.D. Woods, Epilogue—Resilience Engineering Precepts, içinde: Resilience engineering: Concepts and Precepts, 2006. ISBN: 9781315605685.

[52] D. Serre, C. Heinzlef, Assessing and mapping urban resilience to floods with respect to cascading effects through critical infrastructure networks, Int. J. Disaster Risk Reduction 30 (2018) 235–243, https://doi.org/10.1016/j.ijdrr.2018.02.018.

[53] M. Ouyang, Z. Wang, Resilience assessment of interdependent infrastructure systems: with a focus on joint restoration modeling and analysis, Reliab. Eng. Syst. Saf. 141 (2015) 74–82, https://doi.org/10.1016/j.rss.2015.03.011.

[54] D. Serre, Advanced methodology for risk and vulnerability assessment of interdependency of critical infrastructure in respect to urban floods. E3S Web of Conferences, EDP Sciences, 2016, p. 07002, https://doi.org/10.1051/e3sconf/20160707002, 7.

[55] C. Nan, G. Sansavini, A quantitative method for assessing resilience of interdependent infrastructures, Reliab. Eng. Syst. Saf. 157 (2017) 35–53, https://doi.org/10.1016/j.rss.2016.08.013.

[56] N. Goldbeck, P. Angeloudis, W.Y. Ochieng, Resilience assessment for interdependent urban infrastructure systems using dynamic network flow models, Reliab. Eng. Syst. Saf. 188 (2019) 62–79.

[57] S.A. Markolf, M.V. Chester, D.A. Eisenberg, D.M. Iwaniec, C.I. Davidson, R. Zimmerman, H. Chang, Interdependent infrastructure as linked social, ecological, and technological systems (SETSS) to address lock-in and enhance resilience, Earth's Future 6 (12) (2018) 1638–1659, https://doi.org/10.1029/2018EF000926.

[58] G. Bambara, L. Peyras, H. Felix, D. Serre, Developing a functional model for cities impacted by a natural hazard: application to a city affected by flooding, Natural Hazards Earth Syst. Sci. 15 (3) (2015) 603–615, https://doi.org/10.5194/nhess-15-603-2015.

[59] National Academies of Sciences, Engineering, and Medicine, Assessment of the U. S. Outer Continental Shelf Environmental Studies Program: III. Social and Economic Studies, The National Academies Press, Washington, DC, 1992, https://doi.org/10.17226/2062.

[60] H. Tatano, S. Tsuchiya, A framework for economic loss estimation due to seismic transportation network disruption: a spatial computable general equilibrium approach, Natural Hazards 44 (2) (2008) 253–265.

[61] A. Rose, D. Wei, Estimating the economic consequences of a port shutdown: the special role of resilience, Econ. Syst. Res. 25 (2) (2013) 212–232, https://doi.org/10.1080/09535314.2012.731379.

[62] D.R. Marples, The Political Consequences of the Chernobyl Disaster in Belarus and Ukraine, Chernobyl: The Event and its Aftermath, 2006.

[63] N. Rania, I. Coppola, F. Martorana, L. Migliorini, The collapse of the Morandi Bridge in Genoa on 14 August 2018: a collective traumatic event and its emotional impact linked to the place and loss of a symbol, Sustainability 11 (23) (2019) 6822, https://doi.org/10.3390/su11236822.

[64] K. Buesseler, M. Dai, M. Aoyama, C. Benitez-Nelson, S. Charmasson, K. Higley, J. N. Smith, Fukushima Daiichi-derived radionuclides in the ocean: transport, fate, and impacts, Ann. Rev. Mar. Sci. 9 (2017) 173–203, https://doi.org/10.1146/annurev-marine-010816-060733.

[65] N.L. Engle, A. de Bremond, E.L. Malone, R.H. Moss, Towards a resilience indicator framework for making climate-change adaptation decisions, Mitigation Adaptation Strateg. Glob. Change 19 (8) (2014) 1295–1312.

[66] B. Robert, Y. Hémond, Organizational resilience: a multidisciplinary sociotechnical challenge, Resilienc Urban Risk Manag. (2012) 119–125. ISBN: 9780429216688.

[67] G. Zuccaro, D. De Gregorio, M.F. Leone, Theoretical model for cascading effects analyses, Int. J. Disaster Risk Reduction 30 (2018) 199–215, https://doi.org/10.1016/j.ijdrr.2018.04.019.

[68] J.T. Hackl, B.T. Adey, M.T. Heitzler, I.T. Josifescu-Enescu, An overarching risk assessment process to evaluate the risks associated with infrastructure networks due to natural hazards, Int. J. Performab. Eng. 11 (2) (2015) 153–168.
[69] B.R. Ellingwood, Y.K. Wen, Risk-benefit-based design decisions for low-probability/high consequence earthquake events in Mid-America, Prog. Struct. Mater. Eng. 7 (2) (2005) 56–70, https://doi.org/10.1002/pse.191.

[70] W.S. Saunders, M. Kilvington, Innovative land use planning for natural hazard risk reduction: a consequence-driven approach from New Zealand, Int. J. Disaster Risk Reduction 18 (2016) 244–255, https://doi.org/10.1016/j.ijarr.2016.07.002.

[71] Z.W. Kundzewicz, V. Krysanova, R. Dankers, Y. Hirabayashi, S. Kanae, F. F. Hattermann, H.J. Schellnhuber, Differences in flood hazard projections in Europe—their causes and consequences for decision making, Hydrol. Sci. J. 62 (1) (2017) 1–14, https://doi.org/10.1080/02626667.2016.1241398.

[72] G. Kabir, N.B.C. Balek, S. Tesfamamiam, Consequence-based framework for buried infrastructure systems: a Bayesian belief network model, Reliab. Eng. Syst. Saf. 180 (2018) 290–301, https://doi.org/10.1016/j.ress.2018.07.037.

[73] D.P. Abrams, A.S. Elnashai, J.E. Beavers, A new engineering paradigm: consequence-based engineering. Linbeck Lecture Series in Earthquake Engineering: Challenges of the New Millennium, University of Notre Dame’s Linbeck Distinguished Lecture Series, Notre Dame, IN, 2002.

[74] Y.K. Wen, B.R. Ellingwood, The role of fragility assessment in consequence-based engineering, Earthquake Spectra 21 (3) (2005) 861–877, https://doi.org/10.1193/1.19795.

[75] Y.K. Wen, B.R. Ellingwood, J.M. Bracci, Vulnerability Function Framework For Consequence-Based Engineering, MAE Center Report, 2004. 04-04, http://hdl.handle.net/2142/9177.

[76] K. Srinivasan, Transportation Network Vulnerability assessment: A quantitative Framework, Security Papers, 2002, p. 60.

[77] P.V. Carvalho, I.L. dos Santos, J.O. Gomes, M.R. Borges, Micro incident analysis framework to assess safety and resilience in the operation of safe critical systems: a case study in a nuclear power plant, J. Loss Prev. Process Ind. 21 (3) (2008) 277–286.

[78] R. Faturechi, E. Levenberg, E. Miller-Hooks, Evaluating and optimizing resilience of airport pavement networks, Comput. Oper. Res. 43 (2014) 335–348.

[79] X. Liu, Y.P. Fang, E. Ferrario, E. Zio, Resilience assessment and importance measure for interdependent critical infrastructures, ASCE-ASME J. Risk Uncert Engrg. Sys. Part B Mech. Engrg. 7 (3) (2021).

[80] G. Kabir, N.B.C. Balek, S. Tesfamamiam, Consequence-based framework for buried infrastructure systems: a Bayesian belief network model, Reliab. Eng. Syst. Saf. 180 (2018) 290–301.

[81] H.R. Heinimann, K. Hatfield, Infrastructure resilience assessment, management and governance–state and perspectives. Resilience and Risk, Springer, Dordrecht, 2017, pp. 147–187, https://doi.org/10.1007/978-94-024-1123-2-5.

[82] D.L. Alderson, G.G. Brown, W.M. Carlyle, Operational models of infrastructure resilience, Risk Anal. 35 (4) (2015) 562–586, https://doi.org/10.1111/risa.12333.

[83] R. Makadok, Toward a synthesis of the resource-based and dynamic-capability views of rent creation, Strateg. Manag. J. 22 (5) (2001) 387–401, https://doi.org/10.1002/smj.158.

[84] O. Renn, White paper on risk governance: toward an integrative framework. Global Risk Governance, Springer, Dordrecht, 2008, pp. 3–73.

[85] A. Hills, Insidious environments: creeping dependencies and urban vulnerabilities, J. Conting. Crisis Manag. 13 (1) (2005) 12–20, https://doi.org/10.1111/j.0966-0879.2005.00450.x.

[86] J. Eriksson, K.A. Juhl, Guide to Risk and Vulnerability Analyses, Swedish Civil Contingencies Agency (MSB), 2012.

[87] H. Lindbom, H. Tehler, K. Eriksson, T. Aven, The capability concept–On how to define and describe capability in relation to risk, vulnerability and resilience, Reliab. Eng. Syst. Saf. 135 (2015) 45–54, https://doi.org/10.1016/j.ress.2014.11.007.

[88] I. Gidaris, J.E. Padgett, A.R. Barbosa, S. Chen, D. Cox, B. Webb, A. Cerato, Multiple-hazard fragility and restoration models of highway bridges for regional risk and resilience assessment in the United States: state-of-the-art review, J. Struct. Eng. 143 (3) (2017), 04016188, https://doi.org/10.1061/(ASCE)ST.1943-541X.0001672.

[89] K. Rus, V. Kilar, D. Koren, Resilience assessment of complex urban systems to natural disasters: a new literature review, Int. j. Disaster Risk Reduction 31 (2018) 311–330, https://doi.org/10.1016/j.ijarr.2018.05.015.

[90] I.M. Qeshta, M.J. Hashemi, R. Gravina, S. Setunge, Review of resilience assessment of coastal bridges to extreme wave-induced loads, Eng. Struct. 185 (2019) 332–352, https://doi.org/10.1016/j.engstruct.2019.01.101.

[91] R. Cantelmi, G. Di Gravio, R. Patriarca, Reviewing qualitative research approaches in the context of critical infrastructure resilience, Environ. Syst. Decis. 41 (3) (2021) 341–376.

[92] R. Atkinson, L. Crawford, S. Ward, Fundamental uncertainties in projects and the scope of project management, Int. J. Project Manage. 24 (8) (2006) 687–698, https://doi.org/10.1016/j.ijroman.2006.09.011.

[93] S. Tillement, C. Cholez, T. Reverdy, Assessing organizational resilience: an interactionist approach, Management 12 (4) (2009) 230–264.

[94] X. Zhang, E. Miller-Hooks, K. Denny, Assessing the role of network topology in transportation network resilience, J. Transp. Geogr. 46 (2015) 35–45, https://doi.org/10.1016/j.jtrangeo.2015.05.006.

[95] C. Koopmans, N. Mouter, Cost-benefit analysis. Advances in Transport Policy and Planning, Academic Press, 2020, pp. 1–42, https://doi.org/10.1016/bs.atpp.2020.07.005. Vol. 6.

[96] G. Munda, Social multi-criteria evaluation for urban sustainability policies, Land Use Policy 23 (1) (2006) 86–94, https://doi.org/10.1016/j.landusepol.2004.08.012.

[97] D. Diakoulaki, F. Karangelis, Multi-criteria decision analysis and cost-benefit analysis of alternative scenarios for the power generation sector in Greece, Renew. Sustain. Energy Rev. 11 (4) (2007) 716–727.

[98] Dodgson, J.S., Spackman, M., Pearman, A., & Phillips, L.D. (2009). Multi-criteria analysis: a manual. doi:10.1016/j.rser.2005.06.007.

[99] Mishan, E.J., & Quah, E. (2020). *Cost-benefit analysis*. Routledge. doi:10.4324/9781351029780.

[100] S.L. Proag, V. Proag, The cost benefit analysis of providing resilience, Proc. Econ. Financ. 18 (2014) 361–368, https://doi.org/10.1016/S2212-5671(14)00951-4.

[101] S.C. Gad, Drug Safety Evaluation, John Wiley & Sons, 2016, https://doi.org/10.1002/9781119097440.

[102] I. Linkov, J.M. Palma-Oliveira, An introduction to resilience for critical infrastructures. Resilience and Risk, Springer, Dordrecht, 2017, pp. 3–17, https://doi.org/10.1007/978-94-024-1123-2-1.

[103] I. Van Kamp, P.G. Van der Velden, R.K. Stellato, J. Roorda, J. Van Loon, R. J. Kleber, E. Lebret, Physical and mental health shortly after a disaster: first results from the Enschede firework disaster study, Eur. J. Public Health 16 (3) (2006) 252–258, https://doi.org/10.1093/eurpub/ck1188.

[104] J.R. Freedy, W.M. Simpson Jr, Disaster-related physical and mental health: a role for the family physician, Am. Fam. Physician 75 (6) (2007) 841–846.

[105] M.A. Cardoso, M.J. Telhado, M.D.C. Almeida, R.S. Brito, C. Pereira, J. Barreiro, M. Morais, Following a step by step development of a Resilience Action Plan, Sustainability 12 (21) (2020) 9017, https://doi.org/10.3390/su12219017.

[106] Gonzva, M. (2017). *Résilience des systèmes de transport guidé en milieu urbain: approche quantitative des perturbations et stratégies de gestion* (Doctoral dissertation, Paris Est).

[107] M. Gonzva, B. Barroca, Application of the "Behind the Barriers" resilience conceptual model to a flooded rail transport system, in: EGU General Assembly Conference Abstracts, 2017, p. 17893.

[108] P.L. Bescos, P. Dobler, C. Mendoza, G. Naulleau, V. Giraud, V.L. Anger, Contrôle De Gestion Et Management, Montchrestien, 1997.

[109] N.A.B. Mabahwi, O.L.H. Leh, D. Omar, Human health and wellbeing: human health effect of air pollution, Procedia-Soc. Behav. Sci. 153 (2014) 221–229, https://doi.org/10.1016/j.sbspro.2014.10.056.

[110] Direction régionale de l'Environnement, de l'Aménagement et du logement (DREAL), Pays de la Loire. (2014). RN844 Etudes du parti d'aménagement du périphérique Nantais - Dossier diagnostic-Rapport de synthèse.

[111] Dossier de concentration, Direction Régionale De l'Environnement, De L'aménagement Et Du Logement (DREAL), Aménagement du périphérique nord de Nantes, 2014. Pays de la Loire.

[112] Direction Interdépartementale des Routes Ouest (DIRO), Procédure Inondation Du Périphérique Est de Nantes, Rapport interne, 2015, p. 6.

[113] CORDIER B. (2019). *Le cout reel de la voiture en 2018*.

[114] A. Mebarki, N. Valencia, J.L. Salagnac, B. Barroca, Flood hazards and masonry constructions: a probabilistic framework for damage, risk and resilience at urban scale, Nat. Hazards Earth Syst. Sci. 12 (5) (2012) 1799–1809.

[115] L. Chen, E. Miller-Hooks, Resilience: an indicator of recovery capability in intermodal freight transport, Transpor. Sci. 46 (1) (2012) 109–123.

[116] S.E. Chang, Evaluating disaster mitigations: methodology for urban infrastructure systems, Nat. Hazard. Rev. 4 (4) (2003) 186–196.

[117] B.G. Hermann, C. Kroeze, W. Jawjit, Assessing environmental performance by combining life cycle assessment, multi-criteria analysis and environmental performance indicators, J. Clean. Prod. 15 (18) (2007) 1787–1796.

[118] P. Macoun, R. Prabhu, Guidelines For Applying Multi-Criteria Analysis to the Assessment of Criteria and Indicators (Vol. 9), CIFOR, 1999.