# A core multi-criteria framework for assessing the performance of policies related to risk: a case study on risk policy for high risk sites

Scarlett Tannous, Terje Aven, Myriam Merad

► To cite this version:

Scarlett Tannous, Terje Aven, Myriam Merad. A core multi-criteria framework for assessing the performance of policies related to risk: a case study on risk policy for high risk sites. Energy Strategy Reviews, 2025, 61, pp.101870. 10.1016/j.esr.2025.101870. hal-05252430

HAL Id: hal-05252430

https://hal.science/hal-05252430v1

Submitted on 12 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# A core multi-criteria framework for assessing the performance of policies related to risk: a case study on risk policy for high risk sites

Scarlett Tannous<sup>a, *</sup>, Terje Aven<sup>b</sup>, Myriam Merad<sup>a</sup>

## ARTICLE INFO

Handling Editor: Dr. Mark Howells

**Keywords:**

Risk management

Risk policy

Policy assessment

Industrial high-risk sites

Multi-criteria decision analysis

Risk governance

## ABSTRACT

Today most public authorities have implemented some type of risk governance framework or system, which provides structure, approaches, and methods for how to handle societal risks. One main challenge of risk-related policies and frameworks is to adequately and effectively take into account risks. To meet this challenge there is a need for knowledge about the performance of the various policies on risk. However, such knowledge is not easily derived since the performance is subject to uncertainties and is difficult to measure - especially before the occurrence of impacts, which can be significant when focusing on high-risk sites. The present paper discusses this issue of risk policy performance or “effectiveness”. The main aim is to establish a set of suitable criteria for assessing the performance of such risk policies by relying on both (1) foundational theoretical and methodological studies and (2) empirical and methodological case studies. Results provide new insights into risk policy assessment by relating the discussion to current risk science knowledge and multi-criteria decision analysis. Consequently, a novel multi-criteria framework consisting of 16 assessment criteria is proposed to cover the spectrum of various improvements, degradations, or stagnations of the effectiveness of risk policies. The discussion is illustrated by a case study on a risk policy linked to a high-risk site in the chemical and petrochemical industry. Based on 25 interviews with risk actors, the study findings reveal how perceived policy improvements or degradations vary under the proposed set of criteria including efficiency, efficacy, and reputational effects. The main contribution demonstrates the practicability of the proposed risk policy assessment framework consisting of 16 criteria. Further research can include the investigation of the Multiple-Criteria Decision Aiding (MCDA) techniques to support the application of this framework.

## 1. Introduction

A major contemporary issue for governments, public authorities, and policymakers is the question of how they can address the performance of their policies, and in particular policies on risk (‘risk policies’) which are the focus of the present paper. A risk policy can be explicit with respect to risk by, for example, stating that the precautionary principle shall be an integrated part of the governance frameworks adopted. It is common to distinguish between three main categories of basic risk policies: (i) risk-informed relying on risk assessment, (ii) cautionary/precautionary, which highlight robustness and resilience, and (iii) discursive [1–3]. A fourth category is often included: (iv) codes and standards; it is applied in situations characterized by very strong knowledge. In practice, a mixture of these policies is often preferred, balancing different concerns such as ensuring acceptable risk, risk reduction, and cost efficiency.

However, many risk policies relate to risk in a more indirect way. As an example, consider a public policy aiming at achieving increased production from some industrial plants. This policy could affect risk in different ways, including a possible higher accident risk with consequences for personnel, environment, and production. Assessing the performance of this policy obviously cannot be done by considering risks in isolation from the benefits that come from the planned increased production.

Assessing the performance of a risk policy is challenging, as the consequences of the policy are difficult to predict accurately (e.g., life loss, long-term environmental impacts, etc.). The performance typically relates to many dimensions (value types: life, health, environment, assets, etc.), and the uncertainties are commonly large. The assessment is especially challenging when the policy relates to rare events.

The scientific literature presents many approaches and methods for

<footnote><sup>a</sup> LAMSADE, Paris Dauphine University - PSL, 75016, Paris, France</footnote>

<footnote><sup>b</sup> Department of Safety, Economics and Planning, University of Stavanger, 4021, Stavanger, Norway</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail addresses: scarlett.tannous@dauphine.psl.eu (S. Tannous), terje.aven@uis.no (T. Aven), myriam.merad@lamsade.dauphine.fr (M. Merad).</footnote>
such assessments, including the use of cost-benefit analysis, cost-effectiveness analysis, multi-attribute/criteria analysis, and checklists [2,4–12]. Practically, CBA is one of the most commonly used methods for assessing risk policies. It is considered attractive as it provides a rationale for how to best use available resources in relation to the decision alternatives at hand. It contributes to ensuring efficiency, proportionality, and consistency in decision-making. CBA is well-established and, to a large extent, standardized with a high degree of traceability for the arguments used [2,4–8]. All costs and benefits are transformed into one common unit, in most cases, money, by applying concepts like the value of a statistical life (VSL). However, the CBA method also has some weaknesses and limitations, as discussed for example in Refs. [4,6,8]. These weaknesses and limitations have urged the need to either (i) couple CBA with other methods/inputs, in particular broader “qualitative” judgment of risks and uncertainties and stakeholder involvement, and/or (ii) use other methods (e.g., cost-effectiveness analysis, other MCDA methods, etc.) [2,7]. Three main challenges/problems related to CBAs include the following: (i) the conditions of use imposing the monetization of all costs and benefits; (ii) the reliance on expected values; and (iii) the choice of discount rate. For more literature review and details on the other methods, Appendix A provides a more detailed comprehensive overview of the policy assessment methods.

The originality of the present paper seeks to obtain new knowledge on the topic by rethinking the fundamental issue questioning the selection of important criteria to consider when assessing the performance of a risk policy. Many types of criteria have been suggested, ranging from efficacy and efficiency [4] to robustness and ethical considerations [13] to cover the multiple conditions that determine an effective policy based on foundational theoretical, methodological, and empirical case studies. We review and discuss these suggestions, with the purpose of developing an assessment framework and structure for risk policies which provides guidance on what criteria to consider for different types of risk policies. The added value of this paper relies on input from contemporary risk and policy science knowledge which are used to obtain new perspectives on fundamental principles relevant for developing this multi-criteria framework and guiding assessment practices for

risk policies in particular. The proposed framework is also empirically tested for the first time on an industrial risk policy through a collection of 25 interviews.

The paper is organized as follows. Section 2 follows the above introduction and provides some fundamentals on risk policies. Section 3, which is the main contribution of the paper, presents the announced assessment framework with criteria. Section 4 illustrates the framework on a case linked to the chemical and petrochemical field. Section 5 discusses the criteria and the case, and the final Section provides some conclusions. The appendix provides a brief review of common methods used to assess policies, including cost-benefit analyses.

## 2. Core concepts: risk policies and their input

The basic concepts of this study and their interrelationships are illustrated in Fig. 1 and explained as follows.

In general, governments aim to meet and balance public interests and improve the welfare of its citizens, by producing ‘coherent’ and ‘effective’ policies. For this, information about both the intended and unintended effects of the policies is needed. Analysts must provide relevant knowledge and evidence to support the choice of policies and the political decisions.

Public governance refers “the formal and informal arrangements that determine how public decisions are made and how public actions are carried out, from the perspective of maintaining a country’s constitutional values when facing changing problems and environments” [17]. Different types of public governance can be encountered such as performance-based, law-based, and consensus-based governance [18–20].

A policy can be defined as a set of ideas, principles, or plans for what to do in a particular situation to achieve desirable outcomes, and is typically expressed as an authoritative statement [4,14,21]. It relies on a defined understanding (hypothesis, theories, models, and methods) of the world. This is where science plays an important role. Policy analysis supports the development of policies. It assesses and compares the ‘performance’ of one or more alternatives (existing or potential) in relation to relevant objectives and [14,21–23]. Policy analysis combines

<figure><img src="image_3.png" /><figcaption>Fig. 1. A methodology for linking public governance, policies, and policy and risk analysis (based on [2,14] and OECD [15,16]).</figcaption></figure>
the objectives/values and the understanding (science) to establish the policy and the policy statement. Policy analysis aims at the creation, critical assessment, and communication of policy-relevant knowledge [4]. Science and knowledge-generating assessments are some main inputs to policy analysis. Policy makers use policy analysis, together with political input, to develop a suitable policy [24]. For instance, depending on the political orientations of the governing concerned authorities, the political input can be related to the need of ensuring efficient distribution of public expenditure and resources, maintaining political legitimacy towards public opinion by justifying public actions through numbers and analyses, sharing power across multiple levels of governments, restoring a reputation after major political, industrial, or other types of accidents, crises, or controversies, and several other reasons [25–28]. The focus of the assessment can address the policy capacity determined by the set of analytical, operational, and political skills and resources necessary to address perform the policy function to address the public problems whether at the individual, organizational, or systemic level as demonstrated in Refs. [27,29,30]. The methods used for the policy analysis depend on the context (e.g., institutional settings, administrative structure, etc.). Basic assessment tools include cost-benefit type of analysis, multi-attribute/criteria decision analysis and risk analysis (briefly listed in Appendix A).

Assessing means reporting on the "performance" of an "object" in terms of criteria/parameters/indicators/attributes that make sense to actors, stakeholders and decision-makers. By performance, we mean the way in which, on a given axis (ascending or descending), the object performs within a "system of preferences" that has been set. The idea is to "measure" or "assess" the consequences of "an object", in our case "public policies", is in practice and for decision-makers and stakeholders. To this end, we need to.

(a) define the object, in this case "public policy", but in a precise manner, since its scope of definition determines the framework for its evaluation and the criteria used.

(b) define what we mean by a "coherent family of criteria", which must meet the conditions of non-redundancy, consistency and exhaustiveness at the very least.

(c) characterize these criteria, their "scales" and their nature (e.g., qualitative, quantitative, etc.). the direction of preferences (increasing or decreasing).

(d) define the "problem" (e.g. description, sorting, ranking or selection).

(e) define the decision rule/aggregation procedure/even the integrating principle of multi-criteria and multi-actor evaluation.

(f) define the form in which the conclusions will be presented to the actors and the decision-makers [11,12,31].

This paper especially focuses on step (b) where the important identified gap calls for a coherent set of criteria in line with risk analysis and management for multi-criteria and multi-stakeholders assessment framework.

Risk analyses provide essential input to policy analysis and policy development, as discussed in Aven (2022). Risk analysis here refers to risk understanding, risk assessment, risk characterization, risk perception and communication, and risk handling (covering risk management, risk governance and policies related to risk) [3]. It is part of risk science which is the most updated and justified knowledge – on concepts, principles, models, theories, approaches and methods–related to these topics [2,3,14,32].

As discussed by Aven [14], a main element of policy analysis can be seen as a risk assessment, since what is of main interest is the future implications, effects and outcomes of the policy (i), and these implications, effects and outcomes are subject to uncertainties (ii). Following SRA (2018), these two features can define risk: the consequences C (implications, effects and outcomes) of the policy and the associated uncertainties U, for short denoted (C,U). Hence, it is clear that policy

analysis, in general, can strongly benefit from risk analysis and risk science knowledge. For example, they can benefit from insights about how people perceive risks, as well as guiding principles on how to understand, characterize, communicate, and handle risk. When it comes to explicit policies on risk, the point made here is that risk analysis and risk science can provide essential input also to the wider class of policies, where the link to risk is more indirect as in the production example in Section 1. Basic risk science knowledge is presented in Refs. [1,33]. For an overview of general risk handling principles referred to in standards and in the scientific literature, see Ref. [33]. The authors distinguish between guidance mainly founded on (a) generic management and governance knowledge, and (b) more specific risk analysis and risk science knowledge.

The eight principles referred to by the ISO 31000 standard on risk management (ISO 2018) are examples of the first category (a): Integrated, Structured and comprehensive, Customized, Inclusive, Dynamic, Best available information, Human and cultural factors, and Continual improvement. Here are some examples of other general management principles [34]: Put the right people in the right place with the right knowledge, incentives, and resources; clearly define leadership and responsibilities; share knowledge and experience across organizations. The statements relate to some of the eight ISO 31000 principles but clearly extend beyond these to cover aspects of organizational capacity, leadership and transfer of knowledge.

Other principles are highlighted by the general governance literature, covering aspects like openness, participation, accountability, effectiveness, coherence, proportionality, subsidiarity, legitimacy and voice, performance, accountability, fairness, rule of law, ethical conduct, competence and capacity, innovation and openness to change, human rights, and social cohesion [15,16,35]. Moreover, the public policy analysis literature commonly refers to criteria such as efficacy, efficiency, adequacy, equity, responsiveness, appropriateness, internal consistency, completeness, non-redundancy, and level of generality/specificity [4,9].

We can summarize these principles/criteria/attributes by a statement like: The policy and risk analysis is based on attributes derived from contemporary management and governance analysis and science, including those referred to above.

For the more specific risk analysis and risk science knowledge (b), some core principles are referred to in the literature in addition to those referred to above (i-iii) [2,33]. They include the following.

* The proper risk level is a result of a value and evidence/knowledge-informed process, balancing different concerns.

* This process of balancing different concerns can be supported by cost-benefit balancing methods, but this type of formal analyses needs to be supplemented with broader judgements of risk and uncertainties, as well as stakeholder involvement processes.

* When faced with uncertainties, weight needs to be given to the cautionary/precautionary approach.

* To protect values like human lives and health, and the environment, the associated risks must be judged to be sufficiently low.

* 'Decision makers review and judgements' are needed for making appropriate risk handling decisions. The decision makers review and judgment is defined as the process of summarizing, interpreting and deliberating over the results of risk assessments and other assessments, as well as of other relevant issues (not covered by the assessments), in order to make a decision.

In the following section we propose a list of criteria to be used to evaluate the performance of a risk policy.

### 3. Suggested framework of criteria for assessing the performance of risk policies

This section presents a framework of criteria for assessing the
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Five research corpora chosen to enrich the list of criteria to assess the performance of high-risk public policy under a conceptual framework.</figcaption></figure>

performance of risk policies (Table 2 and Fig. 3). A criterion represents a specific condition of a certain principle that should be considered when assessing the performance of a risk policy.

These criteria have been established by using two sources of information illustrated in Fig. 2 and explained as follows.

1. Foundational theoretical and methodological studies: the assessment criteria within a public policy context have been thoroughly discussed and debated by scholars throughout the years [1,4,7,12,30, 16,36–51]. However, to date, to the knowledge of the authors, there is no customized list of criteria based on which an industrial risk policy assessment can rely. Instead, there are some general lists of criteria that are continually enriched by research and practice; they are often revised and adjusted according to the purposes of their use. Some methodological initiatives are developed at a national scale, but they do not offer a generic exhaustive list from which assessors can choose the relevant assessment criteria for their case study. Additionally, some selected foundational research works are distributed among different areas focusing on public policy assessment, public governance, risk governance, risk policy, and evaluation criteria in general (Fig. 2). This means that the scope of the criteria is determined by the area in which the performance is being defined such as the public governance, risk governance, and so on.

2. Empirical and methodological case studies where certain risk policies are assessed: this is a bottom-up approach where criteria emerge from the assessment practices that are often driven by the purpose of the assessment, the ease of use of certain criteria, and the information availability to consider certain criteria (e.g., economic efficiency) rather than the others (e.g., flexibility and coherence). Thirty case studies were selected after several search query iterations using policy-related keywords [52–75], p. 2, [76,76–79], p. 20, [80]. This helped in gathering the commonly used assessment criteria when assessing risk policy and going beyond the common CBA for their analysis.

The selection process of the above studies relies on a thorough systematic literature review using two databases (Scopus and Web of Science) with a tailored screening process as summarized in Table 1 and thoroughly explained in Appendix B. Based on all of the above selected studies, no resulting criterion was left out.

To assess a risk policy means considering its short-term and long-term impacts as well as its potential positive and negative impacts. The impacts could relate to, for example, human life and health, environmental damage, and economic values. They could be formulated in absolute terms or considered relative to some objectives/goals/expectations. These impacts are, in general, uncertain so predictions and estimates are required. This is conducted by using metrics, such as expected values, and assessing the impact uncertainties. The knowledge supporting these judgments needs to be stated. In particular, key

assumptions made need to be explicitly stated to assess the extent to which the policy meets goals and expectations with respect to safety/ security/risk. The following list of criteria should be considered (Table 2 summarizes the main points for each criteria illustrated in Fig. 3).

**Efficacy:** The degree to which the risk policy meets its principal objectives/goals/expectations that have been set/prescribed. To determine the efficacy, impact assessment is seen in relation to the stated objectives/goals/expectations of the policy. For a policy to have high efficacy, the risks of not meeting these principal objectives/goals/expectations need to be small. A high efficacy can be considered when the judged likelihood for not meeting the principal objectives/goals/expectations is small and the knowledge supporting this judgment is relatively strong (resulting in a high confidence in meeting the principal objectives/goals/expectations).

**Efficiency:** The level to which the risk policy meets its principal objectives/goals/expectations relative to the resources (including time) used. Efficiency is as efficacy, except that the resources used are also taken into account. A policy has a high level of efficiency if the risk policy meets its principal objectives/goals/expectations with high confidence, using relatively few resources. An example could be a high confidence for a high ratio of improved safety per cost unit for a risk reducing policy.

**Balancing different considerations/concerns:** The extent to which the policy will balance different considerations/concerns. For example, if a policy is focusing on meeting safety risk goals without consideration of cost, the score for this criterion would be poor or low.

**Policy ancillary effects:** The extent of indirect and unintended consequences of the risk policy, i.e., the risk related to such consequences. Policies are developed to reach certain ultimate outcomes (e.g., reduce the number of major accidents, minimize the number of fatalities and morbidities, etc.). However, in real life, policies are not inert; policies have their own force of action, which can sometimes result in unexpected effects that go beyond the set of primary effects needed to solve a problem [28]. Others refer to these effects as external side effects.

**Policy robustness, generality, adaptivity, and flexibility:** The extent to which a risk policy works in different settings/situations and can adapt to developments and changes over time (e.g., emerging needs, changing contexts, and ongoing feedback). A policy scores high on this criterion if, for example, deviations from some key assumptions, on which the policy is built, are not compromising the main goals of the policy. Think about a policy focused on reducing accidents in an industry by improving the risk management frameworks adopted, and suppose the industry is undergoing some major challenges due to market developments. If the policy is still able to meet its objective, it would score high on this criterion.

**Accountability, responsibility, leadership, and contestability in risk policies:** The extent to which a risk policy makes clear who is responsible for the policy's actions, decisions, and impacts related to the development and implementation of the risk policy. This includes the
<figure><img src="image_5.png" /><figcaption>Fig. 3. Criteria to be used to assess the performance of risk policies.</figcaption></figure>

<table><caption>**Table 1**

Detailed search queries and screening steps for both Scopus and Web of Science (WoS).</caption><thead><tr><th>Database</th><th>Scopus</th><th>Web of Science</th></tr></thead><tbody><tr><td>Query</td><td>TITLE (polic* OR "public polic*" OR regulat*) AND (assessment OR evaluation OR analysis OR performance OR appraisal OR examination)) AND (risk OR safety) AND TITLE-ABS-KEY (criteria OR indicator*)</td><td>TI=(polic* OR "public polic*" OR regulat*) AND TI=(assessment OR evaluation OR analysis OR performance OR appraisal OR examination) AND TI=(risk OR safety AND ALL=(criteria OR indicator* OR principle))</td></tr><tr><td>Total number of publications</td><td>456</td><td>523</td></tr><tr><td>Number of publications after title screening</td><td>236</td><td>173</td></tr><tr><td>Number of publications after abstract screening</td><td>113</td><td>60</td></tr><tr><td>Number of publications after paper screening</td><td>15</td><td>20</td></tr><tr><td>Total after removing duplications (4 duplicates)</td><td>35-5 = 30</td><td></td></tr></tbody></table>

roles of Courts, operating companies, frontline workers, leaders, regulators, etc. Evidence availability allows contestability and accountability.

**The Rule of Law or legal implementation of a risk policy:** The extent to which the policy meets laws, regulations, political expectations, etc.

**Ethical considerations:** The extent to which the policy aligns with ethical principles and values/considerations (e.g., justice, equality, promote equal opportunities, reduce disparities, address systemic inequalities for actors and stakeholders etc.).

**Policy openness and transparency:** The extent to which the policy is developed and implemented through open and transparent processes (e.g., clear, accessible, participatory manner).

**Actors' dialogue, cooperation, and participation** (including public participation, interest, acceptance, responsiveness, and inclusiveness): The extent to which the risk policy ensures and stimulates dialogue/discourse, cooperation, and participation in the policy-making process.

**Reputational effects:** The extent to which the risk policy has reputational effects for relevant actors/stakeholders. A company can experience a strong negative reputation following an accident which can result in the emergence of several environmental associations to represent citizens' concerns and needs. Once the company is aware of this local sensitivity, the protection of its reputation can be one of the key drivers enforcing compliance with regulations, efforts towards transparency, and dialogue with regulators. Ignoring uncertainty by using unwarranted scientific precisions can result in far-reaching effects, conflicting power relationships, influences on the quality of the science-based regulatory debates, and public distrust in the science for policy [65].

**Communicability:** The extent to which the policy can be easily communicated. Key factors are related to the terminology used, the clarity in language, difficult logic, etc.

**Policy adequacy:** The degree to which the risk policy's scope of application and artifacts (e.g., tools, governance, market-based, persuasive, and coercive measures) are suitable to actors and territories and relevant to addressing a particular problem or issue. Take for example "criticality grids" in some risk assessment studies or the "spatial object" regulated by urban planning tools. They can be considered inconsistent with respect to the application decrees that set out their contours as they focus on classified facilities rather than risk basins.

**Policy rigorousness/solidness:** The level to which the risk policy is based on a solid basis, including risk science knowledge. The more justified basis, the higher the score.

**Functionality/operationality:** The extent to which the policy will work in operation/practice in terms of resources, costs, expertise, supervision, and so on. This includes implementation risks in the short-term and long-term.

To assess the performance of a risk policy, these criteria are considered, depending on their degree of relevance for the risk policy assessment case study. An example of how the assessment is conducted will be illustrated in the coming section where we will test the framework on a specific major risk policy in the chemical and petrochemical sectors.

## 4. A case study: public policy on industrial risks at high-risk sites (Seveso upper tier)

As an example, let us consider the French public policies especially focused on high-risk industrial sites within the chemical and petrochemical sector. In France, the Environmental Code is one of the legal texts that ensure the management and prevention of risks (more details in Refs. [25,31,81,82]). In particular, the industrial risk prevention approach supported by these policies relies on four main pillars: (i) risk reduction at the source by ensuring a defense-in-depth approach; (ii) urbanization management and vulnerability reduction by avoiding the increase, or reducing the population density, when possible, around industrial sites presenting major industrial risks; (iii) emergency planning since the probability of an accident occurrence is never zero; and (iv) information and dialogue about risk prevention, possible accidents, and ways to react in case of its occurrence [83].

These pillars rely on various actors to drive the organizational “behaviors” in order to prevent major industrial risks and reach national objectives. Operating companies have the main preventive role. Public authorities including the facilities' inspection body, known as DREAL (regional directorate for environment, planning, and housing), plays a key role in the preventive approach as well. Some central risk policy tools include (i) the technological risk prevention plan (PPRT); (ii) safety reports including hazard studies (EdD) and annexed studies; and (iii) emergency plans including external (PPI) and internal emergency plans
<table><caption>**Table 2**
Criteria to be used to assess the performance of risk policies.</caption><thead><tr><td>Criteria</td><td>Definition</td></tr></thead><tbody><tr><td>Efficacy</td><td>The degree to which the risk policy meets its principal objectives/goals/expectations that have been set/prescribed.</td></tr><tr><td>Efficiency</td><td>The level to which the risk policy meets its principal objectives/goals/expectations relative to the resources (including time) used.</td></tr><tr><td>Safety/security/risk</td><td>The extent to which the policy meets goals and expectations with respect to safety/security/risk</td></tr><tr><td>Balancing different considerations/concerns</td><td>The extent to which the policy balances different considerations/concerns.</td></tr><tr><td>External side or ancillary effects</td><td>The extent to which the policy considers indirect and unintended consequences of the risk policy, i.e., the risk related to such consequences.</td></tr><tr><td>Robustness, generality, adaptivity, and flexibility</td><td>The extent to which a risk policy works in different settings/situations and can adapt to developments and changes over time (e.g., emerging needs, changing contexts, and ongoing feedback).</td></tr><tr><td>Accountability, responsibility, leadership, and contestability</td><td>The extent to which a risk policy makes clear who is responsible for the policy's actions, decisions, and impacts related to the development and implementation of the risk policy.</td></tr><tr><td>Rule of Law</td><td>The extent to which the policy meets laws, regulations, etc.</td></tr><tr><td>Ethical considerations</td><td>The extent to which the policy aligns with ethical principles and values/considerations (e.g., justice, equality, promote equal opportunities, reduce disparities, address systemic inequalities for actors and stakeholders, etc.).</td></tr><tr><td>Openness and transparency</td><td>The extent to which the policy is developed and implemented through open and transparent processes (e.g., clear, accessible, participatory manner).</td></tr><tr><td>Dialogue, cooperation, and participation</td><td>The extent to which the risk policy ensures and stimulates dialogue/discourse, cooperation, and participation in the policy-making process.</td></tr><tr><td>Reputational effects</td><td>The extent to which the risk policy has reputational effects for relevant actors/stakeholders.</td></tr><tr><td>Communicability</td><td>The extent to which the policy can be easily communicated to different actors.</td></tr><tr><td>Adequacy</td><td>The degree to which the risk policy's scope of application and artifacts (e.g., tools, governance, market-based, persuasive, and coercive measures) are suitable to actors and territories and relevant to addressing a particular problem or issue.</td></tr><tr><td>Rigorousness/solidness</td><td>The level to which the risk policy is based on a solid basis, including risk science knowledge.</td></tr><tr><td>Functionality/operationality</td><td>To extent to which the policy can work in operation/practice in terms of resources, costs, expertise, supervision, etc. in the short and long-term.</td></tr></tbody></table>

(POI).

We mobilized the framework proposed in section 3 to assess the French risk policy for high-risk sites including the pre-mentioned tools and actors. 25 interviewees in Normandy, a Northwestern administrative region that is known for its industrial Seveso establishments' dense presence, shared their perceptions regarding the risk policy performance. This region is marked by the recent 2019 Lubrizol Normandie Logistique (LNL)<fnref n="1" /> industrial fire that has attracted media attention and public scrutiny, allowing us to look into how the different policy tools were implemented and mobilized by actors within such circumstances.

These 25 interviewees<fnref n="2" /> were recruited<fnref n="3" /> through purposive sampling focused on the Normandy region including some of its departments (e.g., Seine-Maritime and Eure) and industrial basins (e.g., Le Havre and Rouen). The geographical scope of this study is at the regional level and is especially focused on the regional actors. 17 out of the 25 interviewees have more than ten years' experience in the role for which they are being interviewed. This helped in addressing the effectiveness of the risk policy system in achieving its functions through their observations throughout the years,<fnref n="4" /> including before and after the LNL accident.

Some of the results are listed down in Table 3 highlighting the main evolutions knowing that the full list includes more than 14 perceived improvements after LNL marked by the sign (✓), 10 perceived

degradations marked by the sign (\), and no stagnations marked by (-).

It can be noticed that the collected data sometimes shows different views on what makes an effective industrial risk policy and what criteria determine its effectiveness within a specific territory by contributing to its enhancement (✓), deterioration (\), or stagnation (-) over the years and across territories.<fnref n="5" />

For example, improvements especially concern the rule of law, efficacy, transparency, and participation criteria. Degradations mainly concern efficiency and communicability. Also, the flexibility of tools and the openness appear debatable. Among the perceived evolutions explained earlier, some were considered as main evolutions as they were shared by several interviewees from operating companies, control authorities, associations, and so on. This includes the following.

- The number of legal texts appears to be increasing as emphasized by several interviewees. It seems to be coupled with an increasing complexity related to (i) exemptions of certain sites, aspects, conditions, and so on, depending on certain site-based specificities that are translated in the regulation and guides; and (ii) long guides to understand how the implementation must be done. This reflected some degradations in communicability \, efficacy \, and external side or ancillary effects \_.

- It seems that there is a stagnant and even lower number of inspectors in the Normandy region despite the Ministerial orders to increase the number of inspections. This is coupled with an increasing number of new facilities to control and wider types of facilities under the inspectors' scope. This appeared to degrade the efficiency \_.

- A public awareness of the industrial risks appears to be translated as a higher participation in committees within the period of the accident (dialogue, cooperation, and participation ✓ and reputational effects ✓). However, it seems to be difficult to maintain public interest in these committees throughout the years.

- It seems that dialogue and cooperation are fostered among the actors through the PPRT elaboration, site monitoring committees (CSS), and the necessity to publish inspection reports publicly. This was

<footnote n="1">¹ More on this accident and its regulatory reforms can be found in regulatory reforms [85,133-137].</footnote>

<footnote n="2">² In total, 60 individuals and entities were contacted, whether by phone, LinkedIn, contact form, email, or in person. Inclusion criteria required individuals to be actors in the industrial risk governance who are especially involved in elaborating and implementing risk policies within the Normandy region. The response rate is of 41.67 %.</footnote>

<footnote n="3">³ Potential interviewees were identified through online channels and professional recommendations. The online recruitment was conducted via websites' contact forms and their personal LinkedIn page. Some interviewees were also identified through professional networking events (e.g., conferences and research days on risk and safety-related topics). Others were recruited throughout the process based on recommendations of former interviewees.</footnote>

<footnote n="4">⁴ The understanding of the dynamics of complex system's elements, i.e., their behavior over time, can reveal a good deal about the behavior of complex systems [138].</footnote>

<footnote n="5">⁵ A platform that is used a single digital window for the environmental management.</footnote>
<caption>**Table 3**</caption>

The main perceived improvements (marked by ↗), degradations (marked by ↘), and stagnations (marked by -) post-LNL according to 25 interviewees.

<table><thead><tr><th>Criteria</th><th>Interview Summaries</th><th>Int. #</th></tr></thead><tbody><tr><td>Efficacy ↘</td><td>After LNL, many legal texts and ministerial decrees have been modified to make regulations stricter theoretically (e.g., modification of flammable liquids, modification of the texts on warehouses). Therefore, rather than having simple texts applied to any warehouse, regardless of the warehouse type, contents, size, etc., exceptions seemed to be given to specific cases that resulted in some lengthy 180-page guides. This appeared to decrease the efficacy of certain texts whether in implementation or monitoring.</td><td>14 and 20</td></tr><tr><td>Flexibility ↗</td><td>Whenever there was an incident, new layers of regulations seemed to be added to the regulatory framework, which hardened and complexified the intervention of the DREAL. Therefore, some of the questions that were raised included the following: do these changes finally bring anything to the system? Does all of this ultimately lead to a safer industry and a better-prepared population? This reflected a higher possibility and flexibility in adjusting policies after some accidents' lessons learnt.</td><td></td></tr><tr><td>Efficiency ↗ ↘</td><td>An increasing number of inspections was required by the ordering entities. However, difficulties seem to be encountered in increasing the number of inspectors in Normandy. Also, more than a 50% increase in inspections was requested, but it seems that staff increases have not been implemented. It appears that an increase was pushed lately to recruit three additional inspectors in Normandy, knowing that this is not considered enough either. When inspectors were asked about this issue, they advised us to check with the risk prevention general directorate (DGPR) to know their perspectives on this matter. As per Senate and Deputies' recommendations, the Ministry's declarations aimed at strengthening the role of inspectors and increasing their number at the national level. It was noted that the increase has been very small (i.e., 50 vacancies over three years while the 20 already recruited have not compensated those who retired/left). Consequently, there are fewer inspectors, even before LNL, when it was supposed to have more inspectors, so means must be provided to accompany a policy to guarantee safety.</td><td>22, 14, and 17</td></tr><tr><td>Balancing different considerations ↗</td><td>An action plan was elaborated to target warehouses. For this, several inspectors were dedicated to monitoring those warehouses over a given period to communicate feedback on the controls afterward. This reflects the enhanced capacity to balance between different priorities and objectives to prevent risks whenever it is detected.</td><td>22</td></tr><tr><td>External side or ancillary effects ↗</td><td>After LNL, the publication of the inspection reports to the public became mandatory. Consequently, as a side effect, DREALs had to talk to each other across different departments and sections more, which was perceived as an enhancement in terms of dialogue and cooperation.</td><td>30</td></tr><tr><td>Dialogue ↗</td><td>Also, for the past two to three years, it was noticed that there are more and more ways to report things from the DREAL to the upper management referring to the DGPR (e.g., forum of inspectors, online meetings with the DGPR several times per year, etc.), whereas, before that, in certain units, teams were working relatively alone and doing their reports as they wanted. This is to refer to variable reporting practices in the past.</td><td></td></tr><tr><td>Flexibility ↘</td><td>After LNL, some elected officials have asked for revising the risk prevention plans (PPRT) even though PPRTs are not expected to be revised as per the regulatory framework. Once validated, they do not get modified. This lack of flexibility to adjust certain tools to be compatible with the site real-life modifications was perceived as a degradation.</td><td>19</td></tr><tr><td>External side or ancillary effects ↘</td><td>After LNL, some operating companies felt that DREAL had increased the documentary and factual requests. This was perceived as an increasing willingness to check everything and to have proof of everything that had been done by operating companies as was felt by some Seveso UT interviewees.</td><td>18 and 19</td></tr><tr><td>Accountability ↗</td><td>This was also translated by another interviewee as follows: More formal notices seem to be noticed and applied after LNL. The interviewee thinks that DREAL has "tightened the screw" a little bit. Those that were subject to simple instructions before LNL seem to be now subject to formal notice.</td><td></td></tr><tr><td>Rule of Law ↗</td><td>This reflects an increasing reliance on rules enforcement and ensuring that accountability is served whenever efficacy is questioned. Before LNL, there was a forgotten technical culture related to the public. Now, after LNL, more attention seems to be given to the informational documents intended to be shared with the public [as a resident of the city of Rouen].</td><td>18 and 19</td></tr><tr><td>Efficacy ↗</td><td>However, at the same time, another interviewee highlighted that the time availability of the DREALs for consultations and exchanges with the public appeared to be increasingly limited as they seemed to refocus on more regulatory activities. Therefore, the openness to the public and the public participation appeared to be variable and dependent on certain accidents' occurrence.</td><td></td></tr><tr><td>Openness ↗ ↘</td><td>LNL seemed to have played the role of catalyst for creating the industrial association Upside in Rouen. The initiative seemed to exist before, but the idea was struggling to see the light. France Chimie (including several dozen adherents in the field of chemistry) seemed to have pushed the creation of Upside in the same way Synerzip-lh was created to foster cooperation. Upside has industrial members who are sometimes chemists but also grain stockpilers, waste caterers, et cetera. Associations were often perceived as effective cooperative nodes by the different groups of actors (public and private).</td><td>31, 13, and 19</td></tr><tr><td>Actors' Participation ↗</td><td>After LNL, it seems that there was a real awareness of the industrial risks to the population. The accident appeared to have left some traumatic impact: people kept in mind that on September 26, 2019, there was this accident, which seemed to affect their cautious attitudes towards the presence of industrial sites in the area.</td><td>26</td></tr><tr><td>Cooperation ↗</td><td>After LNL, the administration needed to react fairly quickly, so the resulting text seemed to not be accompanied by the necessary clarification on certain points. In the opinion of some interviewees, including inspectors and safety managers, there are some points in the new texts that are perhaps a little too vague. They perceived that as a degrading sign: behind a text misunderstanding, no one wins, so the public interest is compromised; this is because an unclear legal text can mean that things will not be done as expected and meant in the prescriptions.</td><td>18</td></tr><tr><td>Reputational effects ↗</td><td>After LNL, an important issue, that appears to be tackled, was the inspection of sites surrounding Seveso UT within a 100-m radius. This was perceived as an essential and positive improvement. Thus, the neighbors of a Seveso Upper Tier site should not be considered simply as potential victims, so they have to be inspected and monitored.</td><td>20 and 27</td></tr><tr><td>Communicability ↘</td><td>After LNL, the administration needed to react fairly quickly, so the resulting text seemed to not be accompanied by the necessary clarification on certain points. In the opinion of some interviewees, including inspectors and safety managers, there are some points in the new texts that are perhaps a little too vague. They perceived that as a degrading sign: behind a text misunderstanding, no one wins, so the public interest is compromised; this is because an unclear legal text can mean that things will not be done as expected and meant in the prescriptions.</td><td>18</td></tr><tr><td>Rigorousness/solidness ↗</td><td>After LNL, an important issue, that appears to be tackled, was the inspection of sites surrounding Seveso UT within a 100-m radius. This was perceived as an essential and positive improvement. Thus, the neighbors of a Seveso Upper Tier site should not be considered simply as potential victims, so they have to be inspected and monitored.</td><td>20 and 27</td></tr><tr><td>Flexibility ↗</td><td>Since LNL, the public authorities say that things were supposed to be simplified through some tools, but daily operations of DREAL seemed to not have become simpler. This was explained by, for instance, integrating the new GUNenv tool and simplifying the licensing permits processes, which resulted in operational difficulties. Therefore, these appeared as degradations influencing the monitoring of efficiency and operationality.</td><td>30</td></tr><tr><td>Functionality/Operationality ↘</td><td></td><td></td></tr><tr><td>Adequacy/Consistency ↘</td><td></td><td></td></tr><tr><td>Efficiency ↘</td><td></td><td></td></tr></tbody></table>

further boosted by the accident (dialogue, cooperation, and participation ↗).

- The accident appeared to boost and speed up the establishment of the industrial associations, UPSIDE, to copy the successful governance model of Le Havre and foster cooperation among different OCs and actors (dialogue, cooperation, and participation ↗).

- There is a certain tendency to simplify procedures, tools, and tasks through the emergence of the simplified FIRE<fnref n="6" /> sheet for the departmental fire service (Sdis), the GUNenv platform for DREAL, and

<footnote n="6">⁶ A simplified intervention sheet.</footnote>
environmental authorization licenses for Seveso UT establishments (flexibility /, functionality/operationality \, adequacy/consistency \, and efficiency \).

## 5. Discussions and main findings

This paper proposes a framework comprising a set of criteria for assessing the performances of policies on risk. The focus is on industrial contexts, but the criteria considered are to a large extent general. The paper builds on earlier contributions on the topic, integrating policy-oriented work (e.g., Merad [31]) and risk science and governance work (e.g. Aven [15]).

The proposed framework takes the earlier versions [12,13,31] three steps further by (i) adding insights and criteria coming from public administration discipline and risk governance areas; (ii) rendering the application of the assessment framework more operational and practical for assessors. Assessors can hence tailor the assessment as per their assessment scopes and purposes; and (iii) integrating the main findings of the previous case studies conducted internationally on risk policies and regulations, especially focused on the industrial sector. This framework overcomes the limitations caused by choosing one assessment method rather than the other resulting in certain criteria being either excluded or prioritized, which can mislead assessors, especially the less experienced. For instance, the inclusion of certain criteria like the reputational effects, which can sometimes be left out of other assessment framework, was demonstrated as essential. Such criterion can drive the extent to which a risk policy is perceived effective, especially after several accident occurrences. For instance, the distrust and testimonies of negative judgment towards public decisions and public policies within the last crisis management related to LNL were investigated by Negre [84] and aligned with our case findings.

The case study, in Section 4, shows that the performance/effectiveness of a risk policy is a dynamic concept that is, by nature, involving multi-actors and multi-criteria. The framework appeared to be beneficial in (1) representing some of the developments perceived by different actors and (2) drawing conclusions on the trade-offs between criteria. For instance, certain policy tools can be considered effective for State services but lack transparency and openness for other stakeholders.

Results show that between the prescribed objectives and the practical implementation, the effectiveness of such risk policies can fluctuate. For instance, when the regulatory reforms were enacted after the Lubrizol – Normandie Logistique (LNL) accident, they were done with the intent to close the regulatory blind spots revealed by the LNL accident adequately. These new regulatory reforms concerned Seveso sites, storage of flammable and combustible liquids, and warehouses of combustible materials [85]. They align with part of the post-accident recommendations proposed by Sénat [86,87].

However, when these policies started to be implemented, our case study revealed that the LNL regulatory reforms were considered as quick actions despite being taken for legit and valid reasons. They seemed to compromise the robustness and operability of these reforms due to lack of practicality and clarity of certain texts. This appeared to affect their effective and efficient implementation. This appears as a common pattern as Ayres & Braithwaite [38] pointed out on responsiveness of regulation. It can be difficult to capture such dimensions conditioning the effectiveness of risk policies when applying the common assessment practices. For instance, the institutional assessment practices in France for this same region were covered by Tannous et al. [88] that explained that such practices appeared to fall short in covering the spectra of criteria proposed in this paper and hence not meet the actors' expectations. Therefore, we encourage the assessment institutions to consider our assessment framework for such risk policy assessment studies to cover and integrate the largest number of possible criteria perceived as important for an effective risk policy to be sustained.

Additionally, we demonstrated, in this paper, that the effectiveness of risk policies is not linear. It is a process determined by a series of

criteria starting with the transparency and adequacy of the policy, going through its flexibility to balance different concerns till its operability and practicability among others. The 16 criteria of Table 2 cover the spectrum of different criteria that appear to determine the effectiveness of risk policies judged by the 25 interviewees. For instance, one of the added value which turned out to be a drawback in the case of the LNL regulatory reform is its “robustness, generality, adaptivity, and flexibility” covering the capacity of a risk policy to adapt to changing circumstances/contexts and evolving challenges to maintain effectiveness over time. This flexibility appeared to compromise its efficacy and communicability as highlighted by some inspectors. This trade-off was further investigated in Ref. [89]. An added value of the assessment framework that we are proposing is that it covers the spectrum of various trade-offs, allowing the assessors to deepen its assessment depending on the criteria of interest for the assessment purposes. For instance, among the proposed criteria, two main criteria (i.e., ‘dialogue, cooperation, and participation’ and ‘adaptivity and flexibility’) were revealed by Tannous et al. [89,90] as cornerstone for effective risk policy in that same region, Normandy. Under these two criteria, a mapping of the main interactions between key actors revealed the importance of mediator actors to coordinate between private and public actors.

The list of assessment criteria of the assessment framework proposed in Table 2 can be challenged by the capacity of each criterion to be fully satisfactory on its own. However, the underlying rationality (technical, economic, legal, social, and substantive rationality) behind each criterion allows ad infinitum iterations on such criteria.

A risk policy needs considerations of several if not all criteria of Table 2 to obtain a complete assessment basis. Some criteria are interrelated and to some extent overlapping. The unique combination of both foundational theoretical and methodological studies as well as empirical and methodological case studies, explained in Section 3, resulted in an assessment framework that is capable of covering the different criteria perceived as key criteria determining an effective risk policy. This was tested on a single case study in Section 4 but further work could focus on testing the framework on other risk policy cases by considering multi-regional and cross-country to enrich the validation of the proposed framework. In particular, this can enrich the criteria characterization and refine their boundaries for other case studies. Further testing could also lead to grouping criteria in a hierarchical form (e.g., using a decision tree) that could make it possible to help in highlighting the relevance of some criteria, and easing the higher-level interpretations of the issues as per the communicated trade-offs [13,91]. Further work can investigate in greater detail (1) the inter-relations between criteria and (2) the multiple criteria decision aids techniques that can be grouped under full-aggregation (e.g., Analytic Hierarchy Process (AHP)), outranking (e.g., ELECTRE), or goal-, aspiration-, and reference-level (e.g., TOPSIS) approaches [12,31,92,93]. For simplicity, a starting point to easily apply this framework would be by coupling it with a five-point Likert Scale. Another challenge that was faced by this case study is the relatively limited number of interviewees for the Normandy region case study. This was due to the sensitivity of the concerned region and sector. The interviews were conducted after approximately two years from the accident, which can be at the same time a unique opportunity to collect data on the before and after LNL accident but also a challenge to recruit a high number of interviewees. However, as many of the interviewees had more than 10 years of experience and interviews lasted around 1 h, the data saturation was reached at the nineteenth interview.

## 6. Conclusion

Just like the world’s problems, risk policy’s problem(s) are complex: multi-actors, multi-objectives, and multi-criteria – prone to media misinformation; efficiency cannot be the only assessment criteria to be considered when addressing a policy that can save lives, prevent
accidents, and maintain a vital sector. Public authorities and regulators are facing increased pressure and scrutiny to develop modern forms of risk policies and regulations that are more “effective” while also preserving certain cautiousness towards the public sector expenditure and private sector competitiveness – especially after COVID-19. Value trade-offs can be ubiquitous in such cases. For this, a thorough examination of the risk policy system must allow monitoring an effective decision-making reflecting the complexity of risk policies in real life [25].

This can be accomplished through policy assessment, a multidisciplinary complex process that necessitates a large amount of data, critical thinking, and policy arguments from various sources. The originality of this paper is that it offers a novel tailored assessment framework (Table 2 and Fig. 3) that encompasses trade-offs between different dimensions or what is so called in this study criteria that are considered important for a risk policy to be considered effective by various actors based on theoretical, methodological, and empirical studies. The resulting criteria include efficiency, flexibility, accountability, and efficacy to assess the effectiveness/performance of a risk policy.

This assessment framework is also tested at a regional level in France (Table 3): Normandy Region – a densely industrialized region in France [89,90]. Based on 25 interviews with industrial risk policy actors including inspectors, safety managers, and local actors in the Normandy region, this study revealed that improvements (✔) and degradations (✔) can be perceived differently under explicit criteria depending on the interviewees' scope and interests. After the Lubrizol-Normandy Logistic (LNL) accident that occurred in 2019 in that region, the regulatory reforms were perceived differently. Some considered them as an improvement by referring to the enhancement of their rule of law, efficacy, transparency, and participation criteria. Others pointed out the degradations related to the efficiency and communicability criteria. The flexibility of tools and the openness appeared debatable depending on the group of interviewees. This case study demonstrated the practicability of the proposed framework to cover the different spectra of improvements, degradations, or stagnations in the risk policy effectiveness

under 16 criteria.

Some limitations concerning the assessment framework and its application were also discussed. Future work can enrich and refine the criteria boundaries by investigating in greater detail the inter-relations between them. Therefore, we encourage testing this framework on various case studies on risk policy assessment by referring to the case study in Section 4 as an example. This can support the generalization and formalization of this assessment framework. Further research to support the application of this framework includes the investigation of the Multiple-Criteria Decision Aiding (MCDA) techniques.

### CRediT authorship contribution statement

**Scarlett Tannous:** Conceptualization, Methodology, Formal analysis, Investigation, Visualization, Writing – original draft. **Terje Aven:** Conceptualization, Methodology, Visualization, Supervision, Writing – original draft. **Myriam Merad:** Supervision, Resources, and Writing – review & editing.

### Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

### Acknowledgments

This work is part of doctoral research that was funded by LAMSADE – UMR CNRS 7243 – Paris-Dauphine University – PSL, to whom authors are grateful. Our thanks also go to the Institutt for sikkerhet, økonomi og planlegging (ISØP) at University of Stavanger where a two-month research stay nurtured this collaborative work. The case study would not have been possible without the valuable contributions and openness of the 25 interviewees in Normandy, France.

## Appendix A. One way to group the policy assessment methods and a brief review of the existing assessment methods

Policy assessment methods can be grouped under three main categories: exclusively monetary-based (e.g., CBA, cost-effectiveness analysis (CEA)), non-exclusively monetary-based (e.g., multi-attribute analysis, multi-criteria decision analysis (MCDA)), and some other methods (e.g., OECD Reference Checklist for regulatory decision-making [94]). It must be noted that they can be grouped differently as CBA and CEA both fall under the MCD&A groups of methods. For instance, one way to categorize was illustrated by Gasper [95] under checklists types. They can also be grouped under quantitative and qualitative approaches. More information on these methods is explained in the following subsections.

The application of any of these methods can differ depending on (a) the time when the assessment of the “effectiveness” of risk policies was conducted, (b) the aims of the assessment, and (c) the criteria that the assessments were founded on, such as efficiency, effectiveness, etc.

Concerning a), the assessment can be conducted at different stages of the policy cycle, for example, (i) being an ex-ante assessment, (ii) a mid-term assessment (i.e., within the implementation period), and (iii) an ex-post assessment. The focus of this thesis is especially on (i), i.e., approaches and methods that are used before policy implementation to support relevant decision-making on what policy to implement. In addition, the assessment can be of a thematic policy type addressing a single theme, e.g., a certain sector. A combination of at least (i) and (iii), if not all, is necessary to cover a full policy cycle.

Concerning b), the assessment aim can relate to (i) how the policies achieve their objectives with respect to efficiency and efficacy, (ii) justification of the government action to the public, (iii) adjustability of plans to what is practical to solve problems in real-life situations, (iv) transparency and accountability, especially after a major accident or a controversy, etc. Concerning c), the criteria are detailed in Section 3.

Ideally, randomized control trials (RCTs) must be applied to assess policies; this involves assigning target groups to experimental/policy intervention and control groups that do not receive the policy treatment. The difference between the two can determine attributes of policy effectiveness, but this cannot be applied to risk policies due to their inter-wined multi-actors and multi-objectives nature. They are also expensive and necessitate an important amount of data and research resources [96]. In this section, an overview of different assessment methods is provided to explain how a certain methodological choice can implicitly favor an assessment criterion rather than the other. By no means can this be considered a thorough review; it is more like a summary (for more details on the pros and cons of each method, refer to Dunn [97]).

### Cost-benefit analysis (CBA)

Jiang & Marggraf [98] traced back its origin to the work of Saint-Pierre in France in 1708 and explained its use in France and the US. CBA is one of the most commonly used methods for assessing risk policies. It is considered attractive as it provides a rationale for how to best use available resources in relation to the decision alternatives at hand. It contributes to ensuring efficiency, proportionality, and consistency in decision-making. The CBA is
well-established and, to a large extent, standardized with a high degree of traceability for the arguments used [97,99–103]. All costs and benefits are transformed into one common unit, in most cases, money, by applying concepts like the value of a statistical life (VSL).

However, the CBA method also has some weaknesses and limitations, as discussed for example in Ref. [97,100,102]. These weaknesses and limitations have urged the need to either (i) couple CBA with other methods/inputs, in particular broader “qualitative” judgment of risks and uncertainties and stakeholder involvement, and/or (ii) use other methods (e.g., cost-effectiveness analysis, other MCDA methods, etc.) [101,103]. Three main challenges/problems related to CBAs are: (i) the conditions of use imposing the monetization of all costs and benefits; (ii) the reliance on the expected values; and (iii) the choice of discount rate.

Concerning i) it is not meaningful to monetize all policy effects as not all of them can be easily monetized (i.e., transformed into one common monetary unit). This is related to the boundary conditions when applying CBAs. Different methods and indicators have been introduced to support the transformation, including the willingness to pay (WTP) approach and the use of concepts like the value of a statistical life (VSL) as an indicator to characterize an assessment criterion (M. W. [104]). There are, however, considerable theoretical and practical challenges related to the use and implementation of these principles and methods, as thoroughly discussed in, for example, M. Jones-Lee & Aven [105] and Weimer [100]. There is considerable literature arguing for the need to go beyond one single dimension to study and compare the effectiveness of different policy alternatives (e.g., Ref. [103,106,107]). Relevant aspects include ethical issues, distributional concerns, equity, individual preferences, individual freedoms, etc., which are often raised when implementing risk-related policies. This type of concern is not meant to be considered in the CBAs. These drawbacks are often related to the analysts’ understanding of methods, their scope of application, and the methodological choice of the analyst.

Concerning ii), the reliance on the expected value is problematic as it means that important aspects of risks and uncertainties are not reflected. For situations with considerable uncertainties or potential for extreme events, expected value-based risk metrics provide poor decision support [103,106, 108–112].

Concerning iii), the challenge is to determine the present value of future costs and benefits. Economists have developed theories for determining suitable discount rates in specific cases (e.g., Ref. [100,113,114]), but the issue is subject to considerable debate as it relates to not only financial considerations but difficult ethical and political questions such as how lives, environmental damage, etc. can be weighted in the future relative to today.

Due to these limitations, CBAs are not suitable as a prescription for the selection of a policy alternative that includes more than efficiency considerations. Rather, CBAs can be used to obtain insights by varying input quantities and parameters. The CBAs always need to be supplemented with separate risk and uncertainty analyses. With large uncertainties and/or potential for extreme events, CBAs are not very informative.

Other monetary methods exist, such as the input-output analysis (IO) method for inter-industry analysis to understand the complexities and inter-dependencies of the economy driven by the need to maintain a general supply-demand equilibrium. It is an economic modeling technique that allows analysts to look into economic tendencies between different sectors affecting demand and supply. For more information about IO's applications, pros, and cons, refer to European Commission [115]; United Nations [116]; Christ [117]; van Leeuwen et al. [118]; and Dietzenbacher et al. [119].

As reviewed in this section, exclusively monetary methods such as CBA and IO have both advantages and disadvantages. When applied to risk policies, it can provide information on economic efficiency (e.g., resource allocations, economic effects, etc.), transparency in trackability terms (e.g., by making the valuation of costs and benefits explicit through quantification), and adequacy (e.g., the consistency of resource allocation with respect to set objectives). This might give the illusion of controllability, preciseness, and arbitrable measurement. Nevertheless, their disadvantages are related to their limited scope of application, especially focused on economic impact. They neglect some equity concerns (e.g., by considering what can be monetized only) that can be related to disproportionate effects on certain groups. They also involve controversial concerns regarding policy sustainability when it comes to uncertainty treatment and life valuation. Thus, other methods are reviewed in the following sections.

## Cost-effectiveness analysis (CEA) and alternative monetary methods

Cost-effectiveness analyses (CEAs) represent an alternative to CBAs. The conventional CEA requires, in general, the monetization of the costs but not the benefits. Expected values are used for both costs and benefits. Typically, benefits express the gain obtained by the policy or measure considered, for example, reduced loss of lives in a safety context. These benefits are, in general, non-monetizable in market prices. An example of a CEA metric is the expected cost per expected saved lives, also referred to as the ICAF (Implied Cost of Averting a Fatality). It provides a link between cost and, at a certain point, safety objectives ([120], p. 514; [112], p. 239) and gives a simple way of comparing alternative policies, for example, related to risk-reducing measures. Like CBAs, CEAs are commonly used to verify ALARP<fnref n="7" /> and SFARP<fnref n="8" /> principles and the gross proportion between cost and benefits.

CEA is based on expected values, and hence, when applied to risk-related policies, its application suffers from the same problems in reflecting all relevant aspects of risk and uncertainty as the CBA. Therefore, it can be noticed that through this method, the effectiveness is limited to its efficacy in economic terms. The potential for extreme outcomes is not adequately reflected, nor is the strength of the knowledge supporting the judgments and assessments made. In practice, computed CEA metrics like the ICAF vary considerably for policies and measures that are implemented, demonstrating that there are many factors justifying a policy or measure that extend beyond the expected cost-effectiveness ratio used in CEA. As for CBAs, there are strong arguments for seeing beyond CEAs to study and compare the effectiveness of different policy alternatives to adequately take into account ethical issues, distributional concerns, equity, individual preferences, individual freedoms, etc.

## Non-monetary methods

The difficulty of monetizing all objectives, expenditures, and impacts of a policy has pushed the emergence of non-monetary methods to accommodate different assessment criteria, objectives, and approaches. This goes beyond the economic theories, practical engineering, and operational analysis from which the CBA and CEA have emerged. It rather relies on decision theories for the purpose of accommodating the socioeconomic, technical, environmental, and political value judgment related to risk-related and environment-related policy [121].

The Multi-Criteria Decision Analysis (MCDA) or multi-criteria policy assessment, provides a flexible way to assess the policy options based on

<footnote n="7">⁷ As low as reasonably practicable.</footnote>

<footnote n="8">⁸ So far as is reasonably practicable.</footnote>
different qualitative and quantitative criteria – or attributes of interest - that can go beyond the economic efficiency criterion [106,110,122]. Unlike CBA and CEA, MCDA is not limited to a one-dimensional monetary assessment criterion; it can, by nature, include a higher - to unlimited - number of quantitative and qualitative criteria such as legitimacy, equity, accountability, etc. and accommodate for the differences in the measurement levels of variables. Tools of multi-criteria decision analysis can help policymakers link values, science, values, and decision options in a more transparent and rigorous way [122]. It can thus better reflect the complexity of the real world by considering a large number of objectives, data, and relations, with all its involved fuzzy information, which are usually in conflict in real life.

MCDA methods, including its aggregation methods, can involve weighing systems for different criteria. Methods can involve actors to determine the relevant criteria to the problem at hand. Additionally, the weighing can stimulate debates concerning the importance of the various criteria. This serves the transparency and pluralistic dimensions by revealing systematic information on the divergent preferences related to the policy problem [121]. Several authors have compared CBA and MCDA using different criteria such as flexibility, transparency, simplicity, etc., see for example ([112], pp. 246–24; [106,110,121,123]).

A main challenge of the MCDA was related to the “problem structuring” phase of the policy, as there was no standard problem structuring method for MCDA [121,123]. Merad & Trump [110] scrutinized the framing bias related to problem structuring within a risk analysis context. The problem, framed by the analyst around the decision maker’s problem stated, is formulated in a formal language within the “problem structuring phase”. In other words, analysts’ perceptions and translation of the problem are reduced to its formal modeling. Based on this, some methodological choices, along with the tools and approaches, can prevail ([110], p. 65). The same policy problem can be structured differently, and this could affect the methodological choice. Also, there were some challenges at the aggregation phase, as there were no obvious and standardized methods for integrating and balancing the different criteria and variables [121,123]. Merad & Trump [110] referred to two ways to group the aggregation procedures. For instance, the aggregation process can adapt different logic, such as (i) compensatory; (ii) partially compensatory; or (iii) non-compensatory. It can also be grouped under: (i) single synthesis criterion; (ii) outranking synthesis; or (iii) local interactive judgment with tests-errors iterations (for more details, refer to Ref. [110,122,124]).

Besides MCDA, the other non-monetary methods are not numerous. Some methods are quantitative and rely on the expected utility theory, first discussed by Bernoulli, which considers that money is not an adequate measure of value. This was challenged empirically. Without scrutinizing the pros and cons of the expected utility-based method for policy analysis, it remains in the economic rationality ([125], p. 88; [126])

CBA and CEA can be seen as MCDA metrics: they are used to focus on specific criteria considered in the MCDA (e.g., economic efficiency). There is always a need to see beyond probabilities and expected values when making decisions in a risk context. Management review and judgment (MRJ) processes are required to take into account the limitations of the analyses and add concerns and issues not reflected in the formal analysis processes. An MRJ process will involve a certain loss of transparency, but this loss must be balanced against the need to see the results of the study in a larger context that considers all aspects of the decision-making process [110,112].

## Other methods

Additional methods, mainly qualitative and case study-based, are used to conduct risk policy analysis such as surveys, scorecards, indicators, interviews, etc. [127–129]. For instance, the case survey method collects qualitative evidence, allowing the review of various experiences of stakeholders and their frequencies in a policy study. It is challenged by aggregation issues (i.e., data compatibility, the scientific rigor of an individual case, etc.) [130,131]. Also, scorecards, like United Nations Office for Disaster Risk Reduction (UNDRR) scorecards, are another methodological initiative aiming to combine both quantitative and qualitative measures to deal with complex policy problems and policy assessment issues [132]. These qualitative methods help in determining the internal validity of a policy and have the flexibility to focus more on processes and actors’ behaviors. Yet, it is challenged when there is a need to go from the specificity of a case study to generalization in the risk policy assessment.

Without claiming any exhaustivity, this section reviews the main assessment methods categorized under exclusively monetary, non-exclusively monetary, and other methods. It can be noted that the “best” methods to address the effectiveness of risk policies are debatable and highly context-dependent, as found in the literature. They are anchored in different sciences, including social sciences, economic sciences, decision sciences, and risk sciences. With this diversity, two key points guide this chapter include (i) methods must not be seen as competitive or substitutional among each other but rather complementary, aiming to reinforce the assessment; and (ii) there is a need to be clear on what assessment criteria and principles must be considered when assessing risk policies in particular. To make these concerns explicit, this can be achieved by orienting the research efforts on the rigorousness of the assessment methods scope, revealing which assessment criteria they can cover.

# Appendix B. Methodology: systematic literature review

## 1. Research question

This appendix looks into the practices adopted by different scholars, under a multidisciplinary angle, to assess risk policies around the world. It aims to explore the following question: “how” can the effectiveness of a risk policy be assessed? More precisely, what are the criteria used to address the effectiveness of risk policies? In order to do so, international literature is examined to look into how the previous studies approached the assessment question characterizing an effective risk policy. Also, when a risk policy is considered effective, is it considered effective in the same way or differently to various actors? Are there some conventional international and national criteria used to assess risk policies? All of these questions guided the search query to target these gaps through a systematic literature review focused on public policies on risks with a preference to the studies addressing industrial risk policies or industrial safety policies (inter)nationally. The ultimate goal of this chapter is to develop a holistic and comprehensive assessment framework, especially tailored to assess risk policies. This framework can guide and help policymakers, public authorities, and involved actors in assessing the effectiveness of their risk policies while considering their complex trade-offs and allowing an explicit choice of their assessment criteria.

These criteria emerge from two sources of information as follows.

1. Foundational theoretical and methodological studies

2. Empirical and methodological case studies where certain risk policies are assessed
## 2. Keyword selection, search query, and database selection

By simply thoroughly reading the pre-liminary literature, five main points can be noticed that the words “analysis”, “assessment”, and “evaluation”/“valuation” were used interchangeably next to the word policy; this guided the search query to include all of these four words. Therefore, the choice of the keywords composition can be explained as follows.

a. **Policy-related:** polic* (for policy & policies), “public polic*”, “regulat*” (for regulatory & regulation). This excludes the word “program” to not alter the main focus on public policies.

b. **Goal-related:** assessment, evaluation, analysis, performance, appraisal, or examination. The words “analysis”, “assessment”, and “evaluation”/“valuation” were used interchangeably next to the word policy. The different variations of these words were excluded to result in a more relevant and manageable number of papers.

c. **Scope-related:** the scope of this thesis is related to industrial risk policies so both “risk” and “safety” are relevant keywords to cover the case studies that assessed any industrial risk- or safety-related studies.

It is recommended to have at least two databases for a systematic literature review (Hiebl, 2021). Therefore, acknowledging the interdisciplinary identity of the topic, the two chosen databases include Scopus and Web of Science.

## 3. Time scope

Some of the reasons to identify a certain starting date can be (i) the appearance of a phenomenon or (ii) the end date of the last systematic literature review (Hiebl, 2021). Based on this preliminary literature review, a 2012 study by Adelle & Weiland (2012) constituted a key starting point as it established the state of the art of policy assessment back then. Therefore, more than a decade later, this study looks into the changes, trends, practices, etc. since then and provides updates on this state-of-the-art as well as other added values as expected in the findings section. The time and the geographical scope were left open including all dates and case studies' locations.

## 4. Inclusion/Exclusion criteria and screening

All the inclusion and exclusion criteria recommended by Hiebl (2021) and considered as essential for transparent literature are briefly listed below.

a. Gray literature: gray literature was excluded from the systematic review for this chapter. Yet some supplementary studies (e.g., institutional studies, etc.) were included.

b. Time period: it covered all years.

c. Keywords in which field (title, abstract, text): keywords are detailed in Table 1.

d. Inclusion of supplementary/additional information: supplementary methodological and conceptual information was added from previous studies. 21 conceptual and methodological are added, contributing to the criteria characterization.

e. Based on the relationship with the research question, several studies were excluded such as the ones addressing (i) chemical analyses focusing on the chemical impacts; (ii) toxicological and biological experiments as well as clinical studies; (iii) the economic criteria only; (iv) monetary, economic, bankruptcy and insurance policy; (v) technical process/systems (e.g., mass spectrometry); (vi) corporate policies and not governmental or public policy which concern different tradeoffs; (v) financial security; (vi) unrelated case studies such as gender equality, immigration policy, police/policing, child protection, gun permits, cyber risks, police patrol, building regulations, genetically modified organisms (GMOs), and (vii) pharmacology-related studies.

As listed in Table 1, 456 studies were found on Scopus using the risk policy assessment-related keywords (i.e., TITLE (polic* OR "public polic*" OR regulat*) AND (assessment OR evaluation OR analysis OR performance OR appraisal OR examination) AND (risk OR safety) AND TITLE-ABS-KEY (criteria OR indicator*)). After screening the titles, 236 were selected by excluding the papers that did not focus on risk-related policies. After screening the abstract, 113 studies were selected by including the methodological, conceptual, and empirical risk policy assessment studies. Subsequently, after screening the papers, 15 studies were selected as relevant to the scope of our study by choosing the ones that focus on risk-related policies and provide either methodological contribution or empirical and conceptual aspects related to a specific risk policy assessment case study. Similarly, 523 studies were found on Web of Science (WoS) using the risk policy assessment-related keywords (i.e., TI=(polic* OR "public polic*" OR regulat*) AND TI=(assessment OR evaluation OR analysis OR performance OR appraisal OR examination) AND TI=(risk OR safety AND ALL=(criteria OR indicator* OR principle)). 173 and then 60 studies were selected after screening the titles and the abstracts, respectively. After scanning the content and excluding the inaccessible ones, 20 case studies were selected.

After this first round, a second round of an optimized search query was applied, leading to 412 papers, including all disciplines, without excluding any paper either by title or by content. They were examined through VOS viewer in August 2023, allowing the integration of the third-year research maturity in a more selective query and the recent studies focused on industrial risk policies or safety policies.
<table><caption>**Table 1**
Detailed search queries for both Scopus and WoS</caption><thead><tr><th>Database</th><th>Scopus</th><th>Web of Science</th></tr></thead><tbody><tr><td>Query</td><td>TITLE (polic* OR "public polic*" OR regulat*) AND (assessment OR evaluation OR analysis OR performance OR appraisal OR examination) AND (risk OR safety) AND TITLE-ABS-KEY (criteria OR indicator*)</td><td>TI=(polic* OR "public polic*" OR regulat*) AND TI=(assessment OR evaluation OR analysis OR performance OR appraisal OR examination) AND TI=(risk OR safety AND ALL=(criteria OR indicator* OR principle)</td></tr><tr><td>Total number of publications</td><td>456</td><td>523</td></tr><tr><td>Number of publications after title screening</td><td>236</td><td>173</td></tr><tr><td>Number of publications after abstract screening</td><td>113</td><td>60</td></tr><tr><td>Number of publications after paper screening</td><td>15</td><td>20</td></tr><tr><td>Total after removing duplications (4 duplicates)</td><td>35-5 = 30</td><td></td></tr></tbody></table>

In general, a reasonable, systematic literature review should have, after the title screening, around 100–300 publications (Hiebl, 2021). In this case, 409 articles in total were found after the title screening, which is not far from the accepted range. Besides the inclusion and exclusion criteria, the classification of references must be conducted by several authors (Hiebl, 2021) in a way to classify references as A (particularly relevant), B (potentially relevant), C (little to no relevance) but for practical reasons, this classification was conducted by one person and hence class A papers were included. Additionally, for a more precise classification, Nguyen et al. (2018) proposed a way to classify studies under dimensions of relevance depending on whether the added value concerns the theory, the methodology, the analysis, the relevance, and the contribution. In this study, the type of paper was listed to highlight the added value of each paper. The non-empirical content of the selected studies is of relevance as it aims to feed the conceptual thinking of this study.

In general, it is recommended to have at least 50 selected articles for a systematic review to be considered as a review article (Hiebl, 2021). A field should be mature enough to warrant a “state-of-the-art” literature review. In this study, a total of 51 papers were included. They encompass 30 case studies and 21 conceptual and methodological works.

## Data availability

Data will be made available on request.

## References

[1] O. Renn, "Risk Governance: Coping with Uncertainty in a Complex World," Routledge & CRC Press. Accessed: April. 28, 2022. [Online]. Available: http://www.routledge.com/Risk-Governance-Coping-with-Uncertainty-in-a-Complex-World/Renn/p/book/9781844072927.

[2] T. Aven, O. Renn, Improving government policy on risk: eight key principles, Reliab. Eng. Syst. Saf. 176 (Aug. 2018) 230–241, https://doi.org/10.1016/j.ress.2018.04.018.

[3] SRA, Society for risk analysis glossary. https://www.sra.org/wp-content/uploads/2020/04/SRA-Glossary-FINAL.pdf, 2018. (Accessed 10 October 2020).

[4] W.N. Dunn, Public Policy Analysis: an Integrated Approach, Routledge, 2017.

[5] D. Pearce, Cost benefit analysis and environmental policy, Oxf. Rev. Econ. Pol. 14 (4) (Dec. 1998) 84–100, https://doi.org/10.1093/oxrep/14.4.84.

[6] D.L. Weimer, Cost-benefit analysis and public policy, J. Pol. Anal. Manag. (2008) [Online]. Available: https://www.wiley.com/en-gb/Cost-Benefit+Analysis+and+Public+Policy-p-9781405190169. (Accessed 16 May 2022).

[7] House of Lords, "Government Policy on the Management of Risk: 5th Report of Session 2005-06 - Great Britain: Parliament: House of Lords: Select Committee on Economic Affairs - Google Books.".

[8] T. Aven, E.B. Abrahamsen, On the use of cost-benefit analysis in ALARP processes, Int. J. Perform. Eng. 3 (3) (Jul. 2007) 345, https://doi.org/10.23940/ijpe.07.3.p345.mag.

[9] D. Gasper, Policy evaluation - from managerialism and econocracy to a governance perspective. https://repub.eur.nl/pub/50685/, 2005. (Accessed 28 April 2021).

[10] OECD, The OECD reference checklist for regulatory decision-making. https://www.oecd.org/regreform/regulatory-policy/35220214.pdf, Sep. 09, 2005.

[11] I. Linkov, E. Moberg, Multi-Criteria Decision Analysis, 2012.

[12] M. Merad, B.D. Trump, Expertise under Scrutiny, Springer, 2020 [Online]. Available: https://link.springer.com/book/10.1007%2F978-3-030-20532-4.

[13] M. Merad, N. Dechy, L. Dehouck, M. Lassagne, Risques majeurs, incertitudes et décisions, in: Approche pluridisciplinaire et multisectorielle, 2016. MA Editions, https://www.decitre.fr/livres/risques-majeurs-incertitudes-et-decisions-9782822404303.html. (Accessed 9 August 2022).

[14] T. Aven, Cases of real-life policies related to risk: how can they enhance risk analysis and risk science? Risk Anal. (2022).

[15] T. Aven, O. Renn, Some foundational issues related to risk governance and different types of risks, J. Risk Res. 23 (9) (2019) 1121–1134, https://doi.org/10.1080/13669877.2019.1569099.

[16] European Commission, European governance A white paper, Text (2001) [Online]. Available: https://ec.europa.eu/commission/presscorner/detail/en/DOC_01_10. (Accessed 16 November 2022).

[17] United Nations, Public Governance Indicators: A Literature Review, 2007.

[18] A.H.F. Li, Centralisation of power in the Pursuit of law-based governance: legal reform in China under the Xi administration, China Perspect. 2016 (2) (Jun. 2016) 63–68, https://doi.org/10.4000/chinaperspectives.6995.

[19] M. Marra, The ambiguities of performance-based governance reforms in Italy: Reviving the fortunes of evaluation and performance measurement, Eval. Progr. Plann. 69 (C) (2018) 173–182.

[20] OECD, Saving Costs in Chemicals Management: How the OECD Ensures Benefits to Society, Organisation for Economic Co-operation and Development, Paris, 2019 [Online]. Available: https://www.oecd-ilibrary.org/environment/saving-costs-in-chemicals-management_9789264311718-en?itemId=/content/component/9789264311718-5-en&csp_866978e5296604c2b9d87607eddced67&itemIGO-oecd&itemContentType=chapter. (Accessed 10 November 2021).

[21] C. Althaus, P. Bridgman, G. Davis, The Australian Policy Handbook: a Practical Guide to the Policy-Making Process, sixth ed., Allen & Unwin, Sydney Melbourne Auckland London, 2018.

[22] M.G. Morgan, Theory and practice in policy analysis: including applications in science and technology, in: Theory and Practice in Policy Analysis: Including Applications in Science and Technology, 2017, p. 590, https://doi.org/10.1017/9781316882665.

[23] D.L. Weimer, A.R. Vining, Policy analysis: concepts and practice, in: Policy Analysis: Concepts and Practice, 2017, p. 480, https://doi.org/10.4324/9781315442129.

[24] M. Ullah, M. Umair, K. Sohag, O. Mariev, M.A. Khan, H.M. Sohail, The connection between disaggregate energy use and export sophistication: new insights from OECD with robust panel estimations, Energy 306 (Oct. 2024) 132282, https://doi.org/10.1016/j.energy.2024.132282.

[25] S. Tannous, An integrated framework to assess the 'effectiveness' of risk-related public policies for high-risk chemical and petrochemical sites: a comparative study in France and Australia, in: PhD Thesis, Université Paris sciences et lettres, 2023 [Online]. Available: https://www.theses.fr/2023UPSLD034. (Accessed 23 June 2024).

[26] P. Cairney, The politics of policy design, EURO Journal on Decision Processes 9 (Jan. 2021) 100002, https://doi.org/10.1016/j.ejdp.2021.100002.

[27] M. Howlett, M. Ramesh, X. Wu, Understanding the persistence of policy failures: the role of politics, governance and uncertainty, Publ. Pol. Adm. 30 (3–4) (Jul. 2015) 209–220, https://doi.org/10.1177/0952076715593139.

[28] P. Lascoumes, P. Le Gales, Introduction: understanding public policy through its instruments? From the nature of instruments to the Sociology of public policy Instrumentation, Governance 20 (1) (Jan. 2007) 1–21, https://doi.org/10.1111/j.1468-0491.2007.00342.x.
[29] X. Wu, M. Ramesh, M. Howlett, Policy capacity: a conceptual framework for understanding policy competences and capabilities, Policy and Society 34 (3-4) (Sep. 2015) 165-171, https://doi.org/10.1016/j.polsoc.2015.09.001.

[30] X. Wu, M. Howlett, M. Ramesh (Eds.), Policy Capacity and Governance: Assessing Governmental Competences and Capabilities in Theory and Practice, Springer International Publishing, Cham, 2018, https://doi.org/10.1007/978-3-319-54675-9.

[31] M. Merad, Aide à la décision et expertise en gestion des risques, 2010. Lavoisier.

[32] T. Aven, E. Zio, Foundational issues in risk assessment and risk management, Risk Anal. 34 (7) (2014) 1164-1172, https://doi.org/10.1111/risa.12132.

[33] T. Aven, S. Thekdi, Risk Science: an Introduction, first ed., Routledge, London, 2022 https://doi.org/10.4324/9781003156864.

[34] E. Paté-Cornell, L.A. Cox, Improving risk management: from Lame Excuses to principled practice, Risk Anal. 34 (7) (Jul. 2014) 1228-1239, https://doi.org/10.1111/risa.12241.

[35] OECD, Mobilising Evidence for Good Governance, 2020, p. 19.

[36] OECD, Risk and regulatory policy: improving the governance of risk, in: OECD Reviews of Regulatory Reform, OECD, 2010, https://doi.org/10.1787/9789264082939-en.

[37] T. Aven, O. Renn, Risk Management and Governance: Concepts, Guidelines and Applications, Springer, Berlin Heidelberg, 2010.

[38] I. Ayres, J. Braithwaite, Responsive regulation: transcending the deregulation debate, in: Oxford Socio-Legal Studies, Oxford University Press, New York, 1992.

[39] C. Cogliane, Measuring Regulatory Performance: Evaluating the Impact of Regulation and Regulatory Policy, OECD, Aug. 2012 [Online]. Available: https://www.oecd.org/gov/regulatory-policy/1_cogliane%20web.pdf.

[40] N. Gunningham, D. Sinclair, Designing smart regulation, in: Smart Regulation: Designing Environmental Policy, Oxford University Press Oxford, 1998 [Online]. Available: https://www.oecd.org/env/outreach/33947759.pdf. (Accessed 21 October 2023).

[41] N. Gunningham, D. Sinclair, Smart regulation, Regulatory theory: Foundations and applications (2017) 133-148.

[42] A. Hale, D. Borys, Working to rule, or working safely? Part 1: a state of the art review, Saf. Sci. 55 (Jun. 2013) 207-221, https://doi.org/10.1016/j.ssci.2012.05.011.

[43] S.S. Nagel, Bridging theory and practice in policy/program evaluation, Eval. Progr. Plann. 13 (3) (1990) 275-283, https://doi.org/10.1016/0149-7189(90)90058-5.

[44] M. Nilsson, T. Zamparutti, J.E. Petersen, B. Nykvist, P. Rudberg, J. McGuinn, Understanding policy coherence: analytical framework and examples of sector-environment policy interactions in the EU, Environmental Policy and Governance 22 (6) (2012) 395-423.

[45] OECD, Evaluation policy and guidelines for evaluations. https://www.oecd.org/dac/evaluation/iob-evaluation-policy-and-guidelines-for-evaluations.pdf, Oct. 2009.

[46] OECD, Applying Evaluation Criteria Thoughtfully, OECD, 2021, https://doi.org/10.1787/543e84ed-en.

[47] T.H. Poister, Performance monitoring in the evaluation process, Eval. Rev. 6 (5) (Oct. 1982) 601-623, https://doi.org/10.1177/0193841X8200600502.

[48] D. Sinclair, Self-regulation versus Command and control? Beyond False Dichotomies, Law Pol. 19 (4) (1997) 529-559, https://doi.org/10.1111/1467-9930.00037.

[49] F.-B. Van Der Meer, J. Edelenbos, Evaluation in multi-actor policy processes: accountability, Learning and Co-operation, Evaluation 12 (2) (Apr. 2006) 201-218, https://doi.org/10.1177/1356389006066972.

[50] J.B. Wiener, Risk regulation and governance institutions, in: Risk and Regulatory Policy: Improving the Governance of Risk, OECD, 2010, https://doi.org/10.1787/9789264082939-9-en [Online]. Available:

[51] OECD, Regulatory Impact Analysis: A Tool for Policy Coherence, Organisation for Economic Co-operation and Development, Paris, 2009 [Online]. Available: https://www.oecd-library.org/governance/regulatory-impact-analysis_9789264067110-en. (Accessed 3 May 2022).

[52] T. Acheampong, E. Phimister, K. Alexander, What difference has the Cullen Report made? Empirical analysis of offshore safety regulations in the United Kingdom's oil and gas industry, Energy Policy 155 (Aug. 2021) 112354, https://doi.org/10.1016/j.enpol.2021.112354.

[53] T. Acheampong, R. Akumperigya, Offshore risk regulation: a comparative analysis of regulatory framework in Ghana, the United Kingdom and Norway, Energy Policy 113 (Feb. 2018) 701-710, https://doi.org/10.1016/j.enpol.2017.10.009.

[54] T. Acheampong, A.G. Kemp, Health, safety and environmental (HSE) regulation and outcomes in the offshore oil and gas industry: performance review of trends in the United Kingdom Continental Shelf, Saf. Sci. 148 (Apr. 2022) 105634, https://doi.org/10.1016/j.ssci.2021.105634.

[55] B.J.M. Ale, Risk analysis and risk policy in The Netherlands and the EEC, J. Loss Prev. Process. Ind. 4 (1) (1991) 58-64, https://doi.org/10.1016/0950-4230(91)80008-1.

[56] H.N. Bang, A gap analysis of the legislative, policy, institutional and crises management frameworks for disaster risk management in Cameroon, Progress in Disaster Science 11 (Oct. 2021) 100190, https://doi.org/10.1016/j.pdisas.2021.100190.

[57] P.H. Bottelberghs, Risk analysis and safety policy developments in The Netherlands, J. Hazard Mater. 71 (1-3) (2000) 59-84, https://doi.org/10.1016/S0304-3894(99)00072-2.

[58] G. Bramwell, F. Wilson, T. Faunce, Mercury Pollution from coal-Fired power Plants: a critical analysis of the Australian regulatory response to public health risks, J. Law Med. 26 (Dec. 2018) 480-487.

[59] M.T. Brown, M.G. Morgan, R.S. Farrow, Expert assessment of the performance of the US system for environmental regulation, J. Risk Res. 7 (5) (2004) 507-521, https://doi.org/10.1080/1366987032000123847.

[60] M. Hammond, et al., A new flood risk assessment framework for evaluating the effectiveness of policies to improve urban flood resilience, Urban Water J. 15 (5) (May 2018) 427-436, https://doi.org/10.1080/1573062X.2018.1508598.

[61] M.P. Howlett, J.S. Cuenca, The use of indicators in environmental policy appraisal: lessons from the design and evolution of water security policy measures, J. Environ. Pol. Plann. 19 (2) (2017) 229-243, https://doi.org/10.1080/1523908X.2016.1207507.

[62] J. Kuronen, U. Tapaninen, Evaluation of maritime safety policy instruments, WMU Journal of Maritime Affairs 9 (1) (2010) 45-61, https://doi.org/10.1007/BF03195165.

[63] X. Li, A.B. Cundy, W. Chen, Fuzzy synthetic evaluation of contaminated site management policy from the perspective of stakeholders: a case study from China, J. Clean. Prod. 198 (2018) 1593-1601, https://doi.org/10.1016/j.jclepro.2018.07.036.

[64] Q. Liu, X. Li, M. Hassall, Regulatory regime on coal Mine Safety in China and Australia: comparative analysis and overall findings, Resour. Safety 74 (Dec. 2021) 101454, https://doi.org/10.1016/j.resoupol.2019.101454.

[65] L. Maxim, J.P. van der Sluijs, Quality in environmental science for policy: assessing uncertainty as a component of policy analysis, Environ. Sci. Pol. 14 (4) (2011) 482-492, https://doi.org/10.1016/j.envsci.2011.01.003.

[66] M. Mazzocchi, M. Ragona, A. Zanoli, A fuzzy multi-criteria approach for the ex-ante impact assessment of food safety policies, Food Policy 38 (Feb. 2013) 177-189, https://doi.org/10.1016/j.foodpol.2012.11.011.

[67] A. Naime, An evaluation of a risk-based environmental regulation in Brazil: limitations to risk management of hazardous installations, Environ. Impact Assess. Rev. 63 (Mar. 2017) 35-43, https://doi.org/10.1016/j.eiar.2016.11.005.

[68] T. Oka, Cost-effectiveness analyses of chemical risk control policies in Japan, Chemosphere 53 (4) (2003) 413-419, https://doi.org/10.1016/S0045-6535(03)00007-9.

[69] A. Osácar, J.B. Echeverría Trueba, B. Meacham, Evaluation of the legal framework for building fire safety regulations in Spain, Buildings 11 (2) (Feb. 2021), https://doi.org/10.3390/buildings11020051. Art. no. 2.

[70] G.A. Papadakis, Assessment of requirements on safety management systems in EU regulations for the control of major hazard pipelines, J. Hazard Mater. 78 (1-3) (2000) 63-89, https://doi.org/10.1016/S0304-3894(00)00217-X.

[71] R.H. Prasetio, R. Arifuddin, F. Maricar, Performance evaluation indicators for implementing occupational safety and health management policy in small qualified construction services business entities, in: Presented at the IOP Conference Series: Earth and Environmental Science, 2020, https://doi.org/10.1088/1755-1315/419/1/012149.

[72] K. Sexton, S.H. Linder, Integrated assessment of risk and sustainability in the context of regulatory decision making, Environ. Sci. Technol. 48 (3) (Feb. 2014) 1409-1418, https://doi.org/10.1021/es4043066.

[73] M. Tremblay, M. Vandewalle, H. Wittmer, Ethical challenges at the science-policy interface: an ethical risk assessment and proposition of an ethical infrastructure, Biodivers. Conserv. 25 (7) (Jun. 2016) 1253-1267, https://doi.org/10.1007/s10531-016-1123-9.

[74] E.-M. Trüdinger, C. Ziller, Considered effective? How policy evaluations and threat perceptions affect support for surveillance in the context of terrorism, Polit. Pol. 50 (5) (2022) 894-912, https://doi.org/10.1111/polp.12498.

[75] B. Van Wee, M. Hagenzieker, W. Wijnen, Which indicators to include in the ex-ante evaluations of the safety effects of policy options? Gaps in evaluations and a discussion based on an ethical perspective, Transp. Policy 31 (2014) 19-26, https://doi.org/10.1016/j.tranpol.2013.11.002.

[76] K.R. Vastveit, K. Eriksson, O. Njå, Critical reflections on municipal risk and vulnerability analyses as decision support tools: the role of regulation regimes, Environment Systems and Decisions 34 (3) (2014) 443-455, https://doi.org/10.1007/s10669-014-9510-9.

[77] E. Vigier, C. Curt, T. Curt, A. Arnaud, J. Dubois, Joint analysis of environmental and risk policies: methodology and application to the French case, Environ. Sci. Pol. 101 (Nov. 2019) 63-71, https://doi.org/10.1016/j.envsci.2019.07.017.

[78] C. Vitale, S. Meijerink, Flood risk policies in Italy: a longitudinal institutional analysis of continuity and change, Int. J. Water Resour. Dev. 39 (2) (Mar. 2023) 211-235, https://doi.org/10.1080/07900627.2021.1985972.

[79] E.Y. Wong, R.A. Ponce, S. Farrow, S.M. Bartell, R.C. Lee, E.M. Faustman, Comparative risk and policy analysis in environmental health, Risk Anal. 23 (6) (2003) 1337-1349, https://doi.org/10.1111/j.0272-4332.2003.00405.x.

[80] A. Zembe, L.D. Nemakonde, P. Chipangura, Policy coherence between food security, disaster risk reduction and climate change adaptation in South Africa: a summative content analysis approach, Jâmba: Journal of Disaster Risk Studies 14 (1) (Feb. 2022). Art. no. 1.

[81] M. Merad, N. Rodrigues, O. Salvi, Urbanization control around industrial SEVESO sites: the French context, Int. J. Risk Assess. Manag. 8 (Jan. 2008) 158-167, https://doi.org/10.1504/IJRAM.2008.016146.

[82] N. Margossian, Risques et accidents industriels majeurs: Caractéristiques, réglementation, prévention, Dunod, 2006.

[83] Ministère de l'Écologie, du Développement Durable et de l'Énergie (MEDDE), La démarche française de prévention des risques majeurs. https://www.ecologie.gouv.fr/sites/default/files/130601_Demarche-francaise-prevention-risques-majeurs_DEF_Web.pdf. (Accessed 25 December 2021).
[84] E. Negre, Crisis management and distrust: study of an industrial accident in France, in: Proceedings of the 54th Hawaii International Conference on Systems Sciences, 2021, p. 2235.

[85] Ministry of Ecological Transition, Les évolutions réglementaires post-lubrizol sur la prévention et la préparation à la gestion des accidents, en un coup d'œil. https://www.ecologie.gouv.fr/sites/default/files/20118-LUBRIZOL%20VF.pdf, Oct. 2020. (Accessed 22 March 2021).

[86] Sénat, Évaluer l'intervention des services de l'État dans la gestion des conséquences environnementales, sanitaires et économiques de l'incendie de l'usine Lubrizol à Rouen. Tome I: Rapport. https://www.senat.fr/rap/r19-4 80-2/r19-480-2.html, Jun. 2020. (Accessed 12 April 2022).

[87] Sénat, Évaluer l'intervention des services de l'État dans la gestion des conséquences environnementales, sanitaires et économiques de l'incendie de l'usine Lubrizol à Rouen - Thome II: Auditions. https://www.senat.fr/rap/r19-4 80-2/r19-480-2.html. (Accessed 12 April 2022).

[88] S. Tannous, M. Merad, J. Hayes, Major accidents and risk prevention policies in the chemical and petrochemical industry in France: Paving the way towards an assessment framework, in: Proceedings of the 32nd European Safety and Reliability Conference (ESREL 2022), 2022, https://doi.org/10.3850/978-981- 18-5183-4_R04-01-259. Dublin, Ireland.

[89] S. Tannous, M. Merad, J. Hayes, A comparative analysis of risk prevention policy tools and governance structures in Normandy (France) and Victoria (Australia): assessing policies for high-risk sites, Int. J. Disaster Risk Reduct. (Jun. 2024) 104600, https://doi.org/10.1016/j.ijdrd.2024.104600.

[90] S. Tannous, D.J. Castro Rodriguez, M. Merad, M. Demichela, Risk policy tools for high-risk industrial sites in Normandy (France) and Piedmont (Italy): more hazards-focused than vulnerabilities-focused, J. Risk Res. (Apr. 2025) 1-27, https://doi.org/10.1080/13669877.2025.2488383.

[91] E.J. Ward, H.T. Dimitriou, M. Dean, Theory and background of multi-criteria analysis: toward a policy-led approach to mega transport infrastructure project appraisal, Res. Transport. Econ. 58 (Oct. 2016) 21-45, https://doi.org/10.1016/j. retrec.2016.08.003.

[92] P. Digkoglou, A. Tsoukiás, J. Papathanasiou, K. Gotzamani, A Meta-analysis of the review literature on multiple-criteria decision aids for environmental issues, Appl. Sci. 14 (23) (Jan. 2024), https://doi.org/10.3390/app142310862. Art. no. 23.

[93] B. Roy, D. Bouyssou, Aide multicritère à la décision : méthodes et cas, London School of Economics and Political Science, 1993 [Online]. Available: https://ba sepub.dauphine.psl.eu/handle/123456789/4522. (Accessed 31 August 2023).

[94] OECD, The OECD reference checklist for regulatory decision-making. http://www.oecd.org/regreform/regulatory-policy/35220214.pdf, 2005, September 9.

[95] D. Gasper, Policy evaluation—from managerialism and Econocracy to a governance perspective. https://repub.eurol.nl/pub/50685/, 2005.

[96] S.M. Kilonzo, A. Ojebode, Research methods for public policy, in: E.R. Aiyede, B. Muganda (Eds.), Public Policy and Research in Africa, Springer International Publishing, 2023, pp. 63–85, https://doi.org/10.1007/978-3-030-99724-3-4.

[97] W.N. Dunn, Public Policy Analysis: an Integrated Approach, Routledge, 2017.

[98] W. Jiang, R. Marggraf, The origin of cost-benefit analysis: a comparative view of France and the United States, Cost Eff. Resour. Allocation 19 (1) (2021) 74, https://doi.org/10.1186/s12962-021-00330-3.

[99] D. Pearce, Cost benefit analysis and environmental policy, Oxf. Rev. Econ. Pol. 14 (4) (1998) 84–100, https://doi.org/10.1093/oxrep/14.4.84.

[100] D.L. Weimer, Cost-benefit analysis and public policy, J. Pol. Anal. Manag. (2008). https://www.wiley.com/en-gb/Cost-Benefit+Analysis+and+Public+Polic y-p-9781405190169.

[101] House of Lords, Government Policy on the Management of Risk: 5th Report of Session 2005-06—Great Britain: Parliament: House of Lords: Select Committee on Economic Affairs—Google Books, 2006.

[102] T. Aven, E.B. Abrahamsen, On the use of cost-benefit analysis in ALARP processes, Int. J. Perform. Eng. 3 (3) (2007) 345, https://doi.org/10.23940/ijpe.07.3.p345. mag.

[103] T. Aven, O. Renn, Improving government policy on risk: eight key principles, Reliab. Eng. Syst. Saf. 176 (2018) 230-241, https://doi.org/10.1016/j. ress.2018.04.018.

[104] M.W. Jones-Lee, M. Hammerton, P.R. Philips, The value of safety: results of a national sample survey, Econ. J. 95 (377) (1985) 49-72, https://doi.org/ 10.2307/2233468.

[105] M. Jones-Lee, T. Aven, The role of social cost-benefit analysis in societal decision-making under large uncertainties with application to robbery at a cash depot, Reliab. Eng. Syst. Saf. 94 (12) (2009) 1954-1961, https://doi.org/10.1016/j. ress.2009.06.011.

[106] M. Merad, N. Dechy, L. Dehouck, M. Lassagne, Risques majeurs, incertitudes et décisions, in: Approche pluridisciplinaire et multisectorielle, 2016. MA Editions, https://www.decitre.fr/livres/risques-majeurs-incertitudes-et-decisions -9782822404303.html.

[107] J.B. Wiener, Risk regulation and governance institutions, in: Risk and Regulatory Policy: Improving the Governance of Risk, OECD, 2010, https://doi.org/10.1787/ 9789264082939-9 en.

[108] A. Sen, The discipline of cost-benefit analysis, J. Leg. Stud. 29 (S2) (2000) 931-952, https://doi.org/10.1086/468100.

[109] M. Merad, Aide à la décision et expertise en gestion des risques, 2010. Lavoisier.

[110] M. Merad, B.D. Trump, Expertise under Scrutiny, Springer, 2020. https://link.spr inger.com/book/10.1007%2F978-3-030-20532-4.

[111] Aven, The Science of Risk Analysis: Foundation and Practice, Routledge & CRC Press, 2019. https://www.routledge.com/The-Science-of-Risk-Analysis-Foundat ion-and-Practice/Aven/p/book/9780367139223.

[112] T. Aven, S. Thekdi, Risk Science: an Introduction, first ed., Routledge, 2022 https://doi.org/10.4324/9781003156864.

[113] M.S. Feldstein, The social time preference discount rate in cost benefit analysis, Econ. J. 74 (294) (1964) 360-379, https://doi.org/10.2307/2228484.

[114] E.R. Morrison, Judicial review of discount rates used in regulatory cost-benefit analysis, Univ. Chicago Law Rev. 65 (4) (1998) 1333-1369, https://doi.org/ 10.2307/1600266.

[115] European Commission, Input-output economics. https://joint-research-centre.ec. europa.eu/scientific-activities-z/input-output-economics_en, 2022.

[116] United Nations, Handbook of input-output table compilation and analysis. http s://digitallibrary.un.org/record/370160, 1999.

[117] C.F. Christ, A review of input-output analysis, in: Input-Output Analysis: an Appraisal, Princeton University Press, 1955, pp. 137-182. https://www.nber.org/ books-and-chapters/input-output-analysis-appraisal/review-input-output-anal ysis.

[118] E.S. van Leeuwen, P. Nijkamp, P. Rietveld, Regional input-output analysis, in: K. Kempf-Leonard (Ed.), Encyclopedia of Social Measurement, Elsevier, 2005, pp. 317-323, https://doi.org/10.1016/B0-12-369398-5/00349-2.

[119] E. Dietzenbacher, M. Lenzen, B. Los, D. Guan, M.L. Lahr, F. Sancho, S. Suh, C. Yang, Input-output analysis: the next 25 years, Econ. Syst. Res. 25 (4) (2013) 369-389, https://doi.org/10.1080/09535314.2013.846902.

[120] A.E. Boardman, D.H. Greenberg, A.R. Vining, D.L. Weimer, Cost-Benefit Analysis: Concepts and Practice, Cambridge University Press, 2017.

[121] G. Munda, P. Nijkamp, P. Rietveld, Monetary and non-monetary evaluation methods in sustainable development planning, Econ. Appl. 48 (2) (1995) 143-160.

[122] I. Linkov, E. Moberg, Multi-Criteria Decision Analysis, 2012.

[123] G. Munda, On the Use of Cost-Benefit Analysis and Multi-Criteria Evaluation in Ex-Ante Impact Assessment, Publications Office of the European Union, Luxembourg, 2017.

[124] B. Roy, D. Bouyssou, Aide multicritère à la décision: Méthodes et cas, London School of Economics and Political Science, 1993. https://basepub.dauphine.psl. eu/handle/123456789/4522.

[125] G. Lucertini, C. D'Alpaos, A. Tsoukiás, Evaluating public policies normative models beyond cost benefit analysis. https://basepub.dauphine.fr/bitstream/h andle/123456789/9265/tesi_Lucertini.pdf?sequence=2, 2012.

[126] L. Wolfson, J. Kadane, M. Small, Expected Utility as a Policy Making Tool: an Environmental Health Example, 151, 1995, pp. 261-278.

[127] OECD, Evaluation policy and guidelines for evaluations. https://www.oecd.org /dac/evaluation/iob-evaluation-policy-and-guidelines-for-evaluations.pdf, 2009, October.

[128] C.V. Patton, D.S. Sawicki, Basic Methods of Policy Analysis and Planning, third ed., Pearson, 2013.

[129] F. Juglaret, J.-M. Rallo, R. Textoris, F. Guarnieri, E. Garbolino, Occupational Health and Safety Scorecards: New Leading Indicators Improve Risk Management and Regulatory Compliance, 2011, p. 16. https://minesparis-psl.hal.science/hal -00660460.

[130] R.K. Yin, K.A. Heald, Using the case survey method to analyze policy studies, Acad. Sci. Q. 20 (3) (1975) 371-381, https://doi.org/10.2307/2391997.

[131] L.E. McCray, K.A. Oye, A.C. Petersen, Planned adaptation in risk regulation: an initial survey of US environmental, health, and safety regulation, Technol. Forecast. Soc. Change 77 (6) (2010) 951-959, https://doi.org/10.1016/j. techfore.2009.12.001.

[132] UNDRR, Disaster resilience scorecard for cities detailed level assessment. https://www.undrr.org/publication/disaster-resilience-scorecard-cities#:~:text=The%20Scorecard%20provides%20a%20set,and%20assess%20their%20di saster%20resilience, 2017.

[133] Senat.fr, CE Incendie de l'usine Lubrizol : compte rendu de la semaine du 1er juin 2020. http://www.senat.fr/compte-rendu-commissions/20200601/ce_lubrizol. html. (Accessed 22 December 2020).

[134] Senat.fr, Présentation des conclusions de la commission d'enquête chargée d'évaluer l'intervention des services de l'État dans la gestion des conséquences environnementales, sanitaires et économiques de l'incendie Lubrizol à Rouen. http://www.senat.fr/fileadmin/Fichiers/Images/redaction_multimedia/2020/2020 -Documents.pdf/20200604-Conf_presse_CE_Lubrizol/Synth_fiches_thematiques. pdf.

[135] legifrance.gouv.fr, Décret n° 2020-1168 du 24 septembre 2020 relatif aux règles applicables aux installations dans lesquelles des substances dangereuses sont présentes dans des quantités telles qu'elles peuvent être à l'origine d'accidents majeurs - Légifrance. https://www.legifrance.gouv.fr/jorf/id/JORFTEXT0000 42364321. (Accessed 15 October 2023).

[136] M. Merad, Prévention des risques industriels : Chronique d'une crise chronique, in: Actes du 55ème Congrès de la SELF, L'activité et ses frontières. Penser et agir sur les transformations de nos sociétés, Jan. 2021 [Online]. Available: https:// ergonomie-self.org/wp-content/uploads/2021/01/SELF-2020-actes.pdf.

[137] S. Tannous, M. Merad, J. Hayes, The Aftermath of 26 September 2019 accident: a focus on risk-related policy analysis, in: Proceedings of the 31st European Safety and Reliability Conference (ESREL 2021), Research Publishing Services, Angers, France, 2021, pp. 1959-1966, https://doi.org/10.3850/978-981-18-2016-8_198- cc.

[138] D.H. Meadows, Thinking in Systems: a Primer, Earthscan, London, 2009.
Scarlett TANNOUS holds a Ph.D. in Management at Paris Dauphine University – PSL. She specialized in industrial risk management regulations and policies for an effective decision-making process. She investigated the effectiveness of risk-related public policies in France and Australia. She also holds a chemical engineering degree, a minor in petroleum engineering, and a master’s degree in engineering management. Her research experience encompasses environmental and socio-economic analyses of different energy systems and petroleum equipment.

Terje AVEN has been a professor in risk science at the University of Stavanger, Norway since 1992. Previously he was also professor (adjunct) in risk analysis at the University of Oslo and the Norwegian University of Science and Technology. He has many years of experience as a risk analyst in industry and as a consultant. He is the author of many books and papers in the field, covering both fundamental issues and practical risk analysis

methods. He has led several large research programs in the risk area, with strong inter-national participation. He has developed many master programs in the field and has lectured on many courses in risk analysis and risk management. Aven is Editor-in-Chief of the Journal of Risk and Reliability, and Area Editor of Risk analysis in Policy. He has served as President of the international Society for Risk Analysis (SRA) and Chairman of the European Safety and Reliability Association (ESRA) (2014–2018)

Myriam MERAD is a Research Director/Professor at CNRS - LAMSADE at Paris Dauphine University – PSL. She leads research in risk, resilience, and sustainability governance and decisions in security, health, safety, and environment. She is primarily interested in the link between Science and Decision and mainly in expertise framing, regulatory and policy analysis, and participative/deliberative decision analysis.