# Using the Functional Resonance Analysis Method (FRAM) in Aviation Safety: A Systematic Review

Authors: Wulin Tian; Carlo Caponecchia
Journal: Journal of Advanced Transportation
 (2020)

---

## Abstract

The functional resonance analysis method (FRAM) is a system-based method to understand highly complex sociotechnical systems. Besides learning from safety occurrences or undesirable states, FRAM can be used to understand how things go well in a system, by identifying gaps between “work as imagined” (WAI) and “work as done” (WAD). FRAM is increasingly used in many domains and can enhance our understanding of a complex system and proposes strategies to refine the work design. This systematic review identified 108 FRAM research papers from 2006–2019. Most of these papers were conducted by European researchers and employed qualitative methods such as document analysis, interviews, and focus groups with subject matter experts (SMEs) and observations to develop WAI and WAD. Despite being used in healthcare, construction, and maritime sectors among others, aviation was the most commonly explored domain in FRAM studies. The 26 FRAM studies in aviation explored many aspects of the aviation industry, including Air Traffic Control (ATC) systems, cockpit operation, ground handling, maintenance, and a range of past safety incidents, like runway incursions. This paper also characterises the FRAM studies focused on aviation in terms of the common methods and steps used to build FRAM and the available software tools to build FRAM nets. Current FRAM illustrates its advantages in capturing the dynamic and nonlinear nature of complex systems and facilitates our understanding and continual improvement of complex systems. However, there are some critical issues in FRAM use and interpretation, such as the consistency of methods and complexity and reliability of data collection methods, which should be considered by researchers and FRAM users in industry.


---

## Body

## 1. Introduction

Complex systems comprise different groups of humans, technologies, and organisations that may interact with each other in many industrial domains. Ladyman et al. [1] argued that a complex system has the following features: nonlinearity, feedback, robustness and lack of central control, emergence, spontaneous order, hierarchical organisation, and numerosity. The essential characteristic of complex systems is nonlinearity; that is, the presence of factor A does not necessarily lead to outcome B, and vice versa [1]. A complex system consists of numerous interacting components. Components affect and are affected by one another. Normally, this kind of feedback does not affect the overall system’s operational status. The system itself is robust enough and able to absorb minor variabilities. Nevertheless, under particular situations, the same set of variabilities may make the system fail. In other words, interactions between different components are dynamic and emergent, rather than static and ancillary. The dynamic conditions and interactions make the system behaviour difficult to predict. The traditional simple linear relationship cannot explain the entire complex sociotechnical systems comprehensively (e.g., [2, 3]).One increasingly common method for analysing complex systems is the functional resonance analysis method, known as “FRAM.” FRAM is a system-based analysis method, which considers the whole system and focuses on its functioning rather than the structure of its components. Originally known as the “functional resonance accident model,” FRAM was initially established by Hollnagel in 2004 [4], to investigate accidents and incidents in complex social-technical systems. However, learning only from accidents/incidents is not sufficient to understand the interactions between technologies, humans, and organisations that make up a complex sociotechnical system. This is especially the case when it comes to ultrasafe systems, such as in the aviation and nuclear industries. Safe operation of complex systems requires a better understanding of how the work is actually carried out (“work as done,” WAD) [5]. FRAM’s scope has been expanded from an accident model to a more general analysis method, termed the functional resonance analysis method. The gaps between WAD and how the work is supposed to be done (“work as imagined,” WAI) generate variabilities in the daily working procedure. Any single instance of variability alone is not able to lead to the accident/incident. However, under specific conditions, these variabilities may lead to functional resonance, causing an undesired outcome or even crashing down the entire system.Since its establishment, experts from different disciplines have applied FRAM to a range of industries or contributed to developing FRAM theory and methods. The advantages of FRAM as a methodology have been suggested by several studies. Applying FRAM can facilitate a better and more in-depth understanding of interactions between complex system functions. For example, Woltjer and Hollnagel [2] applied FRAM to analyse the Alaska Airlines Flight 261 accident. The analysis suggested that FRAM could capture the dynamic and nonlinear nature of this complex system failure. Later on, Hollnagel and colleagues [6] again used FRAM to reanalyse the Comair Flight 5191 accident. The FRAM analysis results suggested a number of additional countermeasures compared to those suggested in the official NTSB report. By monitoring how component variabilities resonate, FRAM modelling can also identify the critical path of variabilities that emerged in the dynamic system [2, 7]. Using FRAM to detect the gap between WAI and WAD helps us improve safety and work design [8]. Compared to currently employed investigation methods and other systematic methods, such as the Domino model, Swiss Cheese Model, and the Bow-Tie model, FRAM is able to analyse complex systems and provide more comprehensive results [6, 9–11]. For example, Hollnagel and colleagues [6] reanalysed the Comair Flight 5191 using FRAM. The National Transportation Safety Board (NTSB) report mainly identified crew members’ mistakes and flaws in the Federal Aviation Administration (FAA) requirements for Air Traffic Control (ATC) clearance authorization. By taking the whole context into consideration, FRAM unveiled further details of why the accident occurred than those presented in the NTSB official report. These included that information regarding runway construction and closure was missing in both Notice to Airmen (NOTAMs) documents and the Automatic Terminal Information Service (ATIS) and that the first officer was too busy to monitor the aircraft’s position on the runway.Given the increasing use of FRAM to better understand complex systems and its potential for use retrospectively and prospectively, this paper aims to systematically review FRAM studies, with an emphasis on how it has been applied in aviation. We first outline the principles of FRAM and how the method should be conducted, before reviewing the literature and analysing the methods used, the locations of FRAM studies, and the systems to which FRAM has been applied.


## 2. Principles of FRAM

The following section outlines the theoretical principles that guide the implementation of FRAM. The first principle is that of the equivalence of success and failures [5]. Traditional safety theories emphasised learning from system failures, such as incidents and accidents [5]. However, learning from failures is not enough for keeping current complex sociotechnical systems safe. FRAM can be applied to analyse either system incident/accident or the normal operational procedure. According to FRAM, to understand what goes right when the daily work is carried out is as important as understanding what failed in the system.The second principle is that of approximate adjustments [5]. Human performance can be influenced by many factors, both internal and external, such as fatigue, stress, emotions and mood, vigilance, task demands, and deadlines. Sometimes organisational factors such as the effectiveness of communication or unclear guidelines can make workers’ tasks more difficult. The complex working context may make the work task more challenging and require workers to make their own decisions. Workers have to adjust their behaviour accordingly to meet the system’s requirements to produce the desired outcome. They usually need to make some trade-offs between being efficient and to make sure the work can be completed as precisely as possible. Hollnagel [4] termed these kinds of adjustments as efficiency-thoroughness trade-offs (ETTOs). These adjustments are necessary and understandable; however, any changed system behaviour may raise variabilities in the system.The third principle is that of emergence [5]. Under each analysed case, the context and combination of variabilities in the system are unique. As the interactions within a complex system are dynamic and nonlinear, the occurrence of an outcome is emergent. To be more specific, minor variabilities always exist in normal system operations and do not affect system safety. However, the particular external environment may integrate variabilities and magnify their influence to generate an undesired outcome.The fourth principle is that of functional resonance [5]. From the FRAM perspective, variabilities exist in normal daily operations. These small variabilities may not be able to crash the system alone, but aggregated with other variabilities in the system may cause resonance, which generates a negative outcome. The whole system should be taken into account, instead of focusing on one segment of the system.In performing a FRAM analysis, several steps are involved, which then takes these principles into account. The following describes the main steps of conducting a FRAM analysis [5]:Step 0: define the purpose and scope of analysis.Before the analysis starts, analysts should clarify whether the analysis would be conducted in relation to an incident scenario or the context of normal operations. As there is no clear boundary to conduct the FRAM and the results of FRAM would be complicated, setting the scope of the proposed analysis can prevent the results from being too detailed and too complicated. For example, to examine the catering delivery procedure during a flight turnaround, it would be good to start from the function “preparing food in the catering department” and end up with “crew confirms catering delivery.” Otherwise, the food preparation process could be traced to very early stage functions, like “purchase ingredients,” “design menu,” or even “grow vegetables.” Without a clear boundary, the FRAM analysis might generate a lot of data and appear comprehensive but be ultimately unhelpful in understanding the system of interest.Step 1: identify and describe the essential system functions.FRAM deconstructs the complex sociotechnical system into “functions.” A function is a task or an activity that is required to produce a certain outcome. According to Hollnagel [5], there are three types of functions: technological functions, human functions, and organisational functions. Each identified function can be described by six aspects (see Figure 1):Input (I): input is what activates or starts a function and/or that is used or transformed by the function to produce the output.Preconditions (P): preconditions refer to the conditions that must be satisfied before a function is carried out. Preconditions alone cannot activate a function.Resource (R): resource is something that is needed or consumed when the function is active. Resources will be consumed up as the function is executive.Output (O): output is the outcome of a function.Control (C): control is that which supervises, regulates, or monitors the function such as guidelines, regulations, or even social expectations.Time (T): time refers to the temporal constraints on the function such as duration and starting point.Function identification can start from anywhere within a complex system. Documents such as an operation manual or task procedures are useful resources for identifying system functions. Each function interacts with other functions via one or more of their aspects. Interactions connect functions together to form a FRAM net. Figure2 shows a simple FRAM net related to the cabin crew’s work procedure before take-off. Each hexagon represents a function and its six aspects. For example, the output of the function “Greet passenger” is that all passengers are welcomed on board. This output can transform to be the input of the following function “Confirm the number of passengers.”Functions that operate before others, and have a potential effect on others, are termed “upstream” functions. Functions impacted by others are “downstream” functions. For example, in Figure2, the function “Confirm the number of passengers” is upstream of the function “Close headlock” and downstream of the function “Greet passengers.”Step 2: identify the actual or potential variabilities between functions.According to Hollnagel [5], performance variabilities can be categorized in terms of their origins, internal (endogenous) variability, and external (exogenous) variability. Internal variability refers to the variation caused by the function itself, such as software pitfalls or operator’s experience. By contrast, the external variability refers to the influence of other functions, such as weather conditions and organisation’s culture. Hollnagel [4] classified the external variabilities as eleven common performance conditions (CPCs): availability of resource, adequacy of training and experience, communication, adequacy of interface and operational support, availability of procedures or plans, working conditions, number of simultaneous goals, available time, crew collaboration quality, and adequacy of organisation. In 2012 [5], Hollnagel again suggested an elaborate solution and a simple solution to consider the performance variabilities. The elaborate solution identifies variabilities in terms of speed, distance, sequence, object, force, duration, direction, and timing. Meanwhile, the simple solution considers function output variability in terms of timing and precision. According to the simple solution, the output of a function would generate too early, on time, too late, or not at all and can be precise, acceptable, or imprecise. In our example in Figure 2, for instance, the function “Run safety check” could be completed too late and carelessly/imprecisely, which brings variabilities into the system.Step 3: analyse the aggregation of variability.In the FRAM net, the output of a function interacts or “couples” with other functions which are represented in the net by lines connecting the functions (known as “couplings”). As illustrated in Figure2, the function “Conduct safety briefing” couples with function “Run safety check.” In this way, the output of an upstream function may vary and then transfer the variability to its downstream function(s). All upstream-downstream couplings can be analysed in terms of timing and precision. Taking one of the couplings from Figure 2, for example, if the output of the upstream function “Conduct safety briefing” comes too late, the downstream function “Run safety check” would experience a delay. The aggregation of these variabilities may cause resonance in the system, which leads to an undesired outcome. In the present case, the flight may not be able to take-off on time.Step 4: propose ways to manage variability.The previous steps help to identify the variabilities and their potential aggregation within the system. The last step should be proposing strategies for managing variability. The FRAM considers system success and failures equivalently. We should consider different management strategies for variabilities that lead to positive outcomes and those contribute to an undesired result. Clarke et al. [12] assessed the potential risks of transferring cargo between two floating harbour transhippers (FHTs) using FRAM. The results showed the manner in which variabilities are added together to influence the system. Some recommendations were developed to improve system design. For example, the number of crew should be sufficient on both vessels, and equipment should be regularly checked and placed in designated places [12].Despite the growth in the frequency with which it is used (e.g., [13–15]), there are few systematic reviews that examined FRAM and its implementation. Some reviews have included FRAM papers, but in the context of examining other system-based analysis tools. For example, Hulme et al. [16] reviewed peer-reviewed articles that applied systemic accident analysis methods to understand contributing factors between 1990 and 2018. They chose four groups of system-based accident analysis methods: AcciMap, the Human Factor Analysis and Classification System (HFACS), the System Theoretic Accident Model and Processes-Causal Analysis based on STAMP (STAMP-CAST), and FRAM. Only four FRAM studies were included in their analysis. The authors examined accident contexts, the number of identified functions, source of accidents, the nature of accidents, and features of eligible articles. All these reviewed analysis methods resulted in multiple contributing factors, couplings, and functions. However, they concluded that the results of FRAM would be highly complex and difficult to interpret [16].Little is therefore known about the pattern of use of this emerging systems safety tool. Accordingly, the present systematic review aims to outline how, where, and for what purpose FRAM has been used, with a particular focus on how FRAM has been applied in the aviation industry.Figure 1
A hexagon representing a function, with the six aspects of input (I), output (O), preconditions (P), resources (R), control (C), and time (T).Figure 2
A simple FRAM net related to cabin crew’s work before take-off.


## 3. Methodology

FRAM was initially developed to investigate accidents and known as the functional resonance accident model. By adopting the Safety-II perspective [5], FRAM transformed to the functional resonance analysis method and expanded its analysis scope to system normal operation (e.g., [17]). Preliminary searches showed that searching for “functional resonance analysis method” alone returned thousands of papers concerned with “functional magnetic resonance imaging” (fMRI) from medical journals and other methods or theories. Accordingly, the functional resonance analysis method was joined with “FRAM” in subsequent searches. In order to cover all eligible FRAM research studies in English, we used the search item “functional resonance analysis” OR “functional resonance accident” and “functional resonance analysis method” AND “FRAM” in the title and abstract of as a keyword across five online databases: ProQuest, PubMed, ScienceDirect, Scopus, and Web of Science. The search timeframe was unlimited although it is recognised that most FRAM studies have occurred since 2012 when the key resource on the method was released. EndNote X9 for Mac was used to organise all references.At the screening stage of the review, all records were assessed manually. Records were excluded if they focused on other systematic analysis methods rather than FRAM, such as STAMP (without a focus on FRAM). Book chapters, theses, commentaries, newspaper articles, and corrigenda were excluded. Documents where the full text was unavailable (after library databases and web searches) or the full text is in a language other than English were also excluded.A taxonomy of current FRAM studies was developed to organise the resulting papers. Our analysis examined the distribution of the number of published papers across years, the regions of current FRAM studies, and the contexts of current FRAM instantiations.


## 4. Results

Figure3 is a PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [18] flowchart diagram showing the search process. The initial search across five databases identified 2212 records. After removing duplicates, the remaining 1651 records were then screened manually by reviewing titles and abstracts. As mentioned above, the functional resonance analysis method is highly similar to fMRI in medical research, and 1481 records were excluded. Furthermore, when the remaining 170 records were assessed for eligibility, 75 records that did not meet inclusion criteria, such as full text being unavailable or the full text was not in English, were removed. In addition to the 95 eligible papers, 13 additional papers were identified from the reference lists of papers that had been found in the search. Finally, 108 papers were included in the analysis (see Figure 3).Figure 3
The literature identified and screened is presented as a PRISMA flowchart diagram (the 26 aviation papers were included in the total 108 papers included in the qualitative synthesis).Figure4 shows the number of published papers over time. Before 2012, 10 out of 16 FRAM studies used accidents or incidents as instantiations. Later on, by adopting the perspective of Safety-II, FRAM expanded its analysis scope to normal operational conditions. In 2012, Hollnagel published the first book about FRAM, FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-Technical System. This book provides systematic background information, steps, and principles to use FRAM. The number of FRAM-related papers increased around two years later. However, since 2017, published FRAM papers reduced to 19 and 16 in the following two years, respectively.Figure 4
The number of published papers from 2006–2019.The FRAM has attracted researchers’ attention worldwide. Among the eligible 108 papers, over half (56.48%,n = 61) were conducted by researchers from European countries, such as Denmark, Sweden, and Italy. Asian researchers from China, Japan, and other countries published 22 papers, accounting for 20.37%. South American researchers contributed 10.19% (n = 11) FRAM papers, while North American researchers published seven studies (6.48%). Oceanian researchers published seven studies (6.48%) related to FRAM. Studies on FRAM also facilitated international cooperation; however, the analysis above was based on the lead author’s location. For instance, Damen et al. [19] analysed preoperative anticoagulation management (PAM) in normal operations using FRAM. The study was conducted in surgery departments in both Australia and the Netherlands.As a general safety analysis method, FRAM could be applied to a variety of complex sociotechnical systems. Researchers from different areas expanded and illustrated the application of FRAM in a wide range of instantiation contexts. Figure5 shows that aviation, healthcare, construction, and maritime contexts are the most popular domains of FRAM application. Those contexts with fewer than three papers were categorized as “others” and included applications of FRAM that focused on the environment, policy-making, sport and recreation (hunting), and natural disasters (flood). As some researchers illustrated their proposals by analysing several cases, the total number of selected papers in Figure 2 is beyond 108. For example, Amorim and Pereira [20] applied FRAM to understand the accidents resulting from improvisation in workplaces and used aircraft maintenance, construction, and shoe manufacturing as case studies. Similarly, Moškon and colleagues [21] demonstrated their proposed method in five cases, including emergency room triage, fire prevention, construction management, aircraft take-off, and flight operations.Figure 5
The number of FRAM applications across domains.In terms of existing research methods, the results indicated that current FRAM is still predominately a qualitative method. Eighty-five papers (78.7%) employed qualitative approaches, such as interviews, documentary reviews, focus groups, or observations to develop WAI and WAD. More specifically, a majority of the papers that mentioned their method to map WAI indicated that they used document review and analysis [14, 19, 22–24]. In order to get an insight regarding WAI, the most popular methods are semistructured interviews, direct observations, and workshops with operators and regulators [3, 10, 23, 25–33].The authors of the remaining 23 (21.3%) papers applied FRAM by using quantitative or semiquantitative methods, such as Monte Carlo simulation and modelling. Furthermore, 19 out of these 23 studies integrated FRAM with other methods to quantify their proposed models, including the Cognitive Reliability and Error Analysis Method (CREAM) (e.g., [34]), the Analytic Hierarchy Process (AHP) (e.g., [35, 36]), and finite state machine (FSM) (e.g., [37]). Among these quantitative and semiquantitative studies, it was much more common practice to characterise the performance variability in terms of timing and precision, rather than the 11 CPCs as originally identified by Hollnagel [5].


## 5. FRAM in Aviation

As reported above, FRAM has been used most frequently in the aviation industry compared to other domains. Hence, we further analysed the 26 papers which applied FRAM to the aviation industry (see Table1). The pattern of aviation FRAM studies is consistent with what we found in relation to characteristic of FRAM studies in other industries in terms of locations and methodologies.Table 1
Studies using FRAM in Aviation from 2006 to 2019.
AuthorYearAviation areaCase/system detailsInstantiation typeTimeframeFRAM toolsFRAM building methodsVariabilityResearch typeSawaragi et al. [38]2006AccidentAmerican Airlines Flight 965 accident, 1995AccidentRNAAccident report reviewCPCsQualitativeWoltjer and Hollnagel [2]2007AccidentThe Alaska Airlines Flight 261 accident, 2000AccidentRNAAccident report reviewCPCsQualitativeWoltjer and Hollnagel [17]2008Air Traffic Management (ATM)ERASMUS projectNormal operationPNALiterature review, interview, observation, experimentCPCs and variability phenotypesQualitativeWoltjer [39]2008AccidentThe Alaska Airlines Flight 261 accident, 2000AccidentRNAAccident report reviewCPCsQualitativeHollnagel et al. [6]2008AccidentComair Flight 5191 accident, 2006/departure routineAccidentRNAAccident report reviewTime, precisionQualitativeMacchi et al. [40]2009Landing approach-ATCMSAW systemNormal operationPNAWorkshop, literature review, interviewTime, precisionQualitativeHerrera and Woltjer [41]2010IncidentNorwegian Air Flight NAX541 incidentAccidentRNAAccident report review, interviewCPCsQualitativede Carvalho [42]2011AccidentMidair collision between Flight GL1907 and Flight N600XL, 2006AccidentRNALiterature review, interviewNAQualitativeNouvel et al. [43]2011Landing approachFlight to Paris-Orly, 1997IncidentRNACPCsQualitativeHerrera et al. [25]2011Helicopter operationHelicopter operationNormal operation and incidentRNALiterature review, observation, interview, workshopNAQualitativeMartinie et al. [44]2013Cockpit operationWeather radar interactive systemNormal operationPNANAQualitativeFrost and Mo [45]2014Airline operation control system (OCC)Airline operation control system (OCC)Normal operationPNASMEs workshopEarly, delayed, absent, wrong rate, underspecified, overspecified, time-pressureQualitativeRagosta et al. [46]2015ATMAircraft route changeNormal operationPNATime, precisionQualitativeYang and Tian [47]2015Landing process-crewLanding process based on the flight, crew, training, and manualNormal operationPNADocument analysisTime, precisionQualitativeDuan et al. [9]2015AccidentScandinavian Airlines Flight 686 and a Cessna Citation CJ2 business jet collision, 2001AccidentRNAReport reviewOtherSemiquantitativeAmorim and Pereira [20]2015MaintenanceAircraft maintenanceAccidentRNATime, precisionQualitativeHirose et al. [34]2016AccidentAmerican Airlines Flight 965 accident, 1995AccidentRNACPCsSemiquantitativePatriarca et al. [13]2017ATMNormal operationPFMVTime, precisionSemiquantitativeStudic et al. [3]2017Ground handlingGround handling servicesNormal operationRFMVLiterature review, observation, interview, and expert judgmentInternal, external, couplings between functionsQualitativeYang et al. [14]2017ATMMinimum Safe Altitude Warning (MSAW) in ATMNormal operationPFMVGuidelines reviewTime, precision, 6 LEVELSSemiquantitativeStogsdill and Ulfvengren [11]2017ATSAir Transport SystemNormal operationPNAQualitativeRutkowska and Krzyzanowski [48]2018ATC-ACC APCTransferring control over the aircraft between the area control centre (ACC) and approach control (APC) unitsNormal operationPFMVDocument reviewQualitativePatriarca et al. [32]2018ATMUS Air Flight 1493, 1991 (collided with SkyWest Flight 5569)AccidentRFMVNTSB report review, testimonies, documents and coeval accounts review, SME’s analysisTime, precisionSemiquantitativeFerreira and Cañas [49]2019ATCAUTOPACE project: propose training for ATCo under foreseeable automation scenarios.Normal operationPFMV/myFRAMTime, precisionQualitativeAdriaensen et al. [15]2019Cockpit operationDouglas Aircraft DC-9 cockpit operation during approach and landingNormal operationPFMV/myFRAMDocument analysis, workshop, interviewEndogenous variability, exogenous, couplingsSemiquantitativeMoškon et al. [21]2019Flight operationLarge aircraft flight operation and light sport aircraft take-offNormal operationPFMVNAQualitativeR = retrospective; P = prospective; NA = not applicable.In most of the 26 aviation FRAM studies, 19 (73.08%) were completed by researchers from European countries, while researchers based in Asia, South America, and Oceania published 4 (15.38%), 2 (7.69%), and 1 (4.35%) paper(s), respectively.FRAM studies in aviation were also dominated by qualitative methodologies, with 20 papers (76.92%) using qualitative methods. Building the FRAM network to understand WAI and WAD, the common methods were literature reviews (such as examining previous relevant accident reports and operational manuals), direct observations, workshops, and interviews with SMEs, such as pilots and air traffic controllers. After completing the initial FRAM network, researchers invited SMEs to check the completeness and validate the FRAM network. Only 6 (23.08%) of the aviation papers attempted to quantify FRAM. Hirose et al. [34] adapted CREAM to use FRAM in a systematic and quantitative way. Yang et al. [14] used Simple Promela Interpreter (SPIN) to demonstrate the functional resonance in system. Patriarca and colleagues [32] discussed using the Resilience Analysis Matrix (RAM) to enhance FRAM-based accident analysis. Moreover, Patriarca et al. [13] proposed a method to quantify function variabilities in relation to the ATM system based on Monte Carlo simulation.Hollnagel [5] suggested that variabilities can be defined from several perspectives. Except for five studies that did not clarify how they categorized identified variabilities, most of the existing research used the simple solution, considering variabilities in terms of only time and precision (n = 9). Seven studies used CPCs to identify variabilities. Some researchers proposed new ways of defining variabilities. For example, Duan et al. [9] considered variability “within” functions and “between” functions. Variability within a function refers to how an output could be influenced by variabilities from the other five aspects of the same function. Variability between functions refers to how the output variability of upstream functions could influence aspects of downstream functions. Frost and Mo [45] suggested using two sets of guidewords to identify potential variabilities. Firstly, the variability in each of the five aspects (input, time, control, precondition, and resource) is rated as early, delayed, absent, wrong rate, underspecified, or overspecified. Secondly, for the same function aspects, variability from the influence of the specific conditions (such as time pressure, goal conflicts, communication quality, and organisation culture) is assessed.Half of the analysed papers (n = 13, 50%) followed the four-step FRAM analysis method. Few papers (n = 2, 7.69%) indicated that their FRAM analysis consists of five steps. However, some of them started the first step from determining the scope of the proposed analysis (see [3]). Some research only focused on the first three FRAM steps. For example, Macchi et al. [40] employed FRAM analysis to assess the risks of the Minimum Safety Altitude Warning system (MSAW). To explore more possibilities of FRAM, some researchers modified the original FRAM steps to satisfy their research goal. For example, Frost and colleagues [45] consulted expert groups to refine and confirm the baseline FRAM model. The validated FRAM model was used to develop instantiations for later analysis in a hazard identification and analysis (HAZID) workshop. The modified FRAM was then used with System Hazard Analysis (SHA) to identify system hazards. Yang and Tian [47] extended the original FRAM to a 7-step method. In addition to the original first three steps, they defined safety requirements and transferred FRAM models into the MuSMV model checker. The proposed approach was evaluated by analysing the landing process from the flight crew’s perspective. The analysis identified a series of variabilities which may violate safety requirements.Topics of FRAM studies in aviation covered different aspects of aviation systems, including the overall air accident/incident analysis (n = 8, 30.77%), ATM/ATC system (n = 8, 30.77%), cockpit operation (n = 3, 11.54%), landing approach (n = 3, 11.54%), Airline Operation Control System (OCC) (n = 1, 3.85%), helicopter operation (n = 1, 3.85%), ground handling procedure (n = 1, 3.85%), and aircraft maintenance (n = 1, 3.85%).Among these 26 aviation FRAM studies, 13 of them used FRAM in a prospective way to assess risks, while the remaining 13 studies used FRAM retrospectively to analyse what happened in aviation incidents. Meanwhile, some studies also used FRAM to assess the target system’s resilience characteristics. For example, Woltjer [39] reanalysed the Alaska Airline Flight 261 accident using FRAM and evaluated resilience characteristics, such as buffering capacity, flexibility, margin, tolerance, and cross-scale interactions, based on FRAM analysis.Currently, there are two software tools developed specifically for FRAM: the FRAM Model Visualiser (FMV) and myFRAM. Both of them are useful tools to build a FRAM model net.FMV works using Adobe Air, while myFRAM can be used in Microsoft Excel. The output of myFRAM can be exported to FMV. Only 7 out of 26 studies indicated the software they used to build and visualize their FRAM. All of these seven used FMV, while two of them also used myFRAM.


## 6. Discussion

The present study aimed to review existing FRAM studies to understand how, where, and for what purpose FRAM has been used, with a special interest in its application in the aviation industry. Our analysis provides a taxonomy of current FRAM studies with respect to their origins, contexts, and research methodologies. It can assist users of the method to interpret the assumptions in the implementation of FRAM and evaluate recommendations for system improvements.Another recent review (see [16]) considered a range of systems safety analysis tools over a similar time period yet had a limited inclusion of FRAM, finding only 4 FRAM papers. By contrast, our systematic review found 108 FRAM papers from 2006–2019. Most FRAM research was conducted in Europe. While there was a wide distribution of focal domains including healthcare, construction, and maritime, most papers focused on systems in aviation such as ATC and aviation incidents. This suggests that FRAM is an emerging methodology in aviation safety, which may influence its adoption in other domains. Most of the aviation papers adopted a qualitative approach to gather information to build the FRAM net (e.g., using focus groups to collect data) and were evenly distributed in terms analysing past incidents and current systems operations.Despite there being similarities in the general approach adopted (such as qualitative methods of interviews and focus groups and qualitative methods including Monte Carlo simulation) across all papers selected in the review, there was no standard method for completing FRAM. Hollnagel [5] indicated particular steps to conduct FRAM analysis in the first book about FRAM. Nevertheless, there was a significant variation in the number of FRAM steps used in papers (e.g., [34, 47, 50]).Similarly, this diversity was reflected in the manner in which variability is indexed within FRAM. Several researchers used the “simple solution” (e.g., [7, 13, 19, 33, 51–55]), while others use more detailed indices of variability, such as the 11 CPCs (e.g., [23, 56–60]).Given that FRAM is focused on mapping and understanding variability within complex systems, the divergent approaches to indexing variability could be a concern in relation to consistency and reliability. FRAM analysis is used to understand a sociotechnical system under specific conditions [5]. The result from one analysis cannot be generalized to another context [22, 61], and typically, it is not intended to be generalized. However, the issue of consistency of implementation of the method is relevant to evaluate the method itself. That is, like all systems analysis tools, while we may not be able to compare the outcomes because we are analysing different systems, it is still important to assess and compare the methods used to generate these outcomes. Doing so can result in refinement of the method and identification of practice that violates it assumptions.Variations in methods when implementing FRAM has implications for users of FRAM and those interpreting FRAM results in industry. It is important to consider how these different methods could be used and compared: why they are used and whether they are appropriate for the system and users in question and how they may have influenced the outcomes of tool. In addition, it may be that particular methods for indexing variability are useful for particular purposes. For example, the simple solution of indexing variability only by time and precision may be most effective for FRAM users in industry. In this context, more streamlined versions of the method may be desirable so that it is easier to collect data in the field and to interpret and use the method in practice. Alternatively, other systems for which highly precise data are already available may lend themselves to collecting and analysing additional metrics of variability and to more complex analysis methods.As reflected in our analysis, the majority of existing FRAM studies employed qualitative methods. While FRAM has been shown to be a very useful tool for analysing complex systems (e.g., [2, 19, 33, 53, 62, 63]), it has also been suggested that its qualitative nature means it cannot provide an accurate calculation of risks [64, 65]. Furthermore, due to such qualitative methods, FRAM can be very time-consuming and complex to learn and to interpret the results [56, 63, 64, 66]. Ways of simplifying FRAM while maintaining its rich analysis of complex systems require further exploration.A consequence of the reliance on qualitative methods is that current FRAM results largely rely on the knowledge of subject matter experts to inform WAI and validate WAD. The typical SMEs in existing FRAM studies include accident investigators, ATC officers, safety experts, and pilots. Compared to novice users of the system, SMEs have more of an idea of how different system components work together. At the same time, the experience of (nonnovice) system users is desirable for FRAM. While it is difficult to train day-to-day system users in the use of FRAM, methods for collecting data from their direct experience are needed to inform WAD with sharp-end operational performance.An alternative to qualitative analysis is to apply FRAM to simulated systems. Based on our analysis of the overall FRAM papers, the majority of quantitative research uses simulation data rather than data from real-working scenarios [8, 13, 45, 48, 60]. Simulation makes it possible to get a vast amount of data quickly and at low cost, while also avoiding the difficulty in data collection in the real world. Both approaches have their advantages and disadvantages. As mentioned above, qualitative methods are problematic because the time involved and the lack of quantification while quantitative methods do not take account of data from real-working scenarios.One solution might be a hybrid model, whereby in addition to using SMEs’ knowledge in constructing WAI, the FRAM net is refined by data collected from system users, who are not experts in FRAM but are experts in the system use. Examples would include flight and ground crew, front-line maintenance engineers, and ramp workers. These data can be generated from real processes and collected automatically or from system user inputs and ratings.A further potential criticism of FRAM is that it is usually applied to a relatively small part of a complex system rather than the entire system, while it attempts to understand complex systems. For example, aviation is a complex system that consists of numerous complex components, ATC being just one of them. Existing FRAM studies on ATC systems are more likely to analyse only one phase, such as the aircraft landing phase (e.g., [40]). FRAM results from the selected phase are already complex and difficult to interpret. Using the same method to analyse the whole system would be impractical in terms of time and resources. Some research has used FRAM to examine a more extensive system, such as accident reanalysis studies (e.g., [2, 6]). The functions were identified at macrolevels, where macrolevel functions could be examined further in detail to understand microlevel functions within them. In this way, FRAM could be used at different levels within a system, to build a “FRAM of FRAMs.” For example, a higher-level function may be a summary of a group of functions (a FRAM net of a lower level set of functions).Interestingly, most aviation FRAM studies did not specify which software products they used to generate FRAM nets. The analysis software should be reported in the future to facilitate replication of results, as well as an understanding of the particular constraints under which FRAM nets are constructed and analysed. This is especially the case given that there are platforms other than FMV and myFRAM with which to build FRAM nets. Microsoft Visio and Power BI are two options that provide data visualisation solutions and are compatible with other system information. Using these new tools to visualize FRAM may provide more detailed, dynamic, and interactive information. These platforms are consistent with those currently used in industry for displaying a range of other data, such as finance and marketing [67], and should be further explored for FRAM and related complex systems analysis tools.One of the limitations of this paper is that grey literature was not examined. There may be instances of FRAM use that exist in the grey literature or that have not been published. The focus of this review was to assess publicly available and published accounts of FRAM use. It would be interesting to analyse all instances of FRAM; however, companies may wish to maintain the confidentiality of their systems.


## 7. Conclusion

This study introduced and described the FRAM perspective, methods, and steps. We identified 108 existing FRAM studies from 2006 to 2019 and further analysed those applied in the aviation industry. Our findings added to our understanding of how, where, and in what systems FRAM has been used so far and its potential for understanding aviation safety occurrences. The present analysis also identified critical issues in FRAM use, which need be considered by researchers and FRAM users in industry, such as consistency, data collection methods, ease of use, and expanding the range of subsystems analysed and the analysis scope.These results highlighted a range of issues for future research. While FRAM has been applied to various aviation areas, there are many other unexplored aviation systems that may benefit from FRAM analysis. Examples include cabin crew’s operations and catering. As discussed previously, the scope of the FRAM analysis could be expanded, to cover broader systems and understand how components interact. Nevertheless, while considerable effort has been made to implement FRAM in industry, some obstacles limit their application in practice. Future studies need to adapt FRAM implementation by simplifying methods for training in how to implement and interpret FRAM and methods for collecting system user data. The potential for using new tools to represent and analyse FRAM nets in a manner consistent with existing platforms already embedded in industry needs to be considered.



---

*Source: 8898903-2020-10-21.xml*