System Hazard Analysis of a Complex Socio-Technical System : The Functional Resonance Analysis Method in Hazard Identification

The value of characterising systems in high-risk industries as complex and socio-technical systems is increasing. Using complex and socio-technical system view-points, high profile industrial and transport accidents can be investigated by focusing attention to interactions between skilled operators, technology, and automation in geographically dispersed operations, as well as unintended design risk factors to arise from complex and non-linear interactions can be highlighted. Currently there are methods for identifying and assuring the safety of interactions between and within systems but the modeling is incomplete. However, one potentially valid method is the Functional Resonance Analysis Method (FRAM). FRAM is a qualitative approach generating a functional (rather than structural) model of the relationships between sub/systems, and has the potential to produce inputs suitable for safety assurance and risk analysis methods. This paper presents a methodology for incorporating a modified FRAM technique within a System Hazard Analysis (SHA). The application of the FRAM/SHA methodology in this case study is to explore the validity of this approach for assessing hazards arising from structural and process changes to the operational control center of an international airline, as a result of the introduction of a new software system into an existing suite of COTS software tools. This paper will explore the methodology in terms of requirements that include the creation of a new work team, the functional division of an existing operator role, and changes to system performance and safety critical processes. Keywords: Socio-Technical system, Functional Resonance Analysis Method, FRAM, System Hazard Analysis, operational control center, risk analysis, COTS requirements. 1 Introduction A significant challenge in designing and operating complex systems is the potential for unpredictable system behavior to ‘emerge’ from the complex (and often transient) interconnections that can arise under dynamic operating conditions. Limited understanding of these factors can manifest as an undefined gap between the system as intended and the system as implemented/operated (Leveson, 2011a). This arises partly as a result of the limited guidance available to support the design of effective interactions between system elements (such as human operators and technology), particularly where they are separated by time and geography, and where system function is constrained by time, conditions of information uncertainty, and decentralised control mechanisms (Vicente, 1999). In addition, the development of tools for analysing and modeling complex and non-linear system behavior (particularly for degraded performance modes), and the assessment of organisational, social, and human factor impacts, are in their relative infancy when compared to the traditional reliability driven approaches used across the system engineering lifecycle (Allenby & Kelly, 2001). This paper describes a novel systems modeling technique, the Functional Resonance Analysis Method (FRAM), modified for use within the context of a System Hazard Analysis (SHA). FRAM has been used within the Cognitive Systems Engineering domain to model and assess the complex functional interactions between elements within socio-technical systems, but primarily for accident investigation purposes (Hollnagel, 2012). However, the technique potentially has greater utility than traditional hazard identification techniques (e.g. HAZOPs variants, FMEA/FMECA approaches, etc.) in supporting risk analysis through identifying system level hazards and defining the specific (and often transient) scenarios/conditions under which they arise. The system case study highlighted in this paper is the Operations Control Centre (OCC) of a large international airline, where the modified FRAM/SHA method was applied in order to define system functional and safety requirements prior to the introduction of new industry mandated software interfacing airline operations and the Australian air traffic management system. The utility of the FRAM technique will be outlined for identifying scenario/condition-specific hazards, specifically to inform design requirements at the different levels of system performance and function, safety critical processes, and operator function and role accountabilities. 2 Complex System SHA The identification of potential hazards in complex systems is fundamental to establishing safety and reliability across the design, production, and operational stages of the System Engineering (SE) lifecycle (Bahr, 1997). A complete and comprehensive picture of the hazards present or likely to exist in a system, and determination of the escalation of hazards to become mishap or accident scenarios, supports the early establishment of performance and safety requirements for design and operational objectives. This supports a more complete and comprehensive analysis to reduce the uncertainty of any subsequent risk quantification, while providing higher levels of safety assurance (Seligmann et al, 2012). The SHA is a SE hazard analysis process applied during the design phases to assess the integration of designs. The SHA aims to identify hazards arising from the functional interfaces between subsystems, as well as the presence of latent design hazards with the potential to escalate into interrelated fault events (Ericson, 2005). Roland and Moriarty (1990) list the primary objectives of the SHA process as including consideration of: • Compliance with specified safety criteria; • Hazardous events, including failure or degradation of safety devices, controls, and safety constraints and functions; • Degradation of the system’s safety levels under normal or abnormal conditions; • The impact of design or engineering changes, and; • Human System Interfaces (HSI) including human performance errors and control functions. The SHA process may draw from a broad range of established hazard identification techniques, but will typically commence with qualitative techniques to establish the causality of credible mishap/accident scenarios before proceeding to quantification techniques (NASA, 2011). The foundation of hazard identification and analysis (HAZID), therefore, is identifying credible mishap and accident event scenarios. However, HAZID within safety critical systems is often challenging due to the stochastic effects of interactive and dynamic system complexity, and the presence of system intractability (or under-specification) as commonly found in complex and socio-technical systems (Hollnagel, 2012). Established risk analysis practice adopts the scenario driven approach to the systematic review of safetycritical systems, in order to identify potential mishap and accident scenarios (CCPS, 2008; Mannan, 2005). The development of credible scenarios enables system complexity to be reduced to discrete ‘snapshots’ of the system under a range of conditions and in different system states, where they can be modeled as deterministic functional relationships between system elements (Bossel, 2007). A scenario consists of an expected situation/characteristic sequence or combination of events, and describes a generic situation that encompasses and relates a set of reasonably probable events/situations. Khan (2001) cautions against focusing on identification of the ‘worst-case’ scenarios through risk assessment activities, as this may unnecessarily restrict the scope and coverage of the scenario set identified: he suggests that it is preferable to focus efforts on identifying “. credible accident[s] . within the realm of possibility and likely to be severe enough to cause significant damage”. An effective risk analysis process for a complex system must therefore combine appropriate hazard identification and analysis techniques in order to generate as complete and comprehensive a set of mishap and accident scenarios as possible for subsequent quantification (Cameron & Raman, 2005). Siu (1994) particularly identifies dynamic system dependencies (e.g. common-cause initiators, functional coupling, and shared equipment/components) as requiring ‘complexity decomposition’ before the modeling approaches commonly used in risk analysis can provide valid scenario quantification. In addition, Leveson (2011a) suggests that the selection of HAZID techniques should consider the system’s complexity and socio-technical characteristics, so that the HAZID process is able to describe system scenarios resulting from dependent incredible events, and/or transient system states. 2.1 Complexity System Properties Complex systems are typically dynamic, with changes in system complexity leading to changes in the system’s needs and objectives, which in turn ‘orients’ system behavior towards these new objectives. As a result, complex systems can exhibit changes in characteristic behaviors and states that are not easily observed or predicted, except through changes in state variables (Bossel, 2007). Perrow (1984) describes complexity simply as “. those of unfamiliar sequences. Or unplanned and unexpected sequences, and either not visible or not immediately comprehensible. ” Expanding upon this definition through use of Systems Theory, complexity can take a number of forms (Leveson, 2011a): • Interactive complexity between components/elements within the system, or between sub/systems. • Dynamic complexity, or system changes in relation to time. • Decompositional complexity, where the system’s structure and function are not obviously consistent/linked. • Non-linear complexity, where cause and effect are intractable or not easily described or specified. Hollnagel (2012) employs a Cognitive Systems Engineering perspective to relate dynamic complexity and non-linear complexity as a reflection of the extent to which system function is discernable or tractable: “Dynamic complexity refers to situations where cause and effect are subtle, and where the effects over time of inte