Towards a New Safety Assurance Method for Complex Safety-Critical Systems

Know your enemy and know yourself and you can fight a hundred battles without disaster. Sun Tzu, The Art of War This essay investigates the making of timely and acceptable safety assurance decision throughout a safety-critical system’s lifecycle in a complex environment. This ‘battle’ of assuring a safety-critical system is safe enough to avoid the ‘disaster’ of accident is a long and treacherous journey. It demands a structured and persistent method to support safety analysis despite challenges from the non-deterministic and non-linear environment throughout the system lifecycle. Learning from Sun Tzu, this journey begins by searching for clarity regarding the two important questions: ‘know yourself’ by understanding the current way of conducting safety management throughout a system lifecycle; and ‘know the enemy’ by examining the potential risk and hazards cause by the unique characteristics of complexity. This is followed by a survey of possible methods that can be used to analyse and mitigate the risk under a complex system. Systems Engineering Essay Competition 2015 Page 2 of 25 Jointly organised by Temasek Defence Systems Institute Department of Industrial and Systems Engineering, Faculty of Engineering, NUS The literature reviews make three observations. First, while there are established safety management processes to analyse safety at specific milestones of the system lifecycle, there is a lack of continuity in managing and utilising the system knowledge regarding safety throughout the system lifecycle. Secondly, safety assessment must adapt to changing social-technical context especially when a system acquisition lifecycle comprises multiple distinct phases that introduce different constraints to safety assessment. Lastly, uncertainty in the complex environment can never be eliminated but there should be more efforts to minimise the surprises due to incomplete and imperfect information so as to create the confidence in the safety analysis. The paper concludes by providing a preliminary approach to develop a safety assurance method that aims to be applied throughout the system lifecycle to provide a structured and continual way to support decision making. Introduction: Where are the Hazards? As technology advances, more and more systems in domains like defence, air traffic management, railway transport, nuclear power plant, offshore drilling and health care are becoming highly networked and complex. These are described as safety-critical systems as any failure can potentially lead to the loss of life and damage to property or environment. For example, in the defence industry, networked of large-scale safety-critical systems (often refers to as Network Centric Warfare or System-of-Systems 1 [1]) have been revolutionising the applications of military technology as machines and computers are used to carry out complex and time-critical tasks. These machines are inter-connected to form larger inter-dependent systems and continuing to expand in numbers and complexity as the armed forces attempt to accomplish more challenging missions. Unfortunately, while there are established safety management processes, accidents that led to the loss of Systems Engineering Essay Competition 2015 Page 3 of 25 Jointly organised by Temasek Defence Systems Institute Department of Industrial and Systems Engineering, Faculty of Engineering, NUS life and damage to properties continue to surface from such highly complex safety-critical systems. Some of the most notable accidents and incidents in recent years include:  (Space) NASA space shuttle Challenger (1986) [2] and Columbia (2003) [3] disasters  (Military Operation) B-1B Lancer bomber friendly fire on coalition soldiers in Afghanistan (2014) [4]  (Drilling) Piper Alpha offshore oil production explosion (1988) [5]  (Maintenance) F-111 (Fuel Tank) De-seal/Reseal program leading to chemical exposure (2001) [6]  (Health-care) Overdose of radiation during Therac-25 radiation therapy (1985) [7]  (Rail) Wenzhou high speed railway collision (2011) [8] While the faults for each of these disasters are unique, one common observation is that it is extremely difficult to narrow down to a specific failure mode. Unlike complicated but linear systems where traditional safety analysis method is capable of using linear reductionist approach to deduce the root causes, the failure modes in complex safety-critical systems are different. Reiman [9] observes that effects from a complex system have “several parallel contributing factors, instead of one or few causal chains 2 as in linear systems”. Dekker [10] also believes that “the behaviour of such complex system cannot be reduced to an aggregate of the behaviour of its constituent components”. Hence, even if one root cause has been identified, decision makers may face the frustrating but real challenge of not being able to fully comprehend the full casual chains of complex relations. One good example of a complex safety-critical system is Air Traffic Control (ATC). On 12 Dec 2014, an air traffic disruption at the Swanwick ATC resulted in numerous flight cancellations across Heathrow, Gatwick and London City [11]. The System Flight Servers failed when more workstations were being brought online during the transition between 1 The System-of-Systems (SoS) refers to a set of systems that are cooperating for a common purpose while simultaneously working as independent entities. 2 A causal chain refers to the path or sequence of events that runs from a root cause to problem symptoms in the real world. Systems Engineering Essay Competition 2015 Page 4 of 25 Jointly organised by Temasek Defence Systems Institute Department of Industrial and Systems Engineering, Faculty of Engineering, NUS normal and standby operation [12]. This affected air traffic control as it was impossible for controllers to access aircraft flight plans. While the final report has yet to be released, NATS has announced that the failure is unprecedented in its 13 years of operations. The incident highlights the difficulties in managing system safety in complex environment. There will be intense pressure when failures occur and rightfully so since safety-critical systems are utilised in critical situations (e.g. ATC keeping the airspace safe) where there are severe consequences (e.g. disruption to commercial flights and passengers’ safety) when the systems fail. It is also possible that certain failures may never manifest themselves during system development as it is impossible to predict and conduct safety assessment on all operational scenarios (e.g. overloading of the ATC System Flight Servers). Active sharing of knowledge regarding safety throughout the system lifecycle can help to better anticipate and identify such ‘blind-spots’ during operation. In terms of managing safety-critical projects, social, technical and organisation tensions continue to pose safety challenges when it comes to acquiring a system in such complex environment. The following shows a sample of such concerns highlighted by Atkinson [13] and Saunders [14] from their surveys on managing project uncertainty of complex safety-critical system:  Novelty of design and technology  Diverse and conflicting stakeholders expectations and belief  Failure to anticipate concurrency of activities and capture dependency relationships  Ineffective communication and knowledge management with changing stakeholders throughout system lifecycle  Lack of continuity in personal and responsibilities when managing different interoperating systems  Incomplete and imperfect information  Lack of systematic process to capture corporate knowledge and lessons learned While the list represents uncertainty in managing projects, it is equally relevant to safety management and highlights the diversity of social-technical context that would affect the Systems Engineering Essay Competition 2015 Page 5 of 25 Jointly organised by Temasek Defence Systems Institute Department of Industrial and Systems Engineering, Faculty of Engineering, NUS effectiveness of any safety analysis. Most safety-critical systems comprise human-machine interactions and it will be important to consider explicitly the impact of changing socio-technical context during safety analysis so as to create attention to potential hazards in such circumstances. The System Lifecycle – where it begins One approach to appreciate the safety hazards facing a safety-critical system is to consider its lifecycle. A system can be considered as a “combination, with defined boundaries, of elements that are used together in a defined operating environment to perform a given task or achieve a specific purpose”[15]. Taking reference from the military, a system is realised by following the system acquisition lifecycle. A full-scale system acquisition lifecycle includes multiple stages, milestones and decision points that shape the development of a system till its operationalisation. Two examples of military acquisition lifecycles are shown in Figure 1. They are the UK MoD CADMID 3 acquisition cycle and the US DoD Defence Acquisition Process. Figure 1 Categorisation of System Acquisition Lifecycle in defence A system lifecycle has two distinct phases: development and operation. Both phases are subjected to risk and safety hazards in the complex environment but exhibit different system characteristics. The following table provides a comparison of the two phases. Characteristics System Development System Operation 3 CADMID refers to the six phases of acquisition lifecycle: Concept, Assessment, Demonstration, Manufacture, In-service and Disposal. Systems Engineering Essay Competition 2015 Page 6 of 25 Jointly organised by Temasek Defence Systems Institute Department of Industrial and Systems Engineering, Faculty of Engineering, NUS Characteristics System Development System Operation Type of processes Design, plan, production, testing, and deployment Operation, maintenance and support, ret