Incorporating resilience in design is important for the long-term viability of complex engineered sys- tems. Complex aerospace systems, for example, must ensure safety in the event of hazards resulting from part failures and external circumstances while maintaining e/uniFB03cient operations. Traditionally, mitigating hazards in early design has involved experts manually creating hazard analyses in a time-consuming pro- cess that hinders one’s ability to compare designs. Furthermore, as opposed to reliability-based design, resilience-based design requires using models to determine the dynamic e/uniFB00ects of faults to compare recovery schemes. Models also provide design opportunities, since models can be parameterized and optimized and because the resulting hazard analyses can be updated iteratively. While many analysis frameworks have been presented for early hazard assessment, these frameworks are di/uniFB03cult to apply without reference implementations, and most currently-available fault modelling tools are meant for the later stages of design. This paper describes fmdtools, a Python-based resilience-based design and analy- sis environment that solves these problems by enabling the designer to represent the system in the early design process, simulate the e/uniFB00ects of faults, and quantify corresponding resilience metrics. This toolkit is then demonstrated in the hazard analysis and architecture design of a multi-rotor drone. /one.pnum Introduction Resilience, the ability to prevent and mitigate hazards, is a key consideration in the design of complex engineered systems (Cottam et al., /two.pnum/zero.pnum/one.pnum/nine.pnum; Yodo and Wang, /two.pnum/zero.pnum/one.pnum/six.pnuma). In the aerospace industry, for example, it is important that aircraft adapt and recover from hazards (Choi et al., /two.pnum/zero.pnum/one.pnum/zero.pnum), airports recon/uniFB01gure runways in the event of damage (Faturechi et al., /two.pnum/zero.pnum/one.pnum/four.pnum), and supply chains that mitigate disruptions (Treuner et al., /two.pnum/zero.pnum/one.pnum/four.pnum). Because resilience factors heavily into the risk, safety, and functional reliability of a system, it is important to proactively consider resilience in the early design phase, when there is the most freedom to consider alternatives and allocate PHM features (Yodo and Wang, /two.pnum/zero.pnum/one.pnum/six.pnumb; Youn et al., /two.pnum/zero.pnum/one.pnum/one.pnum). It is often helpful to support this consideration with a cost-bene/uniFB01t analysis that can then be used to provide a coherent business case for PHM system development (Banks et al., /two.pnum/zero.pnum/zero.pnum/nine.pnum). Considering hazards in early design involves representing the system in a high-level function structure to identify hazards and develop resulting design requirements (Stone et al., /two.pnum/zero.pnum/zero.pnum/five.pnum). In the design of aircraft, this process is called functional hazard assessment and follows the ARP/four.pnum/seven.pnum/six.pnum/one.pnum guideline (ARP, /one.pnum/nine.pnum/nine.pnum/six.pnum; Allenby and Kelly, /two.pnum/zero.pnum/zero.pnum/one.pnum). Model-based functional hazard assessment has been an active research area (Krus and Lough, /two.pnum/zero.pnum/zero.pnum/nine.pnum; Kurtoglu and Tumer, /two.pnum/zero.pnum/zero.pnum/eight.pnum; Noh et al., /two.pnum/zero.pnum/one.pnum/one.pnum), with many new methods focusing on how to model di/uniFB00erent aspects of the system, such as human errors (Irshad et al., /two.pnum/zero.pnum/one.pnum/nine.pnum), dynamic behaviors, new /uniFB02ows resulting from failures (Jensen et al., /two.pnum/zero.pnum/zero.pnum/nine.pnum), and operational decision-making (Short, /two.pnum/zero.pnum/one.pnum/six.pnum). However, there has been less demonstration of how to use this information to compare design alternatives, and the research codes underlying these methods have not been shared within the research community. To enable this resilience-based design, then, there is an opportunity to develop a tool that enables one to assess the resilience of a model without re-implementing underlying data structures and fault propagation methods. The goal of the fmdtools project is to aid in the application of risk and resilience-based design frameworks 1 Subsystem TestingSystem TestingDesign Validation Existing Risk/Safety/ResilienceAnalysis Toolsfmdtools Functional Risk Analysis Network Metric Quantiﬁcation Static Fault Model FHA/FMEA Detail Design & ImplementationTask Clariﬁcation Conceptual Design Embodiment DesignBehavioral Resilience Analysis: Dynamic Fault Model Temporal Scenario-based FMEA Structural Resilience Analysis: Hierarchical Fault Model Temporal Resilience QuantiﬁcationFigure /one.pnum: fmdtools is intended speci/uniFB01cally to provide fault analysis methods that enable the consideration of risk in early conceptual design processes by providing an open-source environment for modelling, simulation, and analysis of a new system. In doing so, it seeks to enable the practical application of early model-based functional hazard assessment frameworks while explicitly enabling the consideration of resilience. This toolkit has a number of potential applications to considering PHM in aerospace systems by supporting cost-bene/uniFB01t analysis (e.g., (Holzel et al., /two.pnum/zero.pnum/one.pnum/four.pnum)), which can used to allocate resources for PHM (Youn et al., /two.pnum/zero.pnum/one.pnum/one.pnum) and assess design alternatives (e.g. prevention or recovery schemes) (Hulse et al., /two.pnum/zero.pnum/one.pnum/nine.pnuma). While most traditional risk assessment methods focus on how faults lead to system-level failures, the goal of the fmdtools project is to represent the full set of dynamic e/uniFB00ects that result from a given fault scenario, so that one can compare the e/uniFB00ect of di/uniFB00erent recovery actions. This requires advancing the modelling paradigm often used in early design. For instance, one major di/uniFB03culty in modelling failure propagation is representing the system in a way that captures the full set of e/uniFB00ects which would be caused by a fault (Hönig et al., /two.pnum/zero.pnum/one.pnum/seven.pnum). While a number of formalisms have been developed that enable this (e.g. modelica (Bunus et al., /two.pnum/zero.pnum/zero.pnum/nine.pnum), simulink/lustre (Joshi and Heimdahl, /two.pnum/zero.pnum/zero.pnum/seven.pnum), etc), these models are often computationally expensive and are more applicable for later design when one has the complete set of system behaviors. As a result, model-based functional hazard assessment methods are often de/uniFB01ned directly in code in a base language (e.g. MATLAB/Python). In this setting, it is convenient to express the system using a procedural, directed-graph representation of system behaviors where each function is de/uniFB01ned in a method and where the output /uniFB02ows of each function are used as inputs to the next function in the graph. The main problem with this representation is that it can only represent /uniFB02ow propagation that occurs in a single direction–from the source functions where /uniFB02ows originate to the sink functions where /uniFB02ows terminate. De/uniFB01ning a model in a base language also makes it di/uniFB03cult to then structure the model in a way that logically organizes faults, states, and behaviors of functions to be be easily understood, visualized, and modi/uniFB01ed. To improve on this approach, fmdtools uses a object-oriented, undirected-graph representation of system behaviors, where each function’s states, behaviors, and faults are de/uniFB01ned in a function class and a model class is used to connect adjacent functions so that behaviors can propagate in any direction through the model graph and one can easily parse the structure of the system model. The rest of this paper is organized as follows: Section /two.pnum discusses and compares fmdtools with existing fault propagation toolkits to show how it compares with the state-of-the-art as well as uses of fault injec- tion in other /uniFB01elds. Section /three.pnum the underlying model representation, fault propagation, and visualization algorithms provided in the toolkit. Section /four.pnum provides an example of using fmdtools to model on a multiro- tor drone case study with increasing /uniFB01delity in the design process and using it to inform decision making. Finally, Section /five.pnum concludes with some assessment of the usefulness of the tool and directions for future work. /two.pnum Background Fault propagation is widely used in a number of domains to assess the safety and resilience of a system of interest. As shown in Figure /one.pnum, while most current tools focus on modelling the system in the later /two.pnum Toolkit Behavioral/Causality RepresentationModel Input/Platform Availability Use in design HiP-Hops (Papadopoulos and Mc- Dermid, /one.pnum/nine.pnum/nine.pnum/nine.pnum)Dynamic simulation with failure logicSimulink, SimulationX, AADL, etc.Commercial Functional Hazard Assessment, De- sign Optimization (Papadopoulos et al., /two.pnum/zero.pnum/one.pnum/one.pnum) Rodon (Bunus et al., /two.pnum/zero.pnum/zero.pnum/nine.pnum)Behavioral constraint network with failure logicModelica-like Rodelica modelCommercial Model-based engineering process (Lunde et al., /two.pnum/zero.pnum/zero.pnum/six.pnum) Modelica fault libraries (van der Linden, /two.pnum/zero.pnum/one.pnum/four.pnum; Minhas et al., /two.pnum/zero.pnum/one.pnum/four.pnum; Gun- dermann et al., /two.pnum/zero.pnum/one.pnum/nine.pnum)Undirected behav- ioral/failure logicModelica Open SourceDesign exploration (Lattmann et al., /two.pnum/zero.pnum/one.pnum/four.pnum) OpenErrorPro (Morozov et al., /two.pnum/zero.pnum/one.pnum/nine.pnum)Probabilistic markov chainSimulink/State/uniFB02ow, UML/SysML, and AADLOpen SourceModel-based reliable system de- sign (Morozov et al., /two.pnum/zero.pnum/one.pnum/eight.pnum) SHyFTOO (Chiacchio et al., /two.pnum/zero.pnum/two.pnum/zero.pnum)Dynamic simulation with probabilistic hybrid fault treeSimulink and toolkit- de/uniFB01ned modelOpen SourceModel-based design (Chiacchio et al., /two.pnum/zero.pnum/one.pnum/nine.pnum) OpenCossan (Patelli et al., /two.pnum/zero.pnum/one.pnum/eight.pnum)Probabalistic semi- markov transitions, external simulationMATLAB toolkit-de/uniFB01ned modelOpen SourceResilient, reliable, robust design under uncertainty (Patelli and Broggi, /two.pnum/zero.pnum/one.pnum/five.pnum) IBFM (McIntire et al., /two.pnum/zero.pnum/one.pnum/six.pnum)Bond graph with failure logicText-based model, PythonOpen SourceFunctional decomposition and de- sign optimization (Hulse et al., /two.pnum/zero.pnum/one.pnum/nine.pnumb) fmdtools Dynamic un-directed be- havioral simulation with failure logicPython toolkit-de/uniFB01ned modelOpen SourceEarly resilience-based design, anal- ysis, visualization and optimization Table /one.pnum: Comparison of Model-based Fault Simulation Toolkits for Design design stages and in the veri/uniFB01cation and validation process, when there are detailed models of the system, fmdtools is meant to support early design processes. To do this, it provides analyses that support each phase of the function-behavior-structure design process commonly used in engineering design (Howard et al., /two.pnum/zero.pnum/zero.pnum/eight.pnum). In this process, one creates a functional decomposition of the tasks the system is to perform, /uniFB01nds solution principles to achieve those tasks, and then synthesizes those principles into a realized design concept. These design processes are supported by static failure-logic functional hazard assessment and network models, dynamic behavioral models, and hierarchical fault models, respectively. A number of modelling formalisms have been developed to assess the risk of hazards in engineered sys- tems, including fault trees, bayesian networks, and stochastic petri nets (a type of discrete event simula- tion) (Chemweno et al., /two.pnum/zero.pnum/one.pnum/eight.pnum). While the simpler methods (fault trees, bayesian networks) enable stronger proofs of system dependability (Chemweno et al., /two.pnum/zero.pnum/one.pnum/eight.pnum), they do not express the system resilience–the behavior over time resulting from a fault. In these situations, a discrete event simulation or continuous dynamic model is used. Discrete event simulation (e.g. (Matlo/uniFB00, /two.pnum/zero.pnum/zero.pnum/eight.pnum)) has been used to assess and de- sign resilience into systems, including maintenance (Wang et al., /two.pnum/zero.pnum/one.pnum/five.pnum) and recovery aspects (Miles, /two.pnum/zero.pnum/one.pnum/eight.pnum). Similarly, Monte Carlo techniques are often used with continuous simulation models to quantify risk of haz- ardous states (Hu, /two.pnum/zero.pnum/zero.pnum/five.pnum). While these approaches describe the underlying general approaches to simulat- ing faulty behaviors in a system, the following sections describe speci/uniFB01c toolkits that use these approaches to quantify risk and resilience. /two.pnum./one.pnum Related Work As shown in Table /one.pnum, there have been some prior e/uniFB00orts to develop generally-applicable toolkits for the design of resilience in a model-based engineering process. One major di/uniFB00erence between toolkits is the way they represent causality in the system to determine the e/uniFB00ects and propagation of faults, which has a number of potential aspects, including the types of failure paths able to be represented (through the system behavior, failure logic, state transitions or a hybrid) (Hönig et al., /two.pnum/zero.pnum/one.pnum/seven.pnum), and the nature of causality (probabilistic or deterministic). To integrate with design activities, currently-available frameworks are built around either a standardized modelling language (e.g. AADL), or an existing systems modelling tool such as Simulink or Modelica. While this enables one to use a single uni/uniFB01ed model for multiple analyses through the process, it can also be limiting if certain aspects of fault propagation are di/uniFB03cult to represent in the given formalism. Finally, while each of these toolkits are claimed to support di/uniFB00erent design processes, the design being performed is later stage embodiment design–not the early conceptual design shown in Figure /one.pnum. The main exception to this is Hip-Hops (Hierarchically Performed Hazard Origin and Propagation Studies) and IBFM (Inherent Behaviors of Functional Models), which both target early design processes. As a result, the implementation of early functional risk assessment tools has been cited as a necessity to bridge the gap between model-based hazard assessment methods in literature and their adoption in industry (Grigoleit /three.pnum et al., /two.pnum/zero.pnum/one.pnum/six.pnum). Development of the fmdtools toolkit was initiated to address limitations with the previously-developed IBFM simulator (McIntire et al., /two.pnum/zero.pnum/one.pnum/six.pnum). While IBFM was developed for the early, high-level functional de- sign of engineered systems and was able to determine the end-states of large sets of system faults, the behavioral representation was limited to /uniFB01nite states (Zero to Highest), a set of given possible operations on input and output /uniFB02ows, event-based dynamic propagation, and a given syntax for specifying failure logic. This limited its ability to express all possible behaviors that could occur in a system as a result of a given fault. Additionally, the text-based model representation made it di/uniFB03cult to parameterize models and simu- late them iteratively in an optimization algorithm. In this work, these limitations are addressed by de/uniFB01ning the model as a set of interacting Python classes with possible faults and (nominal or faulty) behaviors to simulate over a set of discrete time-steps. With this model representation, one can easily express and optimize the dynamic behavior of the system without being limited by expressiveness of the underlying modelling tool. fmdtools has additionally since expanded in scope to include not just fault propagation tools but the necessary analysis and visualization tools needed to interpret simulation results. In doing so, it comprises an early resilience-based design environment that enables quick, iterative analysis over a design model. /two.pnum./two.pnum Other Fault Modelling Tools In addition to the fault propagation toolkits mentioned above, there are additionally a number of toolkits used for fault propagation in safety assessment and for application-speci/uniFB01c resilience assessment. Safety assessment toolkits are used to verify that a given design meets desired safety and reliability criteria (Hönig et al., /two.pnum/zero.pnum/one.pnum/seven.pnum; Joshi et al., /two.pnum/zero.pnum/zero.pnum/six.pnum). Typically, these tools take a design model speci/uniFB01ed in a formal language (e.g. Simulink (Joshi and Heimdahl, /two.pnum/zero.pnum/zero.pnum/five.pnum), Lustre (Joshi and Heimdahl, /two.pnum/zero.pnum/zero.pnum/seven.pnum), Modelica (van der Linden, /two.pnum/zero.pnum/one.pnum/four.pnum), AADL (Stewart et al., /two.pnum/zero.pnum/one.pnum/eight.pnum), UML (Combemale et al., /two.pnum/zero.pnum/zero.pnum/eight.pnum)) and apply a model checker to the system description to /uniFB01nd cases where the system does not work as intended. While this process can be performed in a nominal system model, model-based safety assessment tools extend this process to assess safety by including failure modes and faulty behaviors in the system description (Joshi and Heimdahl, /two.pnum/zero.pnum/zero.pnum/five.pnum, /two.pnum/zero.pnum/zero.pnum/seven.pnum). However, because the intended purpose of these tools is to verify safety requirements post-design (see the right side of the v-model in Figure /one.pnum), they typically rely on detailed, fully-speci/uniFB01ed models of the design that are not available early in the design process (Grigoleit et al., /two.pnum/zero.pnum/one.pnum/six.pnum). Furthermore, these tools are not intended to assess resilience–the dynamic e/uniFB00ects of failures due to faults–but instead assess the safety or reliability of the system (i.e. an overall fault tree or failure probability). As a result, they are not applicable to the design of resilience in the early design process when the system has not been fully speci/uniFB01ed and the designer is interested in assessing the system’s dynamic response to faults. Fxn 3Fxn 2 Fxn 1 Fxn 4Network Model Fxn 3Fxn 2 Fxn 1 Fxn 4Fxn1 Fxn2 Fxn3Fxn1 Fxn2 Fxn3Fault F1 F2 F3Fxn1Fxn2 Fxn3Fxn4 Fxn1Fxn2 Fxn3Fxn4 Fxn4Fxn2 Fxn3Fxn1 Fl 1 Fl 2 Fl 3 t=0111 t=1101 t=2101States over timeResult T able/FMEA Hierarchical ModelDynamic ModelStatic ModelBehavior Plots Graph V iews Network SimulationDynamic SimulationFault InjectionQuadrature Sampling E(V)Design SimulationAnalysis Figure /two.pnum: The fmdtools design, simulation, and analysis environment. /four.pnum Fault injection is used widely in software and hardware engineering to assess a computer system’s ability to manage the di/uniFB00erent types of faults. Existing simulators vary by domain (e.g. distributed systems (Martins et al., /two.pnum/zero.pnum/one.pnum/three.pnum), servers (Kollárová, /two.pnum/zero.pnum/one.pnum/four.pnum), stand-alone systems) and type of faults (hardware-originated (Geor- gakoudis et al., /two.pnum/zero.pnum/one.pnum/nine.pnum; Schirmeier et al., /two.pnum/zero.pnum/one.pnum/five.pnum) or software components/algorithms (Goldstein et al., /two.pnum/zero.pnum/two.pnum/zero.pnum; Arribas et al., /two.pnum/zero.pnum/one.pnum/eight.pnum)), though generic simulators also exist (Winter et al., /two.pnum/zero.pnum/one.pnum/five.pnum). One of the advantages of software (as opposed to hardware/systems) engineering is that this testing can be iteratively performed on a running prototype of the system at a low computational cost, and as a result, these tools do not operate on models of the system as is needed in engineering design. There are additionally a number of application-speci/uniFB01c resilience assessment toolkits that interface with speci/uniFB01c system simulators to model the resilience-related attributes for that domain. For example, in- tegrated circuits have well-de/uniFB01ned properties that have led to a number of specialized fault simulation algorithms, which have since been implemented in software (May and Stechele, /two.pnum/zero.pnum/one.pnum/two.pnum; Niermann et al., /one.pnum/nine.pnum/nine.pnum/two.pnum). Infrastructure often has speci/uniFB01c hazards to assess, which has resulted in tools to consider natural disasters for cities (Fraser et al., /two.pnum/zero.pnum/one.pnum/six.pnum; McKenna, /two.pnum/zero.pnum/one.pnum/one.pnum) and cyber-physical threats in smart grids (Wadhawan and Neuman, /two.pnum/zero.pnum/one.pnum/seven.pnum). Finally, assessing the hazards of autonomous vehicles in a real system is both costly and hazardous (Gambi et al., /two.pnum/zero.pnum/one.pnum/nine.pnum), which has led to the development of a number of simulators that en- able one to try di/uniFB00erent policies to approaching hazards which the vehicle will encounter (Jha et al., /two.pnum/zero.pnum/one.pnum/eight.pnum, /two.pnum/zero.pnum/one.pnum/nine.pnum). While these toolkits can assess resilience in speci/uniFB01c design contexts with high /uniFB01delity, they are not applicable to a generic systems design context. 1. Establish overall structures of functions and ﬂows in model Network Analysis, Graph V isualization Static Propagation Model2. Associate Functions with modes and failure logicNetwork Model Quantify fault probabilities and end-state costs3. Associate Functions with behaviors and Model with timerangeFault Propagation Analysis and Visualization Functional Hazard Assessment Static scenario- based FMEA Nominal and Non- nominal fault behavior plots/tables Quantify fault rates, operational phases, dynamic ef fects costs. Choose sample approachDynamic Propagation ModelDynamic scenario- based FMEA4. Associate Functions with Component Architectures Hierarchical Propagation ModelExpected resilience quantiﬁcation and visualization Figure /three.pnum: Fault model types and analyses enabled by fmdtools. Note that since model types build on each other, the analyses from less detailed model types (e.g. static propagation models, network models) can apply to more detailed model types (e.g. dynamic propagation models, hierarchical propagation models). /three.pnum Methods and Algorithms The fmdtools toolkit aims to provide a design, analysis, and simulation environment that enables the de- sign of resilience into a system. To accomplish this, it provides a number of tools to represent, simulate, and analyze the system as it progresses through the design process, as shown in Figure /two.pnum. As a result, it can accommodate a number of modelling and analysis use-cases to progress from the early, abstract represen- tations of the design (e.g. network and static propagation models) to more detailed representations of the /five.pnum 1 1EE 1Import EE EE_1 V oltage = V oltage_In Current_In = EE_1 Current faults: Voltage_In = 0 EE 2Transport EE EE 2 V oltage = EE 1 V oltage * a EE 1 Current = EE 2 Current * b if EE 2 current >10, a=0 faults: a=0, b=0 Export EE EE 2_ Current = EE 2_V oltage/R faults: R=0, R=inf 111Wire Model Functions: Import EE, T ransport EE, Export EE Flows: EE 1, EE 2 Graph, Bipartite Graph Cost = 10 *Nominal Power - EE 2 Current * V oltage 1 EE 2 Voltage CurrentWire States: a, b Faults: a=0, b=0 EE 2 V oltage = EE 1 V oltage*a EE 1 Current = EE 2 Current*b if EE3 current > 10, a=0, b=0EE 1 Voltage CurrentTransport EE Flows: EE_1, EE_2 Components: Wire Wire behavior(EE 1, EE 2)Import EE Flows: EE_1 States = V oltage_In, Current_In Faults: V oltage_In = 0 EE_1 V oltage = Input_V oltage Current_In = EE_2 CurrentExport EE Flows: EE 2 States: R Faults: R=0, R=inf EE 2_ Current = EE 2_V oltage/Rmodel class diagramdesign modelFigure /four.pnum: Example model representation. A Model is composed of function objects with internal states and faults, (optional) instantiating component objects and relationships to /uniFB02ow objects. system structure (e.g. dynamic and hierarchical propagation models) and behaviors in the same modelling environment, as shown in Figure /three.pnum. The full implementation of this work is provided in a publicly-available repository, along with examples and documentation at github.com/DesignEngrLab/fmdtools or (Hulse et al., /two.pnum/zero.pnum/two.pnum/zero.pnum). While an exhaustive description of every method and class is out of the scope of this paper, this section will discuss some of the underlying concepts and structure of the toolkit. The fmdtools toolkit is organized into di/uniFB00erent modules used for model representation, simulation, and analysis. The modeldef module provides the classes to de/uniFB01ne a system model from functions, /uniFB02ows, and components as well as a fault sampling approach for resilience quanti/uniFB01cation. The faultsim module then provides methods for propagating faults in a model and quantify network metrics. Finally, the resultdisp module provides a number of methods to pro- cess and visualize simulations of the model, including behaviors over time, FMEA tables, fault graphs, and heatmaps. /three.pnum./one.pnum Model Representation In this work a system consists of functions (modules that perform a task), components (speci/uniFB01c solutions to a function), and /uniFB02ows (variables) that are connected with each other in a bipartite graph. In a model (itself de/uniFB01ned by a user-de/uniFB01ned class), functions, /uniFB02ows, and components are represented by objects instantiated from user-de/uniFB01ned classes that are connected by a graph, as shown in Figure /four.pnum for a model of a wire. In this representation, each function (e.g., Transport EE) consists of its associated /uniFB02ow objects (e.g., EE /one.pnum, EE /two.pnum), internal state variables, set of faults, behavior methods, and constituent component objects (if a component representation is used). Flows are in turn de/uniFB01ned by dictionaries of states with corresponding values and components are de/uniFB01ned by internal state variables, a set of faults, and behavior methods. Model objects are then composed of their constituent functions, /uniFB02ows, and components as well as a graph object used to track the connections between functions and /uniFB02ows and a classi/uniFB01cation method used to quantify the costs of a fault scenario propagated in the model. This model is constructed in fmdtools by de/uniFB01ning a subclass that extends the Model class to represent a system of interest. This model class is constructed by de/uniFB01ning the simulation parameters (e.g. units, timesteps, starting and ending simulation times, design parameters), adding each /uniFB02ow in the model, adding /six.pnum each each function in the model and connecting them with /uniFB02ows to construct the model graph, and creating a classi/uniFB01cation method which determines the rate, cost, and expected cost of a given scenario given the results and parameters for that simulation–the end-state of functions and /uniFB02ows (e.g. values and faults present), the scenario properties (e.g. rates, faults, etc.), and the history of model states (values for /uniFB02ows over time). By instantiating this class, one can then use the resulting object to model the evolution of system states over time for a given set of de/uniFB01ning parameters. /three.pnum./one.pnum./one.pnum Functions In design, functions represent the high-level tasks performed by the system (Pahl and Beitz, /two.pnum/zero.pnum/one.pnum/three.pnum). In fmd- tools, the FxnBlock class is used to represent these functions, which constitute the main building block of the system model de/uniFB01ning faults and system behaviors. To use this class, users de/uniFB01ne a subclass for each function which uses inherited FxnBlock attributes to represent the properties of the function. At its most basic, a user-de/uniFB01ned function class is de/uniFB01ned by the /uniFB02ows going in and out of the function, a behaviour method which relates values of input /uniFB02ows to values of output /uniFB02ows, and a set of faults which modify the input-output relationship de/uniFB01ned in the behavior method. However, more attributes can be de/uniFB01ned, in- cluding states (internal variables of the function tracked in the model history), conditional fault methods de/uniFB01ning input/output /uniFB02ows or function states result in faults, timers that can be used to express delays in behaviors, and components, described in Section /three.pnum./one.pnum./three.pnum. To enable expected resilience quanti/uniFB01cation, each fault to inject in the function can be associated with a probability model that expresses the likelihood of the fault occurring. This probability model is de/uniFB01ned by a function-wide rate, the proportion (or rate/probability) of faults resulting from the mode, an opportunity vector expressing the relative likelihood of a fault occurring at a speci/uniFB01c phase of operation, and a speci/uniFB01- cation of whether these values are a rate (with units) or a probability. Each fault can additionally be given a cost of repair that can be used to calculate the costs of fault scenarios. /three.pnum./one.pnum./two.pnum Flows Flows represent the states (e.g. energy, material, or signal) with which the functions interact to achieve the overall goal of the system. In fmdtools, the Flow class is used to represent these states, which can either be extended in a user-de/uniFB01ned subclass (if there are special properties the designer wishes to represent) or instantiated using a dictionary with the name of the /uniFB02ow, name of the values characterizing the state of the /uniFB02ow, and initial quantity for each value. Because of the undirected nature of the model graph and associative relationships between functions and /uniFB02ows, /uniFB02ows are accessible (i.e. values can be changed) in all connected functions. /three.pnum./one.pnum./three.pnum Components While functions represent the task a system performs, components can be used to represent realizations of that function. Often in risk or resilience-based design, one is interested in designing component archi- tectures which will ful/uniFB01ll an overall function, even when an individual component fails (Youn et al., /two.pnum/zero.pnum/one.pnum/one.pnum). To represent this, fmdtools uses the Component class, which shares many attributes with the Function class, including internal states, a behavior method, and a set of fault modes. Similarly, to use the Component class in a model, one must de/uniFB01ne a subclass for the modelled component with its own states, fault modes, and behavior method. However, unlike the FxnBlock behavior method (which acts on FxnBlock attributes and does not return anything), Component behavior methods explicitly take the inputs of the component behavior as input and return outputs of the behavior as output. This is done to enable the designer to relate the inputs and outputs of the components in the architecture ful/uniFB01lling a function with each other and function states in the behavior methods of its corresponding function. /three.pnum./two.pnum Resilience Simulation fmdtools has two main approaches for assessing the resilience of a model: quantifying network metrics of the system architecture and propogating faults in the system model. Network metric quanti/uniFB01cation, de- scribed in Section /three.pnum./two.pnum./one.pnum and implemented in the networks submodule enables some consideration of the structural resilience of the system before specifying the fault logic in the system. Fault propagation, de- scribed in Sections /three.pnum./two.pnum./two.pnum and /three.pnum./two.pnum./three.pnum can then be used to assess the resilience of a model given the model has /seven.pnum faults to propagate in each function (or the function of interest) and the functions each have the necessary fault logic and/or behaviors. /three.pnum./two.pnum./one.pnum Network Metric Quanti/uniFB01cation The network metric quanti/uniFB01cation tools in fmdtools enable the early assessment of the failure toler- ance and resilience of a design. Network analysis is an emerging early design methodology for predict- ing the likely failure tolerance of a design without the need for high /uniFB01delity models (Haley et al., /two.pnum/zero.pnum/one.pnum/six.pnuma; Mehrpouyan et al., /two.pnum/zero.pnum/one.pnum/three.pnum). In this approach, engineered systems are represented as networks in which nodes represent functions, parameters, or components depending on the speci/uniFB01c network formalism. In short, network analysis enables analysis of the topology that emerges from the connectivity between sys- tem elements. The representation of connectivity between system elements is similar to that of a design structure matrix (DSM), and network theory enables visualization and powerful analyses of emergent net- work properties. Networks are used for both a priori assessment of resilience properties as well as for quantifying the network’s response to simulated faults. Resilience Properties of Networks Known relationships between structure and failure tolerance (Boc- caletti et al., /two.pnum/zero.pnum/zero.pnum/six.pnum) in complex networks enable the a priori assessment of a network’s resilience properties prior to simulation. The structure of a network is intrinsically related to its functionality; structural vulner- abilities are therefore relatable to loss of functionality. In networks, failure tolerance is typically studied by attacking or removing nodes and measuring the resultant degradation of the network structure. That is, much as loss of functionality in one component in an engineered system leads to decreased overall perfor- mance, degradation or removal of one node in a network leads to an alteration of the network’s topology. In this way, a network’s resistance to failure is relatable to an engineered system’s resistance to failure. Cer- tain networks are more prone to degradation due to node removal than others. Likewise, certain nodes are more prone to causing degradation than others. In component networks, failure or removal of a compo- nent node implies loss of functionality in that component, and the analysis therefore relates to the e/uniFB00ects of the failure of that function. Network analysis of the e/uniFB00ects of function node failure is complementary to, for example, FMEA in that it enables an assessment of the e/uniFB00ects of a failure and its severity. A common method for characterizing a network is studying its degree distribution. In an undirected net- work, the degree of a node is its the number of connections (edges). Nodes with high degree (hubs) tend to be more critical in retaining a network’s functionality. A network’s degree distribution is a histogram of the degrees of all nodes in the network. Degree distributions that follow a bell-curve tend to be more vul- nerable to random node removal, whereas degree distributions that follow a power law tend to be more vulnerable to targeted node removal (Barabási, /two.pnum/zero.pnum/zero.pnum/nine.pnum) (i.e. removal of a hub). High degree nodes are iden- ti/uniFB01able using find_high_degree_nodes . Degree distributions are provided using degree_dist . The modularity and community structure of a network also have signi/uniFB01cant bearing on the network’s failure tolerance. Modules, or communities, are tightly coupled groups of nodes. The modularity of a network, the degree to which a network exhibits a modular structure, is typically measured using Q-modularity (Newman, /two.pnum/zero.pnum/one.pnum/zero.pnum). Nodes that connect modules – bridging nodes – are functionally important to a network’s failure tolerance (Walsh et al., /two.pnum/zero.pnum/one.pnum/eight.pnum). This is comparable to, for example, identifying high severity failures in FMEA. Networks with high modularity have been found to be less robust overall (Walsh et al., /two.pnum/zero.pnum/one.pnum/nine.pnum). find_bridging_nodes is provided to identify bridging nodes in the network model. calc_modularity is provided to compute the modularity of the network model. Average shortest path length (ASPL) is a measure of the e/uniFB03ciency of the spread of information through a network. Under attack, networks with low ASPL are more likely to retain short to moderate length paths between any given pair of nodes, whereas networks with high ASPL are more likely to disintegrate signif- icantly under attack. ASPL is de/uniFB01ned as the mean of the sum of all edge weights along the shortest path between each pair of nodes in a network and is available as calc_aspl . Simulation-Based Analysis of Network Resilience Rather than using a network’s structure to predict its response to failure, it is possible to use simulation-based approaches to network analysis. First, robustness coe/uniFB03cient simulates the e/uniFB00ect of failure in the network. This approach, implemented as calc_robustness_coefficient , measures the changing size of the largest connected component of a network during successive node re- moval (Viana et al., /two.pnum/zero.pnum/zero.pnum/nine.pnum). A second simulation-based approach is an SFF (susceptible-failed-/uniFB01xed) epi- demic spreading model, which is able to explicitly represent node recovery (Mehrpouyan et al., /two.pnum/zero.pnum/one.pnum/three.pnum). /eight.pnum Figure /five.pnum: Illustration of static fault propagation. Function behavior methods are iteratively run in a list until the states of the system no longer change. Rather than attacking or removing nodes as in the robustness coe/uniFB03cient, this model considers nodes to be in a susceptible, failed, or /uniFB01xed state. Failed nodes may cause susceptible nodes to fail, similarly to infected persons spreading an epidemic. After a node is /uniFB01xed, it is unable to fail again by the same cause (immunity). The SFF model is available as sff_model . /three.pnum./two.pnum./two.pnum Fault Propagation Propagation of faults in a model has two major aspects: static fault propagation and dynamic fault propa- gation. Static fault propagation is the process of determining the immediate e/uniFB00ects of a fault in a system, as illustrated in Figure /five.pnum. First, all of the behavior methods are run. If a new value occurs in one of the functions (e.g., because of fault injection) or its associated /uniFB02ows, that function and functions adjacent to the changed /uniFB02ows are added to a list of functions to update. The behavior functions for those functions are then run and new functions to update are again added if they receive new input values. This process is run iteratively until the system reaches a stable end-state, if a stable end-state is possible. Thus, one necessary property for fault models is stable fault behavior–behaviors in one function should not change behaviors in other functions that will in turn change the original behaviors in the original function repeatedly, indef- initely. As shown in Figure /four.pnum, functions have associative relationship with /uniFB02ows, meaning functions have full access to (and can change the values of) the states of both “input” and “output” /uniFB02ows. Because the propagation of system states is undirected, functions have the ability to propagate new system states to any other func- tions in the graph–not just the function that would be placed “next” in the sequence of tasks to perform. However, because of the undirected system representation, con/uniFB02icts between function behaviors can oc- cur when di/uniFB00erent functions specify di/uniFB00erent values for the same /uniFB02ow state, resulting in a non-convergent system state at a given time-step. This must be avoided in model setup, which can be achieved by rep- resenting /uniFB02ows with states that propagate forward through the model graph (i.e. “e/uniFB00ort” variables such as voltage or potential in a bond graph representation) and states that propagate backwards through the model graph (i.e. “/uniFB02ow” variables such as current or rate). Dynamic fault propagation is the evolution of states in the model over time necessary to quantify resilience as a time-dependent property of a system. The implementation of dynamic fault propagation used here is illustrated in Figure /six.pnum. As shown, the static propagation procedure is iteratively run over a set of time-steps from fault injection time to the end of the simulation time. To fully assess resilience, a history is kept of all of the states of the model (/uniFB02ow values, function state values, faults in functions, etc.) over the set of time- steps. Based on a simulation like this, one can then quantify metrics for the simulation such as dynamic costs, recovery time, or worst state over time. /nine.pnum Figure /six.pnum: Dynamic fault propagation. A model is iteratively updated at each discrete time-step from fault injection to the end of simulation. /three.pnum./two.pnum./three.pnum Fault Injection Depending on the scope of the analysis, one might be interested in injecting faults in di/uniFB00erent ways. Before injecting faults, it is important to determine that the model performs as expected by simulating the system without any faults. Then, while setting up faults and fault behaviors (and in systems with single faults), one can propagate a single fault at a given simulation time to verify that the simulated behavior matches the expected behavior for that fault. Once faults are encoded, the list of faults can be propagated in the system at times de/uniFB01ned in the model. While this approach lets one see the consequences of faults injected at set times, it may not be for calculating expected resilience metrics, since it neglects joint fault scenarios and when in time faults are most probable. To quantify the mathematical expectation of fault-injection based resilience models, the SampleApproach class can be used to de/uniFB01ne the set of fault scenarios to propagate in the model, as illustrated in Figure /seven.pnum. This class uses the dynamic probability model set up in the model, functions, and components, along with user-de/uniFB01ned parameters to create a list of fault scenarios which will be used to represent the statistical expectation of the de/uniFB01ned faults. This approach can be de/uniFB01ned over the set of faults to include, the number of joint faults to inject and the probability model for the joint faults (e.g. assuming independence or a conditional probability), and the times to inject the faults at. The set of injection times is determined by two main parameters: the phases de/uniFB01ned in the model (and opportunity vectors for each fault in the probability model), and the set of times within each phase. Within each phase, these times can be speci/uniFB01ed as every discrete timestep, an evenly-spaced approach with a set number of points, a randomly-spaced approach with a set number of points, or an approach using a given quadrature de/uniFB01ned in the quadpy software package (Schlömer et al., /two.pnum/zero.pnum/two.pnum/zero.pnum). Additionally, Sample Approaches can be re/uniFB01ned post-hoc based on a set of simulation results to represent the behaviors with a small set of sample points. Using these approaches, one quantify expected metrics iteratively with as few fault simulations as possible. /three.pnum./three.pnum Resilience Analysis and Visualization Using the models de/uniFB01ned in Section /three.pnum./one.pnum and simulations in Section /three.pnum./two.pnum, one can then perform analyses on the results. fmdtools provides a number of di/uniFB00erent methods using existing Python libraries, including mat- /one.pnum/zero.pnum Cost Mode RateUsage LossRepair Loss Phase 2Simulation T ime Phase 1Phase 3 modesSample T imesDisruption LossFigure /seven.pnum: Injecting faults according to a fault sampling approach. plotlib (Hunter, /two.pnum/zero.pnum/zero.pnum/seven.pnum), networkx (Hagberg et al., /two.pnum/zero.pnum/zero.pnum/eight.pnum), and pandas (pandas development team, /two.pnum/zero.pnum/two.pnum/zero.pnum) to makes sense of the fault behaviors modelled in the system. To assist with this analysis and visualization, the results of the simulations are processed to summarize the state of di/uniFB00erent aspects of the system as nominal or faulty. This process results in three categorizations for functions, /uniFB02ows, and components: nom- inal, when the data structure behaves as it does in the nominal state; degraded, when the data structure has a di/uniFB00erent behavior than it does in the nominal state; and faulty, when a component or function has a fault. This approach to result processing enables high-level visualization of the status of model structures without the user de/uniFB01ning bounds or conditions for each variable to be listed as faulty or degraded. Using this representation, one can make a number of plots of the model graph structure, system behaviors over time, and tabular summaries of results. /three.pnum./three.pnum./one.pnum Graph Plots To visualize the propagation of faults in the system, fmdtools provides a number of methods that dis- play a graph view of the system using matplotlib (Hunter, /two.pnum/zero.pnum/zero.pnum/seven.pnum), networkx (Hagberg et al., /two.pnum/zero.pnum/zero.pnum/eight.pnum), and netgraph in the graph sub-module. Graph views of the system enable one to see the structure of the model as well as desired states or properties of the functions and /uniFB02ows. Two main graph representations can be plotted: a default graph representation where the /uniFB02ows are plotted as edges between function (which are nodes) and a bipartite graph representation where both functions and /uniFB02ows are nodes and edges are the associative relationships between them. To visualize the model graph in an intuitive layout, methods calling netgraph ’sInteractiveGraph are provided which enable one to place nodes manu- ally (rather than relying on an algorithmic layout). While the default representation is more intuitive to interpret–especially for simple systems–the bipartite representation often makes a better use of space– especially when a /uniFB02ow is connected to more than two functions–because there is less more freedom to ensure that edges do not overlap. Using the graph view, one can then visualize graph metrics as shown in Figure /eight.pnum, the state of the model at the end-state or a particular time (or set of times) in the model history as shown in Figure /one.pnum/one.pnum, and various model run statistics de/uniFB01ned by heatmaps (e.g. expected degradation time, maximum number of faults, etc.). These visualizations give one a view of how faults and behaviors propagate at the system level. /three.pnum./three.pnum./two.pnum Dynamic plots When modelling a dynamic system, it is often important and necessary to plot particular states over time in order to see how the behavior evolves over time. Methods in the plot sub-module use matplotlib (Hunter, /two.pnum/zero.pnum/zero.pnum/seven.pnum) to show the evolution of chosen states of the model over time, with (if the simulation was a run of a fault scenario) faulty states overlaid over the nominal states over time to aid assessment of the fault-induced behavior, as shown in Figure /one.pnum/two.pnum. In addition to modelling system behavior in a particu- lar modelling scenario, time-based plots also have the ability to visualize the cost responses given by the /one.pnum/one.pnum ASPL Modularity Robustness Coe/uniFB03cient /one.pnum./four.pnum/four.pnum /zero.pnum./one.pnum/two.pnum /nine.pnum/five.pnum./eight.pnum/five.pnum Table /two.pnum: Network metrics for default (function) network representation of example drone model. simulations at each injection time. This plot, shown in Figure /one.pnum/three.pnum, can be used visualize how the sample ap- proach de/uniFB01ned in Section /three.pnum./two.pnum./three.pnum approximates the expected resilience costs by showing the rate over time for a particular fault, as well as the modelled cost and quadrature weight for each sample point. Since these plots are plotted in matplotlib , well-known commands and interfaces can be used to edit and save the plots. /three.pnum./three.pnum./three.pnum Tables Finally, it is often helpful to be able to view the results of fault simulations in tabular form. While simulation results are typically returned as nested dictionaries, fmdtools provides methods to view this information as apandas (pandas development team, /two.pnum/zero.pnum/two.pnum/zero.pnum) table to enable results processing and display. Based on the processed results, one can also make tabular summaries of simulations, such as the number of functions and /uniFB02ows degraded over time or in a particular simulation. Tables are most helpful for summarizing the results of a set of simulations, where they can provide an FMEA-style assessment of the functions and /uniFB02ows a/uniFB00ected as well as the rate, cost, and expected cost of each fault simulation, as shown in Table /three.pnum, which can be generated to delineate between or summarize fault e/uniFB00ects over each phase. Since these tables are implemented in pandas , existing interfaces can then be used to display and/or save results (e.g. as a .csv ). /four.pnum Example - Drone Model This section illustrates how one can use the fmdtools software package to aid the conceptual design of a real system by providing analyses that increase in /uniFB01delity and detail with the design process. A number of examples are provided in the repository, including a conceptual model of a pump, a dynamic modelling of virus spreading, a human-operated tank system, and a static model of an electric power system. This example considers the design of a multi-rotor drone which must /uniFB02y to a given location and return to its destination. The functional model of this system is shown in Figure /one.pnum/one.pnum, which includes the rotor lines, structure, electrical power source and distribution, and path planning of the system. In this example, we /uniFB01rst quantify metrics about the system structure, then use a static representation to generate a high- level FMEA and visualize fault propagation, then create a dynamic version of the model to visualize fault behaviors over time and quantify the e/uniFB00ects of injecting faults at di/uniFB00erent simulation times, and /uniFB01nally use a hierarchical model to compare the dynamic fault responses and resulting resilience of di/uniFB00erent component architectures. /four.pnum./one.pnum Network Representation and Analysis First, the model is analyzed using network metrics and algorithms. In fmdtools , it is possible to analyze various network representations of the model, although only one network representation will be shown in each step. Each network analysis function has options to analyze the default (function) network, the bipar- tite (function-/uniFB02ow) network, the parameter network, or the component network. The bipartite network is treated as a unipartite-like network for analysis (Haley et al., /two.pnum/zero.pnum/one.pnum/six.pnumb). Analysis of the various network perspectives provides a more complete understanding of the model’s resistance to failure. The network metrics for the function network are given in Table /two.pnum. Low (close to zero) modularity indicates a high degree of interconnectivity and has been correlated with high robustness (Walsh et al., /two.pnum/zero.pnum/one.pnum/nine.pnum). High robustness coe/uniFB03cient and low ASPL indicate high robustness to random node failure. High degree nodes are highlighted in red in Fig. /eight.pnum. Nodes with high degree, or hubs, are more likely to cause signi/uniFB01cant perfor- mance degradation if in a fault state. Based on this analysis of the system topology, we can conclude that the functions with the most opportunity to a/uniFB00ect other functions at a topological level are path planning, control systems, and the structure of the drone, since these functions have the most connections with other subsystems. Identi/uniFB01cation of these vulnerabilities is similar to the identi/uniFB01cation of high severity fail- ures within an FMEA. The degree distribution of the function network is presented in Fig. /nine.pnum. The relatively /one.pnum/two.pnum StoreEE DistEE AffectDOFCtlDOFPlanpath Trajectory EngageLand HoldPayloadViewEnvHigh Degree Nodes (90th Percentile)Figure /eight.pnum: Visualization of high degree nodes in default (function) network representation of example drone model. 2 3 4 5 6 Degree0123FrequencyDegree distribution Figure /nine.pnum: Degree distribution of default (function) network representation of example drone model. /one.pnum/three.pnum 0 2 4 6 8 10 12 14 Time steps012345678Number of nodes SFF model Susceptible Failed FixedFigure /one.pnum/zero.pnum: SFF model applied to function network representation of example drone model. StoreEE DistEE AffectDOFCtlDOFPlanpath Trajectory EngageLand HoldPayloadViewEnv Force_ST Force_Lin Force_GR Force_LGEE_1 EEmotEEctl Ctl1DOFsEnv1 Dir1 break mechbreak noctl noloc lost crash break break Propagation of faults to AffectDOF: Mechbreak at t=NA Figure /one.pnum/one.pnum: Static fault e/uniFB00ects to the motor breaking: the drone crashes. homogeneous degree distribution in Fig. /nine.pnum indicates that the network is not particularly robust to failure of critical nodes. The SFF model for the function network is provided in Fig. /one.pnum/zero.pnum. This model demonstrates the system’s resilience (based on parameter topology) to a cascading failure, given a user de/uniFB01ned failure rate, /uniFB01x rate, and start node (/uniFB01rst node to fail). If various design alternatives are being considered, their relative resilience can be compared using the SFF model. /four.pnum./two.pnum Static Representation and Analysis To identify how faults lead to failures and begin to quantify risk in the system, the model is elaborated with /uniFB02ow attributes, function states, and failure logic, creating a static propagation model. As modelled in this system, deviations in the input of the Trajectory function block lead to a crash, which in turn propagates faults through the structure to initiate faults in the rest of the functions. Using this model (and an underlying fault probability model), one can create a FMEA-like table of fault e/uniFB00ects, rates, and costs, as shown in Table /three.pnum, to show which faults have the highest impact on the design. As shown, most faults in this model lead to a crash, making the chosen rate the driving factor of expected cost. One of the faults under consideration in this model is a mechanical break causing the A/uniFB00ectDOF function to lose the ability to control the degrees of freedom of the drone. This fault is visualized in Figure /one.pnum/one.pnum, showing how fault leads to a crash and in turn faults in the other functions. This fault will be used to motivate analysis and design through the rest of this example. /one.pnum/four.pnum Fault Degraded Functions Degraded Flows Rate Cost Exp. Cost StoreEE nocharge StoreEE, DistEE, CtlDOF, Planpath, Trajectory... Force_ST, Force_Lin, Force_GR, Force_LG, EE_/one.pnum... /one.pnume-/zero.pnum/five.pnum /one.pnum/eight.pnum/three.pnum/three.pnum/zero.pnum/zero.pnum /one.pnum/eight.pnum/three.pnum/three.pnum/zero.pnum/zero.pnum Planpath degloc DistEE, CtlDOF, Planpath, Trajectory, EngageL... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /eight.pnume-/zero.pnum/six.pnum /one.pnum/nine.pnum/three.pnum/zero.pnum/zero.pnum/zero.pnum /one.pnum/five.pnum/four.pnum/four.pnum/zero.pnum/zero.pnum DistEE short DistEE, CtlDOF, Planpath, Trajectory, EngageL... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /three.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/six.pnum/zero.pnum/zero.pnum/zero.pnum /five.pnum/five.pnum/eight.pnum/zero.pnum/zero.pnum A/uniFB00ectDOF ctlbreak DistEE, A/uniFB00ectDOF, CtlDOF, Planpath, Trajecto... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /two.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/four.pnum/zero.pnum/zero.pnum/zero.pnum /three.pnum/six.pnum/eight.pnum/zero.pnum/zero.pnum A/uniFB00ectDOF ctlup DistEE, A/uniFB00ectDOF, CtlDOF, Planpath, Trajecto... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /two.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/three.pnum/five.pnum/zero.pnum/zero.pnum /three.pnum/six.pnum/seven.pnum/zero.pnum/zero.pnum DistEE break DistEE, CtlDOF, Planpath, Trajectory, EngageL... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /two.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/three.pnum/zero.pnum/zero.pnum/zero.pnum /three.pnum/six.pnum/six.pnum/zero.pnum/zero.pnum CtlDOF noctl DistEE, CtlDOF, Planpath, Trajectory, EngageL... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /two.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/three.pnum/zero.pnum/zero.pnum/zero.pnum /three.pnum/six.pnum/six.pnum/zero.pnum/zero.pnum A/uniFB00ectDOF short DistEE, A/uniFB00ectDOF, CtlDOF, Planpath, Trajecto... Force_ST, Force_Lin, Force_GR, Force_LG, EE_/one.pnum... /one.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/six.pnum/two.pnum/zero.pnum/zero.pnum /one.pnum/eight.pnum/six.pnum/two.pnum/zero.pnum A/uniFB00ectDOF mechbreak DistEE, A/uniFB00ectDOF, CtlDOF, Planpath, Trajecto... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /one.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/three.pnum/five.pnum/zero.pnum/zero.pnum /one.pnum/eight.pnum/three.pnum/five.pnum/zero.pnum A/uniFB00ectDOF openc DistEE, A/uniFB00ectDOF, CtlDOF, Planpath, Trajecto... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /one.pnume-/zero.pnum/six.pnum /one.pnum/eight.pnum/three.pnum/two.pnum/zero.pnum/zero.pnum /one.pnum/eight.pnum/three.pnum/two.pnum/zero.pnum Planpath noloc Planpath, Trajectory Ctl/one.pnum, DOFs, Dir/one.pnum /two.pnume-/zero.pnum/six.pnum /six.pnum/zero.pnum/zero.pnum/zero.pnum/zero.pnum /one.pnum/two.pnum/zero.pnum/zero.pnum/zero.pnum CtlDOF degctl CtlDOF Force_GR, Force_LG, Ctl/one.pnum, DOFs /eight.pnume-/zero.pnum/six.pnum /one.pnum/zero.pnum/zero.pnum/zero.pnum/zero.pnum /eight.pnum/zero.pnum/zero.pnum/zero.pnum A/uniFB00ectDOF propbreak DistEE, A/uniFB00ectDOF, CtlDOF, Planpath, Trajecto... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /three.pnume-/zero.pnum/seven.pnum /one.pnum/eight.pnum/three.pnum/two.pnum/zero.pnum/zero.pnum /five.pnum/four.pnum/nine.pnum/six.pnum A/uniFB00ectDOF propstuck DistEE, A/uniFB00ectDOF, CtlDOF, Planpath, Trajecto... Force_ST, Force_Lin, Force_GR, Force_LG, EE_/one.pnum... /two.pnume-/zero.pnum/seven.pnum /one.pnum/eight.pnum/six.pnum/two.pnum/zero.pnum/zero.pnum /three.pnum/seven.pnum/two.pnum/four.pnum HoldPayload break DistEE, CtlDOF, Planpath, Trajectory, EngageL... Force_ST, Force_Lin, Force_GR, Force_LG, EEmo... /two.pnume-/zero.pnum/seven.pnum /one.pnum/eight.pnum/three.pnum/zero.pnum/zero.pnum/zero.pnum /three.pnum/six.pnum/six.pnum/zero.pnum ViewEnv poorview ViewEnv /two.pnume-/zero.pnum/six.pnum /one.pnum/zero.pnum/zero.pnum/zero.pnum/zero.pnum /two.pnum/zero.pnum/zero.pnum/zero.pnum EngageLand deform EngageLand /eight.pnume-/zero.pnum/six.pnum /one.pnum/zero.pnum/zero.pnum/zero.pnum /eight.pnum/zero.pnum/zero.pnum HoldPayload deform HoldPayload Force_ST, Force_Lin /eight.pnume-/zero.pnum/seven.pnum /one.pnum/zero.pnum/zero.pnum/zero.pnum/zero.pnum /eight.pnum/zero.pnum/zero.pnum DistEE degr DistEE Force_GR, Force_LG, EEmot, EEctl, Ctl/one.pnum, DOFs /five.pnume-/zero.pnum/six.pnum /one.pnum/zero.pnum/zero.pnum/zero.pnum /five.pnum/zero.pnum/zero.pnum EngageLand break EngageLand /two.pnume-/zero.pnum/six.pnum /one.pnum/zero.pnum/zero.pnum/zero.pnum /two.pnum/zero.pnum/zero.pnum A/uniFB00ectDOF ctldn A/uniFB00ectDOF Force_GR, Force_LG, DOFs /two.pnume-/zero.pnum/six.pnum /five.pnum/zero.pnum/zero.pnum /one.pnum/zero.pnum/zero.pnum A/uniFB00ectDOF mechfriction A/uniFB00ectDOF EE_/one.pnum, EEmot /five.pnume-/zero.pnum/seven.pnum /five.pnum/zero.pnum/zero.pnum /two.pnum/five.pnum A/uniFB00ectDOF propwarp A/uniFB00ectDOF Force_GR, Force_LG, DOFs /one.pnume-/zero.pnum/seven.pnum /two.pnum/zero.pnum/zero.pnum /two.pnum Table /three.pnum: Automatically-Generated Scenario-Based Static FMEA from model 0 50 100 time (s)0100mEnv1: x 0 50 100 time (s)0100mEnv1: y faulty nominal 0 50 100 time (s)050mEnv1: elev 0 50 100 time (s)6080100%StoreEE: socDynamic Response of ['Env1', 'StoreEE'] to fault AffectDOF mechbreak Figure /one.pnum/two.pnum: Dynamic behaviors of a motor breaking /four.pnum./three.pnum Dynamic Representation and Analysis In the dynamic model, the drone is given dynamic states and behaviors which iterate over time–in this case the position, velocity, charge, and perceived location of the system. This can then be used to model how the system behaves in faulty and nominal scenarios as shown in Figure /one.pnum/two.pnum for the mechanical break fault. In the nominal case, the drone /uniFB02ies out to a location and returns to land, while in the faulty case the system crashes soon after the fault is injected, leaving it far from the landing location. While this single-fault injection can help one understand how the system behaves, a fault injection ap- proach can be used to quantify the expected e/uniFB00ects of a fault which could occur at di/uniFB00erent points in the simulation. This is shown in Figure /one.pnum/three.pnum. As shown, the cost is high in the ascent and forward /uniFB02ight phase (/dollar.pnum/one.pnum/three.pnum/four.pnumK-/dollar.pnum/one.pnum/eight.pnum/four.pnumK) since the fault then leads to a crash, and low in the descent phase (/dollar.pnum/five.pnum/zero.pnum/zero.pnum), since the drone has already landed. Additionally, the cost increases in the middle of the forward /uniFB02ight phase since the system crashes far from its landing location, which incurs additional cost in the model. While these results are consistent with the results of the static model at the point in time considered in the model (forward /uniFB02ight), they also show how a higher-/uniFB01delity dynamic model model can elicit a more nuanced consideration of fault e/uniFB00ects. /four.pnum./four.pnum Hierarchical Representation and Analysis Given the e/uniFB00ects of failures in this system, it is important to consider how they can be mitigated through the component architecture. In the drone model, for example, one has the ability to consider whether to realize the A/uniFB00ectDOF function with a quadrotor, hexarotor, or octorotor architecture. This example considers the quadrotor and octorotor architectures. /one.pnum/five.pnum 0 20 40 60 80 100050000100000150000200000Cost Cost function of AffectDOF: mechbreak over time ascend forward descend Time (sec)0.02.5Rate1e7 Figure /one.pnum/three.pnum: Modelled cost over time of the motor breaking 0 50 100 time0100mEnv1: x 0 50 100 time0100mEnv1: y faulty nominal 0 50 100 time050mEnv1: elev 0 50 100 time5075100%StoreEE: socDynamic Response of ['Env1', 'StoreEE'] to fault AffectDOF: RFmechbreak Figure /one.pnum/four.pnum: Fault behavior of octorotor To compare how each architecture adapts to faults, the dynamic behaviors over time can be plotted. When considering the break fault, the quadrotor architecture reacts identically to the fault as in Figure /six.pnum, since los- ing one motor causes the system to lose stability. However, when the drone has an octorotor architecture, the system behaves as shown in Figure /one.pnum/four.pnum, faltering due to lost thrust but ultimately recovering and landing in the desired location. Thus the octorotor architecture is more resilient to this fault scenario. However, to make a decision about component architectures on the basis of resilience, the expected cost of all fault scenarios for both architectures must be compared and weighted against the operational and implementation costs. To quantify this cost, each of the faults associated with the realized function (Af- fectDOF) are injected in the model according to a sampling approach. In this case, while the octorotor component architecture mitigates a number of scenarios due to the increased system redundancy, it does not mitigate every fault (e.g. control errors), and in fact increases the chance of other faults (e.g. electrical problems that propagate to the battery) because the larger number of components provides more oppor- tunities for the fault to occur. Statistics from this fault approach are shown in Table /four.pnum. As shown, while the number of scenarios increases in the octorotor architecture, the number of scenarios which lead to a crash (and overall crash rate) is much lower, resulting in a lower overall resilience cost. This shows how fmdtools can be used to assist resilient design decision-making in the early design process. /one.pnum/six.pnum Quadrotor Octorotor Number of Scenarios /one.pnum/zero.pnum/four.pnum /two.pnum/zero.pnum/eight.pnum Number of Crashes /four.pnum/six.pnum /two.pnum/four.pnum Crash Ratio /zero.pnum./four.pnum/four.pnum /zero.pnum./one.pnum/two.pnum Crash Rate /two.pnum./four.pnume-/six.pnum /zero.pnum./eight.pnume-/six.pnum Resilience Cost /four.pnum/five.pnum/five.pnum/six.pnum/five.pnum /one.pnum/nine.pnum/three.pnum/five.pnum/nine.pnum Table /four.pnum: Cost of rotor faults in each architecture /five.pnum Conclusions The fmdtools project enables the consideration of resilience in the design process by providing a set of model classes, simulation methods, and analyses that enable the consideration of risk through the func- tional, behavioral, and structural stages of the design process. As demonstrated here, this enables one to analyze the system as the design becomes more detailed. fmdtools additionally has a number of features that make it relatively unique among fault simulation toolkits, including Python-based model de/uniFB01nition, un-directed fault propagation, and simple model parameterization. This environment not only conducive to design, but may be easily modi/uniFB01ed and extended as needed to ful/uniFB01ll research needs, since all model attributes and simulation/analysis methods are de/uniFB01ned in open-source Python code. While the usefulness of this work is apparent from the demonstrations shown here, there are a number of possible extensions that would increase its practicality in design. First, safety is an important aspect of resilience that has speci/uniFB01c requirements not explicitly taken into account in this work. That is, while this work is concerned with quantifying the costs of faults, safety procedures require one to quantify the overall cost of the entire set of failure scenarios, which often requires taking a deductive approach (ARP, /one.pnum/nine.pnum/nine.pnum/six.pnum). Future work should address this by providing an approach to identify top-level failure events and quantify the risk of those events. Additionally, while the models used in this work are deterministic, failures can often have probabilistic e/uniFB00ects that must be taken into account to accurately quantify overall risk. Future work should incorporate probabilistic state transitions into the model to enable non-deterministic fault propagation. Finally, while de/uniFB01ning models directly in Python code increases model expressiveness, it forces one to use a stand-alone model and may make the toolkit di/uniFB03cult to use without the relevant programming knowledge.Future work should provide an interface for de/uniFB01ning these models in an existing modelling tool-chain or model formalism (e.g. AADL) so the same model used by other design and analysis processes can be used to model resilience. Acknowledgment This material is based upon work supported by the National Science Foundation under Grant No. NSF CMMI #/one.pnum/five.pnum/six.pnum/two.pnum/zero.pnum/two.pnum/seven.pnum. Any opinions, /uniFB01ndings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily re/uniFB02ect the views of the National Science Foundation. The fmdtools package and scripts used to produce this work are archived at (Hulse et al., /two.pnum/zero.pnum/two.pnum/zero.pnum) under open licenses.