Improving Systems Integrity by Using Thread Analysis for Design Validation

The techniques described in this paper have been developed and applied on a series of major systems engineering projects, most recently including the BOWMAN programme. BOWMAN will encompass the design, development, manufacture, conversion (rollout) and support of the new communications systems for the British Army. The BOWMAN contract will be valued in excess of £1.5 Billion and be delivered and installed over a period of 6 years. Thread Analysis is a technique that enables systems design to be systematically inspected with respect to a number of desirable properties. Threads analysis finds application in domains including safety critical systems; security critical systems; concurrent, real-time systems and systems with complex and diverse behaviour. Thread Analysis is concerned with 'unravelling' and documenting sequences of dependent events within complex systems. The consideration of the way in which one event leads to one or more other events in the system and the consequent changes in that systems state can provide deep insight into its functioning and integrity. This paper shows how the essentially simple technique of inspecting dependent sequences of system events can be applied in a surprising variety of ways to yield useful and powerful results. Introduction: The Structure of this Paper The first section of this paper defines some basic terms that are useful in the following discussion. The second section describes why we believe threads analysis is so powerful for inspecting, and thus improving, system integrity. The third section provides outline procedures and methods for performing threads analysis and outlines their application in critical systems development. The fourth section describes some of the problems of large scale analysis and introduces a method of describing threads as dependent series of logical facts. The final section refers to the threads based design. What are Events and Threads? Events are things that happen. In the domain of systems engineering it is normally considered that an event leads to, or causes a change in system state. Events may trigger behaviour. In older systems analysis techniques (including Yourdon) triggering or initiating events were seen as distinct from the processes or behaviour that were caused by those events (Yourdon 1998). In more modern, objectoriented schemes, dynamic behaviour may be regarded as synonymous with the set of events and the changes of object state that are caused by those events (Douglass 1998; Shlaer and Mellor 1992). It is this latter model that is adopted here. Behaviours are changes of state mediated by events. Threads are sequences of dependent events. Our formal definition of a thread is that they are sequences of causatively related events. Cause is important in this context. We are not concerned with events that simply happen to have a time-like relationship. Just because two events happen one after the other does not mean that one causes the other. Threads are linked chains of causes. A message may be considered as a generalisation of an event. Messages pass between system objects and result in state changes. In addition to these essential characteristics for an event, they carry message data. The subsequent sequence of state changes of a system may depend on the information carried in the message. Message data may be used in decision logic that controls future life-cycles. Threads Analysis is the process of tracing sequences of related events (a thread) and inspecting states, processes, functions and interfaces on that thread’s trajectory. Why is Threads Analysis Effective? Systems with complex, concurrent, dynamic behaviour are, by definition, difficult to 'understand', analyse and validate. Tracing threads through a system provides an effective method of systematically inspecting a system. There appears to be an underlying mechanism that causes threads analysis to be an effective and powerful technique: structural spanning, which we shall now describe in some depth. Threads Analysis Spans Functional and Architectural Boundaries Threads analysis tends to operate 'orthogonally' to system design and analysis. We use the term ‘orthogonal’ metaphorically to convey the notion that this analysis ‘cuts across the grain’ or ‘intersects the flow’ of design structures at right angles. Whereas analysis and design activities are typically decompositional with respect to system function, threads analysis tends to be 'function spanning'. That is, because system analysis and design considers progressively more detailed components of a system, defects may be introduced due to the interaction of those components. Whilst each of the components might 'work' the system as a whole exhibits some pathology. Threads analysis tends to treat 'end to end' or global functionality of a system and thus seems to be complimentary to functional decomposition. TA almost invariably spans structural groupings in the system analysis, system realisation and design and analysis process. For example, threads frequently span external system interfaces and inter-subsystem interfaces. Such interfaces are recognised sources of errors and thus warrant some effort during the design validation process (Lutz 1993). In addition to spanning interfaces, TA tends to span architectural structuring of complex systems. System architectural structures are of increasing importance as systems become more complex (Shaw and Garlan 1996; Bass et al 1997). Not only do architectures provide a decompositional mechanism, they also provide important patterns and metaphors within which systems may be considered (Saunders 1998). TA frequently spans architectural structures and thus tests those structures in a manner that is both realistic and yet potentially counterexpectational. This important notion of inspecting to detect the unexpected is discussed more fully in the next section. Threads Analysis Spans Language and Cognitive Boundaries We have said above that system architectures provide systems engineers with ways of thinking about a system. However, this great strength of system architectures poses its own threat. Analysts and designers may be predisposed to certain types of cognitive error that arise specifically as a result of fixed mental models. In short, knowing very well how a system should work can be an insurmountable barrier to seeing that is does not (Collins and Leathley 1995). 1 Such pathologies at the system level have been termed 'Systemic Failure Modes' (SFM). See Collins and Thompson (1995) 2 What might be termed 'intrafaces' It is not simply the case that threads themselves span architecture. Analysts engaged in threads analysis appear less prone to a mental set dominated by an architecture and thus seem more able to detect weaknesses in it. The language we use to describe systems organises and structures our thinking about them (Whorf 1936). If engineering activities are dominated by a decompositional (that is architectural, sub-system or component-based) vocabulary, then it is likely that thinking will also be so dominated. In this context our 'vocabulary' includes not just the spoken language of systems engineering but the centrally important language of diagrams and formal symbolisms we employ. Whilst structured analysis and Object Oriented analysis are different in many ways, what they share is a common foundation of decompositional language. TA provides an orthogonal vocabulary that can be applied to a system and thus provides an escape route from a corporate mind-set concerning that system. The thinking of threads analysts is not dominated by decomposition since it is structured on an entirely different 'vocabulary' the vocabulary of end-to-end functionality, of initial causes and final goals. Threads Analysis Spans Organisational Boundaries In our experience Conway's law (that systems architecture mirrors organisational structure) always appears to be true (Coplien 1998). Consequently a significant barrier to effective interworking of subsystem components is that different groups of experts are responsible for each of them. Knowledge and expertise is commonly clustered within groups of engineers who may be physically, as well as organisationally distributed. Whilst the archetypal systems engineer will have tried to minimise intercomponent pathologies in the architecture it seems to us important that the integrity of the system is validated by inspection across organisational boundaries. The role of the threads analyst is thus an important one, because it forms a bridge between teams of people. If the analysis and design teams are the analogue of a systems architectural components then the thread analyst is the analogue of its intracomponent functionality. Threads analysts are good inspectors precisely because their inspection applies pressure orthogonally to the organisational structures. Methods of Threads Analysis This section describes a series of methods for threads analysis. Each of these is built on the same underlying model. That is, they all involve the systematic inspection of causally related events and the changes of system states that result. They involve the inspection of state changes that result from a series of events and the interfaces that threads intersect. This section is organised so that the methods become progressively more formal. That should not be taken to mean, however, that they become progressively 'better'. We believe that each of these methods has value when applied appropriately. For a method to be successful it must be ecologically valid: it must match the needs of the organisation that applies it. It is common in our experience that an informal analysis performed early can be more effective at improving system integrity than an exhaustive, formal analysis performed late. These methods may be used sequentially. Early analysis can be completed using the less formal methods. The results of these analyses can be used as a basis for l