Incorporating Soft Variables Into System Dynamics Models : A Suggested Method and Basis for Ongoing Research

How to determine the impact of soft variables, including intangibles or social variables, and combining them as necessary with hard variables in system dynamics models is a significant challenge. This paper identifies a weakness in system dynamics modelling practice, that is, in reliably incorporating soft variables into system dynamics models. A method for incorporating such variables and a basis for further research is offered. The method combines systems thinking, research into causality analysis, multiple criteria decision analysis (conjoint analysis) and system dynamics modelling, in an integrated approach. “To omit such [soft] variables is equivalent to saying that they have zero effect – probably the only value that is known to be wrong!” (Forrester, 1961: 57) It is inescapable that from time-to-time we will need to build system dynamics models that take soft i variables into account. The challenge is to incorporate the influences of soft variables in ways that produce meaningful, reliable and repeatable results. We need to develop in-depth understanding of the roles and influences of soft variables. We must avoid making guesses about the influences that soft variables might have. Rather, we must create and repeatedly test dynamic hypotheses about soft variables. This will also demand that we comprehensively document our hypotheses about influences and how those influences are produced and combined, making the results available for peer review. Only when we do this will we build a sound appreciation of the nature and significance of soft variables both in our models and in the real world. Our track record in this area is not as good as it might be, as the following example suggests. Pseudo-algebraic Expressions in System Dynamics Models – An Example It is disturbing to encounter system dynamics models reflecting guesses about soft variables and as a result containing pseudo-algebraic expressions conveniently and artificially contrived to make a model or simulation behave as the modeller intends. Evidence of this practice can be found in models produced by students and experienced practitioners alike. Pseudoalgebraic expressions are a corruption of mathematical logic used to conveniently and artificially combine quantified soft variables. Such expressions may also involve mixes of hard and quantified soft variables. The assumptions underlying this practice seem to be that soft variables, including that class of soft variables known as ‘intangibles’: • conform to numerical scales; • can be quantified in an absolute sense; • quantification is both valid and universally acceptable; and • once quantified, soft variables can be treated as dimensionless variables that can be added or multiplied ii in exactly the same way as ordinary variables encountered routinely in system dynamics modelling are. In their totality, these assumptions are erroneous; when taken individually, caution is needed. Further, any practice leading to the creation of pseudo-algebraic expressions, or their use, must be closely scrutinised. In one particular example seen recently, a management flight simulator was built to assist in training managers to make decisions in the complex dynamic environment a particular firm operates. The underlying model contained around 60 variables, of which some 20 could only be described as being soft. Algebraic expressions made up of interesting mixes of hard and soft variables were built into the model to replicate the modes of behaviour that the modeller had identified as important . One expression in the model contained the soft variables Organisational_Performance, Qualifications_Held_By_ Individuals, and Individual’s_Motivation_Level. Organisational_Performance was intended to describe how well the organisation performed (in an average sense, rather than in relation to discrete events) in making decisions, where decision makers were required to draw upon their own knowledge, skill and competence. Qualifications_Held_By_ Individuals described how well equipped individuals, in terms of formal qualifications held (including skills and competencies), were to make decisions and Individual’s_Motivation_Level indicated motivation of individuals to take action, make the necessary decisions, or to implement a particular strategy. The resulting algebraic expression took the form: _ _ _ _ ' _ Organisational Performance Qualifications Held By Individual x Individual s Motivation Level = _ The abuse of mathematical logic and system dynamics principles in this example should be self-evident. Despite obvious flaws in the model, the paper describing this model was peer reviewed and ultimately accepted for a recent international system dynamics conference. This suggests that as a group, we are prepared to accept these dubious modelling practices? The intent of the modeller in producing this simulator was genuine although surprisingly naïve. The modeller apparently set out to do exactly what we teach, that is, to replicate observed reference modes of behaviour. Our actions, both in terms of learning and system dynamics modelling practice are both misguided and naïve if we do not critically investigate why and how reference modes of behaviour are produced. In this instance, it seems that the modeller was actually mimicking what he saw as an important reference mode of behaviour without having a clear understanding of how that particular reference mode was actually created in the real world, or if he did he did not have the necessary tools (either physical or intellectual) to build a faithful representation of the causal mechanisms. In this instance, and probably of greater concern is that how the particular reference mode of behaviour was actually created is unlikely ever to be questioned by anybody subsequently flying the management flight simulator in a training session. This is because the underlying structure of the model and its code, that making the simulator behave as it does, are masked from the players. Players, like the layperson in many real-life systemic problem situations will focus on the responses to their actions, rather than the (obscured) mechanisms producing the responses. The absence of essential critical analysis in such instances is very likely to lead to the unfortunate consequences that: • poor practices become embedded in the design of simulations; • erroneous conclusions are drawn about systemic causality, where soft variables are involved; and • fallacious learning experiences occur. This suggests we must look seriously at the treatment of soft variables in system dynamics modelling. Need to Take a Hard Look at Soft Variables in System Dynamics Models Sterman (2002: 522-523) makes the following points regarding soft variables in system dynamics models: • Soft variables should be included in our models if they are important to the purpose. • Omitting structures of variables known to be important because numerical data are unavailable is actually less scientific and less accurate than using your best judgement to estimate their values. Omitting concepts because they have no numerical data is a sure route to narrow model boundaries, biased results and policy resistance. • We must evaluate the sensitivity of our results to uncertainty in assumptions – whether we estimated the parameters judgmentally iv or by statistical means. • It is important to use proper statistical methods to estimate parameters and assess the ability of the model to replicate historical data when numerical data are available. Rigorously defined constructs, attempting to measure them, and using the most appropriate methods to estimate their magnitudes are important antidotes to causal empiricism, muddled formulations and erroneous conclusions we often draw from our mental models . • Most importantly, we should not accept the availability of data as given, as outside the boundaries of our project or research. We must ask why concepts our modelling suggests are important have not been measured. Frequently it is because no one thought these concepts were important...[stemming] from the narrow boundaries of our understanding. • Human creativity is great: once we recognise the importance of a concept, we can almost always find ways to measure it. Today, many apparently soft variables such as customer perceptions of quality, employee morale, investor optimism, and political values are routinely quantified with tools such as content analysis, surveys, and conjoint analysis . • Of course all measurements are imperfect. Metrics for so-called soft variables continue to be refined, just as metrics for so-called hard variables are. Quantification often yields important insights into the structure and dynamics of a problem. Often, the greatest benefit of a modelling project is to help the client see the importance of and begin to measure and account for soft variables and concepts previously ignored. Simply put, we need to take soft variables into account when they are likely to have an impact; priority for incorporating such variables, as in the case of hard variables, must be based on the likely extent of their impact. If these variables are likely to have an impact and are, therefore, worthy of inclusion in our models, we should measure them if we can. If we cannot measure them, we should estimate them as best we can by methods that give consistent, repeatable and reliable results. We cannot afford to take shortcuts and indulge in poor modelling practices regardless of how tempting and convenient it might appear to be at the time; to do so could seriously damage confidence in the system dynamics modelling discipline or, even worse, destroy our credibility. Purpose of This Paper How to determine the impact of soft variables, including intangibles or social variables, and combining them through the system dynamics modelling process with hard variables is particularly vexing . This paper offers a method of incorporating soft variables i