Laws of Software Evolution Revisited

Data obtained during a 1968 study of the software process [leh69] led to an investigation of the evolution of OS/360 [leh85] and and, over a period of twenty years, to formulation of eight Laws of Software Evolution.. The FEAST project recently initiated (see sections 4 6 below) is expected to throw additional light on the phenomenology underlying these laws, to increase understanding of them, to explore their finer detail, to expose their wider relevance and implications and to develop means for their beneficial exploitation. This paper is intended to trigger wider interest in the laws and in the FEAST study of feedback and feedback control in the context of the software process and its improvement to ensure beneficial exploitation of their potential. 1 Historical Background The first three of a total of now eight laws of software evolution were formulated in the mid seventies [leh74] arising from analysis of data first acquired during a study of the IBM programming process [leh69]. These three were discussed in somewhat greater detail in 1978 [leh78]. Two further laws were introduced in a 1980 paper [leh80a] with the sixth introduced in a footnote [leh91]. The remaining two have been discussed in presentations but are published here for the first time. All relate specifically to E-type systems [leh80b] that is, broadly speaking, to software systems that solve a problem or implement a computer application in the real world. Section 2 restates and briefly discusses the laws stressing, in particular, the role of process feedback in generating the phenomenology they reflect. This is followed in section 3 by an equally brief discussion of the use of the term laws in describing the observations from which they were inferred. Section 4 introduces the FEAST project, the concepts and observations on which it is based and outlines the planned, now funded FEAST/1 investigation relating it to a broader long term, multidisciplinary collaborative investigation which must follow. 2 The Laws 2.1 I Continuing Change An E-type program that is used must be continually adapted else it becomes progressively less satisfactory. This law is in accord with universal experience. It suggests that the growth of an E-type software system is in some ways akin to that of an organism. It result, however, from feedback pressures caused by mismatch between the software and its operational domain, whereas that of biological organisms results primarily from pressures within the organism itself. This need for continuing adaptation and evolution is intrinsic to E-type applications and software. It is, in part, due to the fact that development, installation and operation of the software changes the application and its operational domain so creating mismatch between the two. Evolution is achieved in a feedback driven and controlled maintenance process. If the consequent pressure for evolution to adapt to the new situation is resisted the degree of satisfaction provided by the system in execution declines with time. 2.2 II Increasing Complexity: As a program is evolved its complexity increases unless work is done to maintain or reduce it. This law may be an analogue of the second law of thermodynamics or an instance of it [sca93]. It results from the imposition of change upon change upon change as the system is adapted to a 21/1/97, 5:20 pm 1 mml556/2[papers] 1 Numbered in order of formulation and publication. Over the years the names and detailed wording of some of the laws have been modified but the underlying understanding they reflect has remained essentially the same 2 Papers identified by a * in the references listing are reprinted in [leh85] changing operational environment. As the need for adaptation arises (first and seventh laws) and changes are successively implemented, interactions and dependencies between the system elements increase in an unstructured pattern and lead to an increase in system entropy. If the growth in complexity is not constrained, the progressive effort [bau67] needed to maintain the system satisfactory becomes increasingly difficult. If anti regressive effort [leh74] is invested to combat the growth in complexity, less effort is available for system growth. Given that resources are always limited the rate of system growth declines as the system ages whichever strategy is followed. In practice the balance between progressive and anti-regressive activity is determined by feedback. 2.3 III Self Regulation The program evolution process is self regulating with close to normal distribution of measures of product and process attributes. The evolution of industrially produced E-type software is implemented by a technical team operating within a larger organisation. The interests and goals of the latter extend far beyond completion of the system in question. Checks and balances will have been established by corporate and local management to ensure that operational rules are followed and organisational goals at all levels are met. The positive and negative feedback controls that implement these checks and balances provide one example of feedback driven growth and stabilisation mechanisms. There will be many others.Together they establish a disciplined dynamics whose parameters are, at least in part normally distributed [cho81] being the consequence of a large number of pseudo independent managerial and implementation decisions. After a while this dynamics determines many of the growth and other development characteristics of the evolving product 2.4 IV Conservation of Organisational Stability (invariant work rate) The average effective global activity rate on an evolving system is invariant over the product life time. Of the eight laws this fourth law was and remains the most counter intuitive. By and large it is still generally believed that the effort expended on system growth and evolution is determined by managerial decision. To some degree corporate and local management certainly do control activity targets and resource allocation to a project, system or activity. Their ability to do this is, however, constrained by external forces, trade unions or the availability of personnel with appropriate skills for example. But as suggested by the third law the effort usefully expended, that is to achieve satisfactory results, is also determined by system attributes, complexity for example, that are analogous to attributes such as inertia and momentum in mechanical systems. As indicated by Brooks [bro75] circumstances may even arise where, for example, providing additional resources may actually reduce the effective rate of productive output as a result of increased communication and other overheads or decreases in process quality. In any practical situation the level of activity is not decided exclusively by management edict but by a host of feedback inputs and controls. Project data so far analysed suggests that in practice this leads to stabilisation at a fairly constant level. 2.5 V Conservation of Familiarity During the active life of an evolving program, the content of successive releases is statistically invariant. One of the factors that determines the progress of a software development is the familiarity of all involved with its goals. The more changes and additions are associated with, say, a particular release, the more difficult it is to for all concerned to be be aware, to understand and to appreciate what is required of them. The rate and quality of progress and other parameters are influenced, even limited, by the rate of acquisition of the necessary information by the participants collectively and individually. The larger work package the more challenging mastery of the matter to be acquired. Data to date suggests that this is not a nice linear relationship but one in which there are one or more critical size levels which if exceed trigger behavioural change. The rapidity of the change suggests that here too feedback mechanisms play an important role. 2.6 VI Continuing Growth Functional content of a program must be continually increased to maintain user satisfaction over its lifetime. At first sight the sixth law, Continuing Growth, appears little different to the first law, Continuing 21/1/97, 5:20 pm 2 mml556/2[papers] Change. In fact, however, the two laws reflect distinct, though not unrelated, phenomena. The first relates closely to the Software Uncertainty Principle [leh89,91] which recognises, inter alia, that, for example, changes in the operational domain will invalidate assumptions embedded in E-type software and so cause unexpected behaviour in execution. As users become aware of consequent imprecision, unsatisfactory or incorrect operation they will demand a fix. The law reflects the feedback impact of system usage on the application, on its domain, on users and on assumptions made during development and maintenance of the software. Change is inevitable in software as it is in any active system. The rate at which pressure for change develops in software relative to human perception and the intolerance for mismatch if changes are not implemented is, however, greater than for other real world systems. Hence the common perception of continuing maintenance and for the view that E-type software must be seen and treated as organisms. The sixth law addresses change deriving from a different source. When a new system is to be developed (whether from scratch or from of-the-shelf (OTS) components) or an existing one is to be upgraded the first input required, though often not provided, is a detailed description (model) of the application in its actual or desired operational domain. This application domain model is the foundation and definitional source of the requirements and specification for the required system. Because of limitations arising from constraints such as budget, delivery dates, technology and understanding of the application in its domain the domain model and the definitions of requirements and sp