METHOD AND APPARATUS FOR METHOD AND APPARATUS FOR SELF-ADAPTIVE CODE

Abstract: A computer-implemented method and system for allowing software to carry its own specification.

Description: METHOD AND APPARATUS FOR METHOD AND APPARATUS FOR SELF- ADAPTIVE CODE Related Applications: This application claims priority under 35 U.S.C. § 119(e) to U.S. Application Serial No. 60/197,909 entitled "Method and Apparatus for Self- Adaptive Code" of Pavlovic et al., filed April 16, 2000, and to U.S. Application Serial No. 60/197,983 entitled "Method and Apparatus for Self-Adaptive Code" of Pavlovic et al., filed April 17, 2000, both of which are herein incorporated by reference in their entirety. This application is related to U.S. Application Serial No. 09/665,179 of Pavlovic and Smith, filed September 19, 2000, which is herein incorporated by reference in its entirety. BACKGROUND OF THE INVEN ION The present invention relates generally to system design and, specifically, to a method and system that allow software code to carry its own specification. The design of systems, such as computer systems or engineering systems is a complex process. While it is possible to design systems from scratch using a minimum of design tools, most modern designers use tools to represent and manipulate designs for complex systems. In the past, most structural problems with software were attributed to lazy programmers, who wrote spaghetti code and not enough comments. Structuring and literate programming were proposed as solutions. What is needed is way to help automate the process of fitting software modules together and of automating the process of determining whether software modules are composable with each other. SUMMARY OF THE INVENTION The present invention allows the design and utilization of "specification-carrying software." In the described embodiment, software is extended with comments in a generic specification language, or with Floyd-Hoare annotations, although other types of specifications can be used. Specifications preferably are extended with executable (partial) implementations, or abstract, but verifiable behaviors. A specification is not merely a static formalization of requirements. (Such requirements are usually not completely determined). Instead, a specification is the currently known structural and/or behavioral properties, functional or otherwise of the software. It is automatically updated, following evolution of the runnable component itself. Use of a specification makes the idea of embedding a model in software precise: a model is a spec. The idea of a model is that it is an abstraction of reality, displaying what we care for, and abstracting away the rest. That is what software specifications do in a formal and systematic way. In a way, the carried specifications are like genes: each component carries the blueprint of itself. It precludes combining incompatible components, as alien species. It can be used for certifying, similarly like protein markers which cells use to distinguish friends from enemies. Accommodate the dynamics of ever changing interfaces, and the unpredictable interactions arising from the ever-changing environments. The fact that the architectures never stop changing should not be taken as a nuisance, but recognized as the essence of the game of software, built into its foundation, and implemented as a design routine. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram of the overall architecture of an embodiment of the present invention. Figs. 2(a) and 2(b) are flow charts showing step-wise refinements of a specification. Figs. 3(a) and 3(b) show a conceptual example of a colimit operation. Figs. 4(a) and 4(b) show another conceptual example of a colimit operation. Fig. 5 shows an example of the colimit operation for a specification. Fig. 6 shows an example of the colimit operation for a hereditary diagram. Fig. 7 shows another example of the colimit operation for a hereditary diagram. Figs. 8(a), 8(b), and 8(c) show an example user interface for the colimit operation of a hereditary diagram. Figs. 9(a)-9(j) show an example of operations initiated by the user to further illustrate the colimit operation for a hereditary diagram. Fig. 10 is a block diagram of a specification-carrying software code system performing automatic code generation in accordance with the specification-carrying software code. Fig. 11 shows an example of specification morphism. Fig. 12 shows an example of composition. Fig. 13 shows an example of a diagram. Fig. 14 shows an example of a parameterized diagram. Fig. 15 shows an example of how specification-carrying software can be used in domain-specific software synthesis. Fig. 16 shows an example of symbols used to explain EPOXI, an embodiment of a specification-carrying software code system. Figs. 17-19 show more EPOXI-related examples. Fig. 20 shows an example of the impact of specification-carrying software code. Fig. 21 shows an example of a Boolean gauge for software composability. Fig. 22 shows an example of a Boolean gauge for software composability of a software wrapper. DETAILED DESCRIPTION General Discussion In order to support dynamic assembly, software components must carry enough information about their structure and behavior. Gauge generators: Many real phenomena yield to measuring only when specially prepared. In order to measure the spreading of a body liquid, physiologists mark it by a radioactive substance. In order to measure the survival rate in a large population, ecologists tag selected parts of it. Similarly, in order to measure the logical distance between software components, we shall prepare a framework in which they come equipped with specific logical markers. This is the idea of specification carrying software (or "self-adaptive code). With the present invention, it is possible to measure the composability and adaptability of software components in terms of the logical distance of their specifications, viz the degree of consistency of their union. Classically, the consistency of a theory is evaluated in the Boolean algebra 2: a theory is either consistent, or not. In constructivist logic, the consistency can be evaluated in a heyting algebra, e.g. of the open sets of a topological space. In categorical logic, the consistency can be evaluated in a pretopos. A preferred embodiment of the invention uses the simplest space of the truth values sufficient for the practical tasks arising in composing and adapting software. The precision gauge, measuring how closely is a software component approximated by the specification it carries, is built in into the very foundation of the framework of the described embodiment of the present invention. Each module comes with an explicit satisfaction relation, establishing the sense in which the executable component satisfies the given structural specification. The satisfaction relation of a composite module is derived from the satisfaction relations of the components, and the logical distance of the two specifications. A composability gauge measures the logical distance, viz the degree of consistency of a union of theories and uses known software theorem. For example, a composibility gauge may address verifying the functionality/safety, and timing constraints of software. Propagation and Adaptability Gauges measure the effects of propagating the refinement of one component of an architecture to localize compliance conditions on another component propagating change specifications into the structure of an architecture A Gauge Generator is specialized to a given architecture from a model of an architecture General Background In the described embodiment, specification-carrying software inherits from one or more of: proof-carrying code, model-integrated software, or a distributed optimization. In the described embodiment of the present invention, a user specifies his design using a specification language. Specification software manipulates the specified design to yield a more detailed system design. Some of these manipulations involve use of a library of specifications. The invention is described herein in connection with an embodiment of a system for specifying software. It should be understood that the invention can be used in a number of different software development systems and the description herein is not to be taken in a limiting sense. Specifications are the primary objects in the described specification language. A specification can represent any system or realm of knowledge such as computer programming or circuit design and describes a concept to some degree of detail. To add properties and extend definitions, the described specification software allows the user to create new specifications that import or combine earlier specifications. This process is called refinement. Composition and refinement are the basic techniques of application development in the described specification software. A user composes simpler specifications into more complex ones, and refines more abstract specifications into more concrete ones. Refining a specification creates a more specific case of it. In the described embodiment, specifications can represent an object or concept. A complex specification can be presented as a diagram of simpler specifications. A software specification is a formal representation of objects or concepts that come about in a software development project. In the described embodiment, a complex specification can be composed and refined as a diagram of simpler specifications; still more involved specifications can be composed as diagrams of such diagrams; and so on. Large specifications are thus subdivided into diagrams of smaller specifications. The process of software design is stratified into such diagrams, diagrams of diagrams and so on. This is what is meant by the expression "hereditary diagrams of specification." A diagram includes: • A set of nodes (or vertices) • A set of arcs (or edges or arrows), and • Two mappings, assigning two nodes to each arc: its source-node and its target-node. The nodes of a diagram of specifications are formal specifications, capturing the relevant objects and concepts to be specified, the arcs of a diagram of specifications are the "specification morphisms," capturing the relationships between the nodes: how some specifications inherit or share the structure specified in others. Diagrams thus provide a graphically based method for software development and refinement, allowing "modular decomposition" and reuse of software specifications. The described embodiments of the software development tool support: • Specification refinement: deriving a more concrete specification from a more abstract specification by adding more structural detail • Code generation: when enough structural detail has been specified to determine concrete programming structures suitable to perform the required task, code in a suitable language is generated. • Colimit determination In general, determination of a colimit is a destructive operation, resulting in the loss of information about the involved diagrams. The described embodiments of the invention protect and retain the diagrams by folding them into a node. Since the described embodiment allow for diagrams of diagrams, this protection can occur in a multi-level diagram of diagrams. Nodes of a diagram show the objects or concepts and arcs between the nodes show relationships (morphisms) between the nodes. Diagrams are used primarily to create sets of objects and to specify their shared parts, so that the individual parts can be combined. Specifications can also be defined to be hereditary diagrams. The described specification software allows a user to derive a more concrete specification from a more abstract specification. In general, the complexity of a specification is increased by adding more structural detail. The following techniques are preferably used (separately or together) to refine specifications: -the import operation, which allows a user to include earlier specifications into a later one; -the translate operation, which allows a user to rename the parts of a specification; and -the colimit operation, which glues concepts together into a shared union along shared sub-concepts. Use of diagrams (and hereditary diagrams) allows the user to retain information about a specification during the design process. The described embodiment of the present invention allows a user to define a specification that is a hereditary diagram and to perform the colimit operation on the hereditary diagram. The described embodiments include specification diagrams and compute co-limits in this category. Furthermore, the described embodiments iterate this procedure, yielding the category of hierarchical diagrams, and computes colimits for these hierarchal diagrams. The described embodiment provides a software tool for building, manipulating, and reusing a collection of related specifications. The tool allows a user to describe concepts in a formal language with rules of deduction. It includes a database (library) that stores and manipulates collections of concepts, facts, and relationships. The present invention can be used to produce more highly refined specifications until a concrete level of abstraction is reached. For example, a specification can be refined until it reaches the computer source code level. As another example, a specification can be refined until it reaches the circuit level. Referring now to Fig. 1, there is shown a block diagram of the overall architecture of an embodiment of the present invention. Fig. 1 includes a data processing system 100 including a processor 102 and a memory 104. Memory 104 includes specification software 110, which implements the refinement methods defined herein. Specification software 110 preferably implements a graphical user interface (GUI) that allows a user to define specifications and morphisms and that allows a user to indicate refinements to be performed on the specifications. Specification software 110 includes or accesses a database 112 that includes definitions of specifications and diagrams. The specification being refined is stored in memory 114. The refinement operations indicated by the user can result in computer code 116 if the user chooses to perform refinements to the computer code level. Figs. 2(a) and 2(b) are flow charts showing step-wise refinements of a specification during an exemplary design process. In element 202 of Fig. 2(a), the user is allowed to define/enter software specifications, diagrams, and hereditary diagrams (also called a "hierarchical diagram" or a "diagrams of diagrams"). Specifications are the primary objects defined by a user. In the described embodiment, specifications can represent a simple object or concept. A specification can also be a diagram, which is a collection of related objects or concepts. As shown in Figure 29, nodes of a diagram show the objects or concepts and arcs between the nodes show relationships (morphisms) between the nodes. Diagrams are used primarily to create sets of objects and to specify their shared parts, so that the individual parts can be combined. Specifications can also be defined to be hereditary diagrams, where at least one object in a node of the diagram is another diagram. Specifications can be defined in any appropriate specification language, such as the SLANG language defined by the Kestrel Institute of Palo Alto, CA. SLANG is defined in the SLANG Users Manual, available from the Kestrel Institute of Palo Alto, CA. The Slang Users Manual is herein incorporated by reference. A specification can represent any system or realm of knowledge such as computer programming or circuit design and describes a concept to some degree of detail. In element 204, the user is allowed to start refining his specifications, diagrams, and hereditary diagrams. To add properties and extend definitions, the described specification software allows the user to create new specifications that import or combine earlier specifications. This process is called refinement. Composition and refinement are the basic techniques of application in the described specification software. A user composes simpler specifications into more complex ones, and refines more abstract specifications into more concrete ones. Refining a specification creates a more specific case of it. The described specification software allows a user to derive a more concrete specification from a more abstract specification. In general, the complexity of a specification is increased by adding more structural detail. The following techniques, among others, are preferably used (separately or together) to refine specifications: -the import operation, which allows a user to include earlier specifications into a later one; -the translate operation, which allows a user to rename the parts of a specification; and -the colimit operation, which glues concepts together into a shared union along shared sub-concepts. Fig. 2(b) is a flow chart of a method for refining a specification. The user indicates a refinement operation, which is then performed by specification software 110. Fig. 2(b) shows three examples of refinement operations. It will be understood that other refinements are possible. In element 216, the user indicates that a specification or diagram is to be imported. In element 218, the user indicates finding a colimit of a hereditary diagram. In element 220, the user indicates a translation of a specification or diagram. In element 206 of Fig. 2(a), the user refines his specification to a level of generating computer code. A user may choose not to refine a specification to this level. The refinement process can be used for purposes other than generating computer source code. For example, the refinement process can be used to help understand a specification. As another example, the refinement process can be used to help verify the consistency of a specification. The Colimit Operation Figs. 3(a) and 3(b) show a conceptual example of a colimit operation. A colimit is also called "composition" or a "shared union." A "pushout" is a colimit in which a colimit is taken of a parent node and its two children nodes. It will be understood that the examples of Figs. 3 and 4 are somewhat simplified and are provided to aid in understanding of the colimit operation. In Fig. 3, the user has defined a specification "car" 302. This specification 302 has been refined by the user as red car 304 and fast car 306. Thus, the arcs from node 302 to 304 and 302 to 306 are labeled with an "i" (for instantiation/import). In Fig. 3(a), the "defining diagram" shows only the spec/morphism diagram from which the colimit is formed. Fig. 3(b) shows a "cocone diagram," which also shows the colimit and the cocone morphisms (labeled "c"). In the described embodiment, the GUI labels arcs as follows, although any appropriate labeling and morphisms could be used (or none). i: instantiation morphism d: definitional translation t: transitional morphsim c: cocone morphism id: identity morphism The defining diagram for a colimit is not limited to a three node diagram. A colimit can be taken of any diagram. An example of a different diagram shape is shown in Fig. 3(b). In the colimit operation, any type of node related by morphisms in the diagrams are mapped to the same type of node in the colimit. Conversely, any unrelated types are mapped to different types in the colimit. The same is true of operations. When you compose specifications, types or operations that have the same names in different component specifications might be mapped to different result operations. For example, suppose specification A and specification B are combined to form specification C. Both A and B have operations named concat, but the operations do not work the same way, and need to be differentiated in specification C. In this case, specification software 110 generates unambiguous names in the colimit. Similarly, types and operations that have different names in the component specifications can be mapped to a single element in the colimit. For example, the operation concat in specification A and add in specification B might both be mapped to a single concatenation operation in the colimit specification C. In this case, the resulting element preferably has both names. Fig. 5 shows a more realistic example of the colimit operation for a specification. In this example, a virtual memory (VM) is a parameter of the operating system (OS). Suppose we want to formally specify a simple operating system (OS). There are large fragments of the theory that can be abstracted away. In other words, the structure of the system does not depend on a particular virtual memory (VM) implementation. Thus, the formal VM requirements can be taken as a parameter of the formal OS specification. Similarly, a particular VM system, VM_0, can be a parametric in paging policies (PP). Thus, the parameter VM can be instantiated to another parametric specification VM_0. In this way, a complex system naturally decomposes into simpler components that can be refined independently. When all components are implemented, an implementation of the whole can be automatically generated: an operating system with a particular virtual memory management and with a particular paging policy. Use of diagrams (specifically, hereditary diagrams) allows the user to retain information about a specification during the design process. Taking the colimit of simple specifications can destroy the structure of the spec. The described embodiment of the present invention allows a user to define a specification that is a hereditary diagram and to perform the colimit operation on the hereditary diagram. This carrying information in a diagram brings the colimit operation into lazy mode. Fig. 6 shows an example of the colimit operation for a hereditary diagram. Various intermediary choices can be made by the user as to how to define a diagram. For example, one may wish to instantiate the virtual memory parameter VM to VM_0, but to keep the page-in policy parameter PP open. The pspecification VM_0 can then be protected as a diagram 650. The colimit operation can then be applied in the category of diagrams, rather than specs. Note that Fig. 6 shows an example of a hereditary diagram in which at least one node is a diagram. The parameter VM to be instantiated for, lifts to a trivial diagram as well as the specification OS. The colimit of the resulting diagram yields the specification OS parametric over PP as a diagram. Fig. 7 shows another example of the colimit operation for a hereditary diagram. Implementation details of colimits of hereditary diagrams are discussed below in connection with Figs. 10-27. Shape changes of even simple diagrams quickly become too complex for human beings to solve intuitively. An automated method is needed, such as that shown in detail herein. Figs. 8(a), 8(b), and 8(c) show an example graphical user interface (GUI) for the colimit operation of a hereditary diagram. The display of Figs. 8 and 9 preferably are generated by specification software 110. In Fig. 8(a), the user has defined a hereditary diagram. An initial (parent) specification is named Bag-Diagram. Fig. 9(c) shows details of Bag-Diagram. (The user may or may not choose to display the detail of the diagram Bag- Diagram and may instead display only the name of the diagram as shown in Fig. 8(a)). In this example, the user has refined the parent specification twice, to yield: Bag-as-Seq-Dg and Bag-Seq-over-Linear-Order. Figs. 9(d) and 9(e) show details of these diagrams. (The user may or may not choose to display the detail of the diagrams and may instead display only the names of the diagrams as shown in Fig. 8(a)). In Fig. 8(b), the user has selected the diagram having Bag-Diagram as its parent node and has indicated that he wishes to refine the hereditary diagram specification via the colimit operation. Although the disclosed interface uses a drop-down menu to allow the user to indicate the colimit operation, any appropriate interface can be used. In Fig. 8(c), the colimit is named Diagram-5. Fig. 9(j) shows details of this diagram. (The user may or may not choose to display the detail of the diagram and may instead display only the name of the colimit diagram as shown in Fig. 8(c)). Figs. 9(a)-9(j) show an example of operations initiated by the user to further illustrate the colimit operation for a hereditary diagram. Fig. 9(a) shows an initial hereditary diagram. Fig. 9(b) shows an example of the result of the colimit operation indicated by the user. Fig. 9(c) shows an expansion of the Bag-Diagram requested by the user. Fig. 9(d) shows an expansion of the Bag-as-Sequence-Diagram requested by the user. Fig. 9(e) shows an expansion of the Bag-Seq-over-Linear-Order-Diagram requested by the user. Figs. 9(f)-9(i) show details of determination of the colimit of the hereditary diagram of Fig. 9(a). Fig. 9(f) shows a shape of the shape colimit, which is the shape that the colimit will eventually have. Fig. 9(g) shows an extension of the Bag-Diagram in accordance with the shape of the colimit. Fig. 9(h) shows an extension of the Bag-as-Sequence-Diagram in accordance with the shape of the colimit. Fig. 9(i) shows an extension of the Bag-Seq-over- Linear-Order-Diagram in accordance with the shape of the colimit. Fig. 9(j) shows an expanded version of Diagram-5, which is the colimit of the hereditary diagram. Note that the colimit has the shape of the diagram of Fig. 9(f). In a preferred embodiment, the specification associated with a particular piece of software can be viewed in a drop-down window or similar user interface device associated with an appropriate node. A description of Specification-Carrying Software Code Fig. 10 is a block diagram of a specification-carrying software code system performing automatic code generation in accordance with the specification-carrying software code. Fig. 11 shows an example of specification morphism. The spec reflexive-relation has an axiomatic reflexivity property, while the spec transitive-relation has an axiomatic transivity property. A spec Preorder-relation must have both these properties and is a colimit as described above. Fig. 12 shows an example of composition. Suppose we want to formally specify a simple operating system (OS). There are large fragments of the theory that can be abstracted away, for example, the structure of the system does not have depend on a particular virtual memory (VM) implementation. Thus, the formal VM requirements can be taken as the parameter of the formal OS specification. Similarly, a particular VM system, VM_0, can be parametric in paging policies (PP). Thus, the parameter VM can be instantiated to another parametric specification VM_0. In this way, a complex system naturally decomposes into simpler components that can be refined independently. When all components are implemented, an implementation of the whole can be automatically generated: an operating system with a particular virtual memory management and with a particular paging policy. Fig. 12 also shows an example of an example screen shot showing the interplay of parameters and diagrams. Fig. 13 shows an example of a diagram. It is important to understand that we want to bring the colimit operation into a "lazy' mode, i.e., to carry around the "aquarium" of information for as long as possible. IN this example, this means that it is desirable to preserve the "module" diagram structure for as long as possible without flattening it out into a colimit. Fig. 14 shows an example of a parameterized diagram. The software tool shown here is highly adaptable. Various intermediary choices are possible, in between the eager option of evaluating colimits at the first opportunity and the lazy option of deferring them. For example, one may wish to instantiate the virtual memory parameter VM to VM_0, but to keep the pagination policy parameter (PP) open. The pspec VM_0 can then be protected as a diagram. The colimit operation can then be applied in the category of "diagrams" rather than "specs." The parameter VM, to be instantiated for, lifts to a trivial diagram, as well as the spec OS. The colimit of the resulting diagrams yields the spec OS parametric over PP as a diagram. In a simple case, such design methods can be supported without any automatic support, since the human user can keep track in his or her head. But, for a scalable design tool, automatic support is not only convenient, but necessary. Fairly small diagrams lead to graph theoretic and logical computations unfeasible for a human user. It is often not immediately obvious how to change the shape of diagrams, and even less which specs to place in association with which node. Moreover, the diagram method lifts from specs to diagrams themselves, to diagrams of diagrams, etc. Fig. 15 shows an example of how specification-carrying software can be used in domain-specific software synthesis. This example illustrates how the "algorithm theories" could be composed with the "domain theories" to produce the scheduling algorithm. In this example, various possible algorithms are available, but the transportation domain, for example, has certain constraints that must be met. Fig. 16 shows an example of symbols used to explain EPOXI, an embodiment of a specification-carrying software code system. Figs. 17-19 show more EPOXI-related examples. Fig. 20 shows an example of the impact of specification-carrying software code. Fig. 21 shows an example of a Boolean gauge for software composability. Fig. 22 shows an example of a Boolean gauge for software composability of a software wrapper. The following paragraphs, which form a part of this specification, contain further discussions of specification carrying code and of the EPOXI system, which is, of course, an example embodiment of the present invention and is not to be taken in a limiting sense. P T/US01/12409 EPOXI 1 Ideas What is it? EPOXI is a software evolution tool supporting: Representation of structure and behavior of systems and components. The structure is represented by algebraic specs; the behavior by programs satisfying them. Currently, programs 1 are presented as evolving specs, or especs. The framework combines the features of Abstract State Machines ' (f.k.a evolving algebras ' ), refinement techniques " based on the with the Floyd-Hoare-Dijkstra-style formalisms, all couched in a categorical setting a la Burstall and Goguen — stratified by our own hereditary diagrams, which should provide a flexible abstraction and composition mechanism, and allowing us to go beyond the low-level program development of Floyd-Hoare-Dijkstra. Development by refinement. On the spec side, this is the familiar Goguen- Burstall story: abstract, reusable specs are refined until provably correct implementations are pinned down. This is the source of the categorical framework of Specware. Related or even identical structures can be given different presentations (e.g. semiJattice as a poset vs. semilattice as a monoid); the spec morphisms (interpretations) are necessary to change signatures, thus allowing the structures to be imported and refined. Simple conceptual components are conjoined into large specs by colimits. Parametric specifications (pspecs) are a special kind of interpretations; they are instantiated by pushouts. Our first challenge is to reflect this categorical framework on the program side. The practice of program refinement is extensive, and includes many 1 As a matter of genericity, other presentation of programs, including the specific programming languages, should be catered for in the future. More in the section States and events. examples in the ASM literature. However, the formal framework of es- pecs, catering for refinement and composition not only of specifications, but also of their implementations, requires a novel categorical framework, encompassing algebraic specification development and abstract state machine programming. We believe that this unified framework will be not only more powerful than the sum of its parts, but also more usable than either part. (More details below.) Preservation of design records. This is the Designware idea. This is what we need diagrams for. A colimit mixes the components together, in general irreversibly. 3 In order to preserve the conceptual templates for reuse or modification, we compute the colimits lazily, and carry around boxed diagrams. They protect the conceptual subdivisions, while displaying the conceptual "associations" . Parametric specifications (pspecs) can be viewed as a special sort of diagrams. Substituting a pspec in another pspec then boils down to a diagram colimit. Since each spec can occur in many different diagrams. Diagrams are therefore not implemented over the actual specs, but using the references (virtual links) to them. 4 Representation of systems of systems. A system can be boxed and made into a component of a higher-level system. This high-level view of systems, abstracting from its functionality 5 to mere structure (of structures... ) is ubiquitous in software engineering: it is one of the main features in all important Architecture Description Languages, as well as in TJML and its kin. This leads to hereditary diagrams. In fact, already catering for the substitution of pspecs into pspecs parametrized over pspecs requires developing the main features of a hierarchy of diagrams. As before, a diagram of diagrams is not built over actual diagrams, but over the references (virtual links) to them. The different aspects of a given system can be captured by different diagrams, cross-cutting the same subsystems in different ways. Domain specific extensions. The current project tasks are geared towards fairly wide domains: 3 The soup cooking metaphor. 4 In this way, the computation of a colimit leaves its nodes intact. 5 Abstracting away the functionality is sometimes viewed as lack of semantics. If semantics concern the denotational, input-output view of computation, and the operational, execution order view, then the architectural concerns are indeed intended to abstract away from semantics. However, the conceptual distinctions within the structural building blocks of software (e.g. components, connectors, ports, roles, styles, instances) do have a meaning, and there is some "structural semantics'' behind them. The problem is that we still didn't uncover its mathematical form. • DASADA requires that we represent the concepts of architectural connector and component in EPOXI, capturing their composition, and to translate some of the typical ADLs, Acme/ Armani, C2, and Rapide being the most interesting cases; • MOBIES requres a representation of Statecharts • PCES will probably require (PCES II surely) generating (or at least outlining the methods to generate) middleware generators, able to adapt to different platforms. • DAML The later tasks should include more specific, and perhaps commercially viable extensions. Simulation, monitoring, adaptation. Representing both structure and behavior would allow us to simulate and test systems and processes under design. Moreover, the tool support for especs allowsus to build specification carrying modules: self-reflective software components, equipped with a record of their own intent, functionality, structure, or any other constraint, a glorified API, or a security certificate. (Cf. model integrated software, and proof carrying code.) Such components would have enough information for design-time and run-time gauging and self-testing, reconfiguring, and adaptation: e.g. changing the algorithm according to the observed distribution of the inputs, and recompiling on the fly. Genuinely autonomous agents will surely need to carry a specification of some sort, and a capability to interpret it and align with the observations. Comparisons Especs vs ASMs. While our first concerns are compositionality and refinement as vehicles of software production, the ASM framework is primarily intended as a universal and practically convenient framework for representing sequential algorithms. 6 Hence the differences: • While we are thus very eager to introduce rich typing distinctions in our language, Gurevich makes a point of keeping the underlying language minimal, and standard: many-sorted algebra is reduced to single-sorted partial algebra (over the union of sorts), and non-algebraic predicate logics are reduced to higher-order algebra. 6 If refinement is viewed as "decreasing the amount of non-determinism" , then this restriction directly contradicts our intent. • While we use a rich categorical framework of translations, diagrams and colimits for composing and maintaining compositional systems, with reusable components, the core ASM is intentionally kept unstratified. (There are efforts to accomodate hierarchical ASMs , but based on action refinement, which seems to contradict the idea of algebras-as-states, with rewrites as computation steps.) While display explicit interfaces, in order to deal with communication as a irst-class citizen, all communication in ASM is modeled via shared variables. EPOXI vs VDM and B. What can we do that other specification development frameworks that represent state cannot? Already on the level of specifications alone, the categorical framework offers deep advantages: controlled sharing: import specs A[R] and B[R] in C; how do you identify the two copies of R in C1 H Jϊ is a basic type, e.g. the reals, it is usually hardwired that its imports are identified. But what then if you really need to import two differently implemented copies of R — e.g. if R are the complex numbers, and you need both cartesian and polar coordinates?) quotients: some of the operations of R can be constrained by their usage in A and in B; what if they are constrained differently in each case, and you want to have just one copy of R in C, which should satisfy both sets of constraints? how do you calculate that pushout? realizing that the imported specs can be quotiented pretty much amounts to discovering the spec morphisms. equivalence of specs: expressing the fact that the idempotent commutative monoids and the posets with all least upper bounds implement the same datatype — requires two full fledged specware-like interpretations. quotienting, i.e. identifying operations, soon leads to the realization that a basic operation from the import can be identical with a complex derivable operation, this pretty much amounts to discovering interpretations. When it comes to representing behavior, the categorical framework for composition is only more important. While categories have been eminently present on the scene of algebraic specs, although perhaps not in truly usable tools, no one has been using categories of behaviors to refine, compose, or maintain software components. Finite automata are represented, various operations on them are defined, without even touching the question which two behaviors are equivalent, or bisimilar. How do you then guarantee that an automaton you have built refines another one, in the sense that it can simulate it? With respect to behavior, extant formal methods, with their direct approaches to building automata, are conspicuously close to everyday approach to building programs. How to build it? EPOXI is made of two basic building blocks: theories and translations 7 (known a theory morphisms). Each of them comes about in two varieties: global and local. • A theory formalizes, in predicate logic, what is known about a system, or an artefact in general. — A spec is a theory valid globally, it gives a static view of some properties that the described artefact is required to maintain at all times of its life. — A state description 8 description, on the other hand, is a theory valid only locally, at some points of computation, but perhaps not at others. The behavioral dynamics of the system or artefact is captured by representing its transitions from state to state. • A translation is a morphism between theories: it maps the language (signature) of one theory into the terms of another one, while preserving their meaning and validity: the main syntactical categories are preserved, and the theorems are mapped to theorems. (The latter is achieved by mapping axioms to theorems, and preserving the logical operations.) — An interpretation is a global translation of one spec into another. Specs and the interpretations constitute the Specware category. — A transition from state A to state B is a local translation from state description (theory) B to state description A. Mapping a term of one state into a term of another one is viewed as term rewriting: a function definition update, or overriding, or as an assignment statement. Transitions are thus construed as computation steps. Example Let us see what a simple Dijkstra-style program derivation would look like in this framework, stad denotes a state description, step a transiton. 'This is not a standard name, but it does tell what they do. 8 I tried to stay away from this overloaded word, and use "mode" as proposed, but it just doesn't sound right: "mode of the world", "ever-growing mode", "Abstract Mode Machines", espec GCD is spec var X, x, y : Nat axiom divisor_x is ex(z:Nat) X*z = x axiom divisor_y is ex (z: Nat) X*z = y axiom greatest.common is fa(Z: Nat) (ex(z_x,z_y:Nat) Z*z_x = x and Z*z_y = y) then Z <= X end-spec prog stad One init [x,y] is end-s d stad Two fin[X] is axiom ax_x is X = x axiom ax_y is Y = y end-stad step Loopl : One -> One is x |-> x-y cond x>y end-step step Loop2 : One -> One is y l-> y-x cond y>x end-step step Out : One -> Two is X l-> x cond x = y end-step end-prog end-espec Suppose we want to refine this espec to output not only the greatest common divisor X but also the least common multiplier Y. Of course, one begins by refining the specification. But the presence of the program constrains the possible refinements: indeed, the added axioms must be invariant under all steps of GCD. For instance, we cannot simply add the axioms analogous to the above are added, saying that Y is a multiplier of x and y, and the least such, because the implemented GCD program does not satisfy them. Indeed, the values of x and y are repeatedly overwritten during the computation; and while the greatest common divisor remains invariant, the least common multiplier of each new x and y is different. So there is no number Y which would be the multiplier of all such intermediary pairs. To refine the specification GCD, we must to find an invariant of the program GCD, expressing the fact that Y is the least common multiplier. The usual trick is: add more variables. espec GCD_and_LCH is import GCD spec var Y, ιι, v : Nat axiom invariant is x*u + y*v = 2*X*Y end-spec prog stad One init[x,y] is axiom ax_u is u = y axiom ax_v is v = x end-stad stad Two fin[X,Y] is axiom ax_uv is Y = (u+v)/2 end-stad step Loopl : One -> One is v | -> v+u end-step step Loop2 : One -> One is u |-> u+v end-step step Out : One -> Two is Y |-> (u+v)/2 end-step end-prog end-espec The phrase "import GCD" means that whatever is written in the spec of GCD_and_LCM gets added to the GCD spec, and whatever is written in a state description, or in a step, gets added to the imported one. Only the increments get displayed. In general, all states and steps of an importing espec should be obtainded by subdividing the states and steps of the imported espec. More details later. Definitions The above syntax is meant to implement the category of especs, with the refinements as morphisms. 1.1 Especs Definition 1.1 An evolving spec, or espec A consists of • a spec spe j i, and • a program prog^, presented as an automaton: its states are represented as spec A -refinements, and its transitions are assigned εpec A -preserving translations between these theories, and also labelled by spec A -formulas as guards. Formally, prog^ consists of - graph T, with * distinguished initial node i, and a set O of final nodes o; * labelling cond, which assigns to each edge u a formula cond(u) in the language of spec A : this is the guard of u; * distinguished edge id„ : n -t n for each node n, making T into a reflexive graph; a graph morphism ΛA ■ T — > (sρec j4 /Spec) op preserving all of the above structure. If n is a node of T, the codomain of st A ( ) is written as stad (n). If u m - n is an edge of , its image SIΛ(U) is usually written as step A (u). More precisely * stad assigns to each T-node n a state description stad(n), which comes with a translation st^(n) : spec^ — stad(n); * step assigns to each T-edge u : m - n a step (or transition) step(u) : stad(τn) +- stad(n), keeping S. invariant, in the sense that the following diagram commutes. It will often be convenient to extend the above structure by specified input and output interfaces. They are represented as parameter subiheories 10 of the initial and the final states, respectively. Instead of Xi'→stad(i) and Xo<→s ad(o), we shall usually write stad (name) iπit[Xi] and stad (name) fin[Xo]. Explanation. In which sense can a diagram s : T — (S/Spec) op be viewed as an automaton? The idea is that • the states of prog^ are models of spec^, while • the category spec A /Spec is an extension of the category od(spec J ) of models. A state of computation can be viewed as a snapshot of the abstract computer performing the computation. A rudimentary computer can be viewed as a set of stores, with an undetermined mechanism rewriting the stored values. The rewrites are the computation steps, or transitions. Since the specification is, by definition, satisfied throughout the computation, each state is a (static, mathematical) model of the specification: the 10 By definition, the parameter X C S of a parametric specification S[X] can be freely instantiated, without causing any inconsistencies in the parametrized specification. This also captures the idea of interface. elements of the signature are allocated some stores, and the values stored there must satisfy the axioms at all times. So the computation of a program specified by spec^, evolving from state to state, can be envisioned as "jumping" from model to model, in Mod(speCy t ). The fact that spe^ is invariant under the rewrites that implement the computation steps means that they correspond to the model homomorphisms for spec^. A rewrite from a state m to n corresponds to a homomorphism the other way around, m «— π. So automata can be envisaged as diagrams in od(spec A ) op . The objects of spec^/Spec can thus be construed as generalized models of speca- Indeed, each refinement s : spec^ — > S is a representative of a class of spec^-models, the ones that extend it, and satisfy S. Since models are states, the objects of spec^/Spec can be viewed as abstract states, and the diagrams in (spec Λ /Spec) op are automata. Indeed, in practice, one can only deal with such abstract states: it is impossible, and unnecessary, to pin down all the values in all stores, in order to reach a particular model. In reasoning about programs, we are only interested in what happens with some of the stores, and abstract the rest of them away. We seldom pin down single models, but rather classes of models, contained among the models of the global spec. That is why we consider the general state descriptions, and pass from Mod peca) to (spec Λ /Spec) op . State descriptions are the extensions of the global spec, approximating the state; in fact, cutting down the model class, and capturing some subclass, containing the models we are interested in at that point of computation. At the moment when we refer to a state, our state of knowledge does not suffice to discern between the different members of a model class of a state description. As we proceed, and refine our knowledge, we refine state description, and further partition the classes of models. Remarks. The language used in the examples refers to the shape T implicitly, by listing which transition connects which states. By default, the translations spec — stad(n) can be left implicit as well, as they were in the above example, assuming that each state description imports the global spec. In this way, only the actual increments need to be displayed. The requirement that a state machine should never be in two states at the same time means, in our setting, that any two different state descriptions should be inconsistent, m φ n implies stad( ) Λ stad(n) =*> . Determinism, similarly, implies that the guards of any two transitions coming out of one state should also be inconsistent u φ v and dom(tι) = dom(t)) imply cond(u) Λ coπd(«) =* ■ ± Of course, to model distributed systems, one must surely drop the first requirement, and usually the second too. On the other hand, in apparent contradiction with the above, the extensions spec — » stad(n) are allowed to be trivial: e.g., the state description One in GCD didn't add anything to the global spec. Indeed, nothing prevents the graph morphism st : T — (spec/Spec) op from mapping some, or all of T to the identity on S. On top of it, nothing prevents the spec S from being empty itself. But this is a feature, that allows us to deal with the situations when we are given just a program, a bare sequence of computation steps, and need to refine some specification that it satisfies. However, in order to maintain the order of the computational steps of the program, we cannot represent them all as the transitions over one single state, with the empty description. A natural thing to do is to introduce an "program counter" variable, increased at each transition: this would keep the states different, and the given program in sequence. In general, the states described by equivalent, or even identical state descriptions, but distinguished by their names, will always be considered different. Intuitively, one may think of the state names as the values of a hidden variable, present in every state, representing, perhaps, a program counter. Only in this way, the order of the unspecified computation steps can be preserved. The fact that the postcondition of one step may coincide with the precondition of another one should not imply that they should be executed one after another. 1.2 Refinements Since all of the structure, naturally kept implicit in the development practices, is brougt to the surface in the formal definitions, the following one may seem complicated. Its natural implementation in EPOXI will prove the contrary. Definition 1.2 Given especs A and B, a refinement f : A — B consists of: = (spec.4, T A (spec Λ /Spec) 0 *) /r ~ B = (spec B , T B (spec B /Spec)°P) StB • α structure map (or interpretation) S ec > a nd • a behavior map (or simulation) / pr0 g = (fr, ψ), where — /r is a reflexive graph morphism, preserving the initial and the final nodes, — ψ is spec^ -preserving natural transformation, which amounts to the commutativity of s t «P »(/r(")) stad UH c Orrø) ¥>(*) vW spec B StB<*l s B (/) stadjg(fc) step β (») - sta βrø /or eυeri / v : A: -> £ in Ts- Together, f spκ and f prog must also satisfy the guard condition: for every node k in TB and every edge u : fr{k) -¥ n in TA holds stadβ(fc) I- fs P κ(∞nd A (u)) J conds(v) om(tι)-=Jt Intuition. The analogy of Mod(spec) and spec/Spec, explained in the preceding section, now lifts into the analogy of the functors / s "J, ec : spcg/Spec — V spec^/Spec and / s * pe( . : Mod(specg) — >■ od(spec > ι), both induced by precom- posing with an interpretation spt : spec^ — ► spec B . The two components of an espec morphism, the structural and the behavioral map, thus echo the Chu-morphism components, capturing the duality of theories and models. Intuitively, the last diagram tells that the components of φ coherently ex¬ But is every transition of A extended in .9? Yes, insofar as its guard is ever enabled. This is ensured by the guard condition, which, in our context, says that the behavior stg can simulate st A - More precisely, the inverse graph of /r is a simulation in the usual sense of concurrency theory: if fr(k) = m and there is a transition u out of m, enabled in a given run, in the sense that cond ι(«) is true, then there must be an enabled transition υ out of k in JS, with fr(v) = u. The other way around, whenever some υ, fr(v) = u is enabled, u must be enabled. Con on), interpreted by B as Λp ec oπd^u)), is always false. Example Let us specify a small software demon, scheduling jobs on a printer. To begin, assume there are only two jobs to be scheduled. espec Print_crude is spec sort Jobs const ji, j2: Jobs var self : Jobs op ext Wants: Jobs -> Bool op prints: Jobs -> Bool end-spec prog stad A[self] init is Wants (self) = true prints (self) = false end-stad stad B[self] is Wants (self) = true prints(self) = true end-stad stad C[self] fin is Wants (self) = false prints (self) = false end-stad step Go [self] : A [self] -> B[self] is prints (self) |-> true cond (fa z:Jobε) prints (z) = false end-step step Done [self] : B[self] -> C[self] is prints (self) |-> false end-step step Ask [self] : C[εelf] -> A [self] is cond Wants (self) = true end-step end-prog end-espec Notation. The occurrance of the variable self next to the names of stads and steps means that it is a state variable, although it is not constrained by any axioms. In other words, writing A [self] means that A is indexed by the possible values of self, of which there are in this case two, so we have the states A[jl] and A[j2]. The equations in state descriptions, e.g. prints (self) = false, of course abbreviate axioms: the full form would be axiom soιαe_name is prints (self) = false On the other hand, this axiom could further be abbreviated to -φrints(sel ), whereas prints (self ) = true to prints (self ), which we shall do in the sequel. Comment. Similarly like in the ASM framework, the external variable Self, and the external function Wants are assumed to be set and changed by the environment. However, unlike ASMs, especs are not allowed to update external variables or functions. Like the "logical constants" of Hoare logic, the external values in especs are immutable, but can be tested, passed as parameters, etc. In this way, it is ensured that all transitions must be made explicit, including those triggered by the updates of the external, unimplemented operations (such as Wants. Espec Print_ crude, of course, does not capture much of the intended meaning. The least one can ask is that two jobs do not print at the same time. So we wish to add axiom exclusion is (fa x, y: Jobs) prints (x) Λ prints (y) => x = y to the spec. In order to produce a correct espec, we must either show that the added axiom is satisfied throughout the program Print_cn.de, or we must refine that program too. But without additional assumptions, program Print _crude cannot be proved to satisfy the axiom of exclusion. If both Want (j l) and Want (j 2) jump to true at the same moment, there is nothing to stop the transitions Ask[ j 1] , thereby triggered, to call the states A[jl] and A[j2] at the same time; the transitions Go[j l] and Go[j2] will then test the condition (fa z: Jobs) print(z) = false at the same time, find that it is satisfied — and flip print (j l) and print (j 2) to true simultaneously. While no single state of Print_crude falsifies the axiom exclusion, the execution of the program may be distributed, and thus reach the states B[ l] and B[j2] simultaneously. 12 It is thus necessary to establish a central test whether the printer is free, rather than let the agents test this independently. We shall implement this using the function Enter, which will first be implemented later. espec Print_excl is import Print_crude ι a Morals: "A global invariant of a distributed program must be satisfied not only at each state, but also for all runs, possibly concurrent." spec op ext Enter: Jobs -> Bool axiom relate is (fa x:Jobs) Enter(x) => (fa y: Jobs) "print (y) axiom exclude is (fa x,y:Jobs) Enter(x) A Enter(y) => x=y end-spec prog step Go is cond Enter (self) end-step end-prog end-espec As before, the refinement Print e r_de — * ■ Pr±nt.xcl is realized by the identities, both on structure and on the behavior. In order to demonstrate a refinement with nontrivial maps, let us implement the function Enter using a different signature. espec Print_mode is spec sort Jobs const j l , j : Jobs var self : Jobs op ext Wants : Jobs -> Bool op mode: Jobs -> v,q,p , s (* waits, queues, prints , satisfied *) op ticket : Jobs -> (Int + undef) op ext Tick : Jobs -> (Int + undef) axiom ge .ticket is (fa x:J0bs) Tick(x) "= undef => Tick(x) = 1 + max-[ticket (y) |y~=x> end-spec prog stad W[self] init is mode (self) = V end-stad stad q[self] is mode (self) = q end-stad stad C[self] fin is mode (self) = s end-stad step Take [self] : W[self] -> Q[self] is mode (self) |-> q ticke (self) |-> Tick (self) end-step step Go [self] : Q[aelf] -> P[self] is mode(self) |-> p end-step step Done [self] : P[self] -> S[aelf] is mode (self) | -> s ticke (self) |-> undef end-step step As [self] : C[self] -> A[self] is mode(self) |-> w cond Wants(self) = true end-step end-prog end-espec refine Modify : Print_excl -> Print_mode is εpecmap prints (x) l -> (mode(x)=p) Enter (x) l -> (ticke (x)"= undef Λ ((fa y.Jobs) y"=x Λ ticket(y) "= undef) => (ticket(x) < ticke (y) \/ (ticket(x) = ticket(y) Λ x<y))) end-specmap progmap W |-> A P l-> A Take |-> id(A) end-progmap end-refine For brevity, the data with the same names are mapped to each other by default. Since the signatures of the state descriptions in Print_excl boil down to the signature of the global spec, the natural transform ψ is completely determined by the structure map. This is usually the case, and checking whether a given structure is a refinement boils down to checking whether the structure map maps theorems to theorems, and whether the guard condition is satisfied. PUSHOUT of especs: • on the spec side, as in specware • on the prog side, — take the pullback of the shapes T, — map each node (a pair), into the colimit of the two state descriptions, glued over the part identified in the pushout of the global specs; — throw away the inconsistent pairs. This is the PAEALLEL COMPOSITION of the represented software components. SEQUENTIAL COMPOSITION (of coalgebras): the input/output interfaces are pspecs. Take colimit: identify the interfaces, disjoint the rest. The Missing at the time of publication Towards semantics of sellfl-adaptiirve software Dusko Pavlovic Kestrel Institute, Palo Alco, USA dnakoCkβatza . «dn Abstract. WThen people perform computations, they routinely monitor their results, and cry to adapt and improve their algorithms when a need arises. The idea of self-adaptive software is to implement this common facility of human mind within t e framework of the standard logical methods of software engineering. The ubiquitous practice of testing, debugging and improving programs at the design time should be automated, and established as a continuing run time routine. Technically, the task thus requinea combining functionalities of automated software development tools and of runtime environments. Such combinations lead not just to challenging engineering problems, but also to novel theoretical questions. Formal methods are needed, and the standard techniques do not suffice. As a first contribution in this direction, we present a bas mathematical framework suitable for describing self-adaptive software at a high level of semantical abstraction. A static view leads to a structure akin to the Chu construction. An dynamic view is given by a coalgehnaic presentation of adaptive transducers. 1 Introduction: specification carrying code One idea towards self-adaptive software is, veryj rougihly, to introduce some kind of "formalized comments", or Floyd-Hoare style annotations as first class citizens of programs. Together with the executable statements, they should be made available to a generalized, adaptive interpreter, extended by an automated specification engine (e.g., SPECWARE™-style), supported by theorem provers and code generators. This adaptive interpreter would not only evaluate the executable statements, but also systematically test how their results and behaviour satisfy the requirements specified in the formal annotations. Such testing data could then be used for generating improved code, better adapted to the specifications, often on the fly. On the other hand, the formal specifications could often also be refined, i.e. adapted to the empiric data obtained from testing a particular implementation. 1.1 Automated testing and adaptation Coupling programs with their specifications in a uniform, automatically supported framework would, at the very least, allow monitoring correctness, reliability, safety and liveness properties of programs, with respect to the specified requirements, as well as the particular distribution of the input data, and any other, aspects of the execution environment that may become available at run time. In some cases, one could hope for. more than mere monitoring of the relationship between an abstract specification and its implementation. Indeed, software can often be improved and adapted to its specifications in a predictable fashion, once its running behaviour, can be observed on concrete inputs. This is, for instance, usually possible in the cases when the correctness criteria are not absolute, viz when the software only approximates its specification. Software that models physical systems, or stochastic processes, or even just computes genuine real numbers or. functions, is usually of this kind: the irmnitary nature of the output data precludes the exact solutions, which can only be approximated. But the approximations can, in principle, always be improved, on an additional cost. Comparing the actual runs of such software with its abstract specifications may suggest optimizing this cost, say, by adjusting the coefficients in the numeric formulas to the observed distribution of the input data. In some cases, difrerent distributions may even justify applying different algorithms, which can be abstractly classified in advance, so that the adapted code can be synthesized on :he fly. Self-adaptive software can perhaps be compared with an engineer, monitoring the results of his computations, updating the methods and refining the model. The point is here that a part of this process of adaptation can and needs to be automated. 1.2 Dynamic assembly and reconfiguration Furthermore, in a complex software system, the adaptation cycle oC a software component) can take into account not only the run time behaviour of the component itspjf, but also the behaviour of the other, components, and the changes in the environment at large. In order to be combined, software modules must contain sufficient information about their, structure and behavior. Conventional application programming interfaces, APIs, are intended to carry such information but APIs are usually under-specified (they contain just signature/ type information), are often based on unfulfilled assumptions, and are prone to change. Conventional APIs are thus insufficient for the task of assured composition. Ideally, APIs would completel capture all assumptions that influence behavior. However, formally verifying such completeness is usually infeasible — some degree ofl API partiality is inevitable in practical software development. Nevertheless, in dynamically adaptable software, API partiality should continually decrease as the interfaces evolve during the lifetime of a system, together with the specifications and implementations of its components and perhaps even its architecture. We believe that the requirement of dynamically adaptable software naturally leads to the idea of specification-carrying code: an adaptable program must carry a current specification of its functionality, an adaptable specification must come with a partial implementation. Adaptability requires a simultaneous and interactive development of the logical structure and the operational behavior. Moreover, specification-carrying code is the way, to accommodate and support, rather than limit and avoid, the rich dynamics of ever changing interfaces, so often experienced in large software systems, with the unpredictable interactions arising from the ever changing environments. The fact that specifications, implementations, interfaces and architectures in principle never stop changing during their lifetime, should not be taken as a nuisance, but recognized as the essence of the game of software, built into its semantical foundation, and implemented as a design, and when possible a runtime routine. Of course, adjusting the behaviour of autonomous software components to each other, tuning them in on the basis of their behaviour, or getting them to interact in a desired way, can be a very hard task. But if their abstract specifications in a generic language are maintained on a suitable formal piatform. and kept available at run time, their possible interactions and joint cons stency can be analyzed abstactly, e.g. using theorem provers. and their implemer_xations can be modified towards a desired joint behaviour. This may involve reircpiemenmng, i.e. synthesizing new code on the fiy, and is certainly not a straightforward task. However, it seems unavoidable for the independently implemented components, necessary for the compositional development of systems — and we believe that it is also within the reach of the current software synthesis technologies. In any case, a software system that needs to adapt its code and behaviour the run time data, or to the changes of the environment, while maintaining its essential functionality, will surely need to carry and maintain a specification of this functionality in some form, perhaps including a history record, and a current correctness certificate. Since this task, and the structures involved, clearly go beyond the existing software methodologies, a careful analysis of the semantical reprecussions seems necessary. Building the suitable design and development environments for the specification carrying self-adaptive software w-iii require a mathematical framework with some nonstandard features. In the present paper, a crude, initial picture of some of these features is outlined. Section 2 describes the structure and the intended interpretation of an abstract category of specification carrying modules. This can be viewed as a first attempt at denotational semantics of such bipartite modules, involving a structural and a behavioral component. Section 3 briefly outlines the structures needed for a dynamic view oE the adaptation process, and tho way to adjoin them in the described semantical framework. 2 Category of specification carrying programs As the name suggests, a specification carrying program consists of a program P, a specification S. and a satisfaction, or model relation (= which tells how they. "cany" each other, ϊ.e. establishes the sense in which P satisfies SI. Formally, a specification earning program is thus a triple (P, |=,S). But what precisely are P, f=, and S? In principle, a formal specification S is a logical theory in a generic specification language (e.g., the higher-order predicate logic). A program P, on the other, hand, here, means a description of a computational behaviour in one of the available formalisms: it can be a transition system, an automaton, or simply a piece of code in a sufficiently general programming language. Finally, the satisfaction relation q = φ tells which of the formulas IU of S are satisfied at each of the states a of P. Although diverse formalisms can be used for the particular presentations of P and S. they can always be uniformly represented as categories. The classifying categories P and S are derived respectively from the program P and the specification S using the known procedures of operational semantics and categorical model theory. The satisfaction relation (= then becomes a functor from F x S. But we shall not attempt to present this abstract framework directly, but rather begin by motivating its main features and the underlying ideas, especially as they arise in extant frameworks. Some of the underlying mathematics will be outlined, but the details are beyond the scope of this presentation. 2.1 Contracts: the game of refinement and adaptation Intuitively, an adaptive module (P. ^=, S) can be thought of as a contract between a programmer and a client: the client specifies the requirements in S, and the programmer provides the program P in response. This intuition yields a conceptual basis for the discipline and theory of software refinement [1, 11]. The process of software adaptation can now be viewed as a game played between the client and the programmer: the former refines the specification 5, say to ≤". and the latter tries to respond accordingly by adapting the program P to P' accordingly, i.e. in such a way that the satisfaction |= is preserved. This means that a predicate φ from S should be satisfied in a state q of P if and only ifi the translation φ' of φ to S' is satisfied in all states q' of P' that simulate the state q. I summary, an adaptation transformation of (P, =,S) into (P' ? f=', S') consists of - a simulation P 4^- P', and - an interpretation S -→. S', such that for all predicates φ in S and all states q 1 in P* holds q' f=' f s (φ) <=> f P (q') |= Ψ (1) The pair / = {fρ,, fβ), satisfying (1), is an adaptation morphism {P, r*, S) - (P', r *',St) An abstract semantical framework for software adaptation is thus given by, the category C of specification carxyiiπg programs, viewed as contracts C. = (Pc, f=c ■ Sc) : ith the adaptation morphisms between them. Contracts as intervals. Note that the. morphism / : (P, -S — ► (P (=. S"f is running concurrently with the specification refinement fs '■ 3. — *• S', but La the opposite direction from the simulation fp : P' — ► P. An imprecise, yet instructive analogy is that a contract (P, |=, S) can be thought of as a real interval [p, s). The desired software functionality, that 5 and P are approximating in their different ways, then corresponds to a point, or. an infinitely small interval contained between p and s. The refinement/adaptation game of the client and :be programmer now becomes an interactive search for greater lower bounds p, and smaller upper bounds s, i.e. for smaller and smaller, intervals, nested in each other, all containing the desired point. This process brings the programs and -heir specifications closer and closer together, so that they better approximate :be desired functionality from both sides, i.e. in terms ofi the behavior and the structure. Viewed in this way, an adaptation morphism becomes like a formal witness of the interval containment p,s] 2 p s'] *=> p ≤ p' Λ s' ≤ s. The morphism (P, {=,≤!) — ► (P\ (=',S1') thus corresponds to the containment Pi 5 2 & a '] ι the interpretation S — ► S' to the relation s > s', and the =rnrulation P' — H P to jf > p. Of course, the analogy breaks down on the fact that a specification 5 and a program P are objects of different types. Nevertheless, the satisfaction relation = measures "the distance" between P and S, and the mathematical structures arising from refinement/adaptation remain similar to those encountered in approximating numbers by intervals. In spite of its imprecision, the metaphor re- raains instructive. Moreover, the view of 5 and P as βhe approximations of an ideal point where the program is optimally adapted to the specification, does =eem conceptually correct. To better approximate this point. S prescribes a minimum of structure necessary, for expressing some part of the desired functionality, whereas P provides the simplest behaviour sufficient for implementing it. The client's strategy is to refine S to S' along fs • S — *■ S', to constrain the input/output requirements more, and make programmer's task harder. The programmer, on the other hand, must enrich the behaviour P to P', in order to better fit the task. This means that P' must at least be able to simulate P, along fp : P' — * P.. In any case, the client provides the input/output requirements S, whereas the programmer supplies in P the computation steps transforming the inputs to the outputs. Sβmantically speaking, the structural, "upper bound" descriptions of software are thus driven by its denotational aspects, viz the structure of the datatypes and the functions that need to be refined and implemented. This is summarized in a spec S. On the other hand, a ■ 'lower bound" description of a desired piece of software is driven by its operational aspects, and in our particular case by the behaviour of a given implementation P, that needs to be adapted, or optimized far better performance. While conceptually: different, the structural/denotational and the behavioural/operational aspects can be captured in a uniform setting [21], which would perhaps make the idea of contracts as intervals more convincing for some readers, or at least give the "intervals' 1 a more familiar appearance. However, as they stand, specification carrying programs can be represented and implemented using mostly the readily available semantical frameworks, on the basis of the extant specification and programming environments. 2.2 Institutions, and satisfaction as payoff The bipartite setting of specifications coupled with programs via a satisfaction relation will probably not appear unfamiliar to the categorically minded members of the software specification community. They will recognize the category C-of contracts as conceptually related to institutions, although not in all details. A institution is a very general model theoretic framework, introduced by Goguen and Burstall in [3], and pursued by many others in several different forms. Its main purpose was to fill a conceptual gap in semantics of software. While, the formal methods of software engineering are in principle based on universal algebra and model theory, with specifications statically describing some computational structures, programs at large are dynamic objects, they change state, and behave differently in different states. And while the mathematical theories completely determine the classes of their static models, as well as the notion of homomorphism between them, the software specifications do not pin down the programs that realize them '2 . In model theory, the Tarskian satisfaction relation j= is a fixed, primitive concept; in theory of software specifications, on the other hand, there are many degrees of freedom in deciding what does it mean for a program to satisfy a specification, in particular with respect to its operational behaviour. It is then reasonable to display an abstract satisfaction relation (= as a structural part of an institution, that can be varied together with theories and models, while stipulating which models satisfy which theories. 3 Following this conceptual lead, the satisfaction relation can be generalized from an ordinary relation, where q f= φ is .evaluated as true or false, to an M- valued relation, where q (= φ can be any element of a distributive Lattice, or Not in the sense t iae ail programs implementing a specification can be effectively derived from the specification, like all mathematical models of a theory, and indeed the whole model category, are effectively determined by the theory. 3 Institutions thus bridge the gap between static theories and dynamic models hy allowing the ahstract satisfaction relation to vary. Another way to bridge this gap is to introduce dynamics into theories. This is one of the ideas behind Gureviob's Ahstract State Machines (formerly known as evolving algebras) [4]. a suitable category, say M, measuring the degree to which the condition ■ is satisfied at the state q. In standard game theoretic terms, the relation (= now becomes the payoff matrix, displaying the value of each programmer's response to each client's call. Indeed, if the formulas of S are understood as the set of moves (or strategies*) available to the client, and the moves available to the programmer are identified with the states of P, then the satisfaction = becomes an P x S-matrix of the elements of M, i.e. a map μ : P X S — + M (2) - a simulation P ^~ P'. - an interpretation S — £*• S', and - for each q" 6 P' and xϋ 6 S an arrow (/ («*) ) - → * («* / β (ιM) 0) in , with the suitable naturality condition. 2.3 Towards functorial semantics of contracts In order to express the above naturality condition, or work out a generic representation of the category C of contracts, one needs to present the specifications, and the programs in terms of their respective classifying categories. Given a specification S., say as a theory in a predicate logic, the objects of the induced classifying category § will be the well-formed formulas of 5, modulo J :he distinction is of no consequence here the renaming of variables (α-conversion). The arrows are the functional relations definable in S, modulo the provability. For instance, take formulas oι(x) and β(y) in S, as the representatives of objects in S. By renaming, we can achieve that their arguments x and y are disjoint. An §-arrow from a(x) to β(y will be a predicate ώ(x, y), such that a(x) \- 3y. ϋ(x,y) tf(x,y Λ( B , y") r- v' = y" can be proved in S. The arrows of S thus capture the theorems of 5, whereas the objects capture the language. More details can be found in (17]. The point o£ presenting a theory S as a category S is that the models of S can be obtained as the functors S — ► Set, preserving the logical structure. This is the essence of functorial semantics [7], and the foundation of categorical model theory [8,9]. The applications to software engineering are discussed in [17]. Related, more direct, but less uniform procedures allow deriving categories from programs. They usually go under, the name of operational semantics [18, 22). and come in too many varieties to justify going into any detail here. Assuming that a specification S and a program P have been brought into a categorical form, and presented as classifying categories S and P, the satisfaction relation (=: § x ? — ► M will transpose to a structure preserving functor S — ► M?. When the category M, measuring satisfaction, is taken to be the category Set of sets and functions, p= will thus amount to a model of! S in the universe Set ? of sets varying from state to state in P. A logically inclined reader, may be amused to spend a moment unfolding the definition of adaptation morphisms in this model theoretic context, and confincing herself: that such morphisms indeed preserve the given sense in which a program satisfies specification, or improve it along the suitable homomorphisms of models. In any case, the naturality condition on the third component of: the adaptation morphisms as defined the preceding section can now be expressed precisely, on the diagram displaying the involved composite functors. Jp 3 Aldaptive interpreter as coalgebra While the described category of contracts, implemented and supported by suitable tools, provides the structural framework for software adaptation, it still does not provide a special handle for automated, on-the-ny adaptation and reconfiguration. The dynamics of self-adaptive software requires an additional dimension, to be added to in the actual implementation of the specification carrying modules. The main issue thus remains: how to implement an adaptive interpreter, able to compute with self-adaptive, specification carrying modules Given a contract (P, f=,S), the adaptive interpreter should be able to: - evaluate P, - test whether the results satisfy S, - adapt P, or. assist program transformation P « — P', - support refinement 5 — t S'. In a standard setting, the denotation off a program P is a function p : A — ► B. where .4 and B are the types of the input and the output data, respectively. An adapted program P' will yield a function p' : A — ► B (where we are ignoring, for simplicity, the fact that the data types .A and B can be refined). If adaptation is viewed as a computational process, all the instances of an adapted function that may arise can be captured in the form p : I7 x - — ► B where Σ is the carrier of adaptation, probably a monoid, or a partial order. The stages of the adaptation σ, σ' ... € 27. can be associated with the successive refinements 5, 5' ... of the specification of the adaptive program, and will be derived from them by one of the model theoretic methods developed for this purpose (i.e. as the "worlds", or the forcing conditions induced by the refinement). 5 AU the instances p,pt ... : A — ^ B of the adaptive function will now arise by evaluating p at the corresponding stages σ, σ' ... 6 Σ, viz p(x) = p(o-,x) p'(x) = p(σ\x) In this way, the process of adaptation is thus beginning to look! tike a rudimentary dynamic system. The process of sef/-adaptation will, of course, be a system with the feedback p : ∑ x A — * B x Σ computing at each stage σ € 17, and for each input x € A not oπlyi the output j; = pό(σ,x) € B. but also the next stage σ' — p\(σ,x) € Σ, with a better, adapted function p'(x) = p(σ',x). Extended in this way along the coordinate of 5 The stages of adaptation should not be confused with the computational states, through which the execution of a program leads. The execution runs of a program from state to state are. in a sense, orthogonal to its adaptation steps from stage tα stage. „ ^ , PCT/US01/12409 adaptation, the specification carrying programs viewed as contracts come in the form : i7 x S x P -→ M x The predictable adaptation stages are structured in Σ and controlled by the resumption component of j=. Alternatively, automated adaptation steps can be encapsulated in specifications and programs themselves, by individually extending the particular functions from stage to stage. Independently on the level on which it may be captured, the denotation of a self-adaptive function will in any case be a transducer p. Transposing it into a malgebra p : Σ → {B * Σ) A brings with it the advantage that the behaviour preserving maps now arise automatically, as coalgebra homomorphisms. (Instructive examples and explanations of this phenomenon can be found, e.g. in {20].) But even more importantly, it allows a considerably more realistic the picture, since it also allows introducing various computational monads T on the scene. A coalgebra in the form i : ∑ → {T(B x Σ)) A captures an adaptive family of computations involving any of the wide range of features (nondeterrninism, exceptions, continuations... ) expressible by monads ;ιoι. In any case, combining monads and coalgebra will ensure a solid semantical foundation not just for adaptive interpreters, but also for implementing the design environments for self-adaptive software. Explaining either of these theories is far beyond our scope here, but monads seem to have been established as a part of. the standard toolkit of functional programmers, and the material about them abuπds. Some coalgebraic techniques for implementing processes have been presented in [5, 15, 16]. References 1. R.-J. Back and J. Von Wright, The Refinement Calculus: A Systematic Introduction. (Springer 1998) 2. M. Ban. *• Autonomous Categories. Lecture Notes in Mathematics 752 QSpringer, 1979) 3. J.A. Goguen and R..M. Burstall, Institutions: abstract model theory for specifications and programming. J. of the A.C.M. 39(1992) 95-146 4. V. Gurevich. Evolving algebras 1993: Lipari guide. In: Specification and Validation Methods, ed. E. Bδrger, (Claredon Press 1995) 9-37 3. B. Jacohs. Coaigehraic specifications and models of deterministic hybrid systems. In: Proc. A MAST. ed. M. Nivat, Springer Lect. Notes in Co p. Sci. 1101(1996) 520-535 Y. Lafont and T. Streicher, Games semantics for linear, logic. Proc 6 th LICS Conf, (IEEE 1991) 43-49 P.W. Lawvere, Functorial Semantics of Algebraic Theories. Thesis (Columbia University, 1963) - Makkai and R. Pare, Accessible Categories: The Foundations of Categorical Model Theory. Contemporary Mathematics 1041 (AMS 1989) M. Makkai and G. Reyes, First Order. Categorical Logic Lecture Notes in Mathematics 611 (Springer 1977) E. Moggi, Notions of. computation and monads. Information and Computation 1993 C. Motigan, Programming from Specifications. (Prentice-Hall 1990) G.C. Necula, Compiling with Proofs. Thesis (GMU 1998) V.C.V. de Paiva, Th'e Dialectica categories. In: Categories in Computer Science and Logic, J. Gray and A. Scedrov, eds., Contemp. Math. 92 (Azner. Math. Soc, 1989) 47-62 D. Pavlovic. Chu I: cofree equivalences, dualities and ^-autonomous categories. Math. Structures Camp. Sci. 7(1997) 49-73 D. Pavlovic, Guarded induction on final coalgebras. E. Notes m Theor. Camp. Sci. 11(1998) 143-160 D. Pavlovic and M. Escaaio, Calulus in comductive form. Proc l? b LICS Conf. (IEEE 1998) 408-417 D. Pavlovic, Semantics of first order parametric specifications, in: Formal Methods '99, J. Woodcock and J. Wing, eds., Springer Lect. Notes in Conrp. Sci. 17O801999) 155-172 G. Plotkin, Structural Operational Semantics. Lecture Notes DAIMI-F 1901981) V. Pratt, Chu spaces and their interpretation as concurrent objects. In: Computer Science Today: Recent Trends and Developments, ed. J. van Leeuwen, Spxiαgeo Lect. Notes in Comp. Sci. 1000(1995) .l.J.M.M. utten, Universal coalgebra: a theony of systems. To appear in Theoret. Compui. Sci. D. Turi and G.D. Plotkin, Towards a mathematical operational semantics. Im Proc. 12 LICS Conf. (IEEE 1997) 280-291 G. Winskel and M. Nielsen, Presheaves as transition systems. In: Proc. of POM'IV 96, DIM ACS Series in Discrete Mathematics and Theoretical Computer Science (AMS 1997) 129-140 CO c Towards semantics 00 O of self-adaptive software m CO I m m en c m Dusko Pavlovic r Kestrel Institute Palo Alto, USA Outline! CO c 00 CO Self-reference and adaptability Specification carrying code: contracts Example: adaptive sorting Semantics of adaptive interpreter Ongoing work CO c "In a high-level language, the programmer is 00 CO deprived of the dangerous power to update his m CO own program while it is running. I m m 73 [...] c m ro The introduction of [references] into high-level languages has been a step backward from which we may never recover." C.A.R. Hoare, Hints on programming language design (1973) (Self-) references are useful | " It is often convenient to introduce objects as records which, beyond the simple fields [...], include fields that refer to other objects. [...] A class may be its own client. [An instance of such a class] represents, e.g., a bank customer that is her own guarantor." B. Meyer, Object-oriented Software Construction (1988) In Object Oriented Programming, references, disciplined into objects, are elevated to the principal building blocks of programs: OBJECTS = REFERENCES + MUTABLE STATE (Self-)adaptation is necessary j Modern software must withstand: • on-the-fiy testing, debugging, and adjustment This is just a one-slide, unfaithful summary of the point which some of the people in the audience have been making in detail, and for years. Schrobe and Laddaga have pointed to the problems with software, and vigorously pursued realizing various projects of "active software * : tolerant, autonomous, negotiating agents, self adaptive. If a program is to adapt, it must at least know what is the game, ie what is it supposed to do. SO IT MUST CARRY SELF-REFERENCE. Ideas towards adaptability Artificial Intelligence: genetic programming Complexity Theory: self-testing programs Software Engineering: model integration Genetic programming strives automatic discovery of optimal algorithms, by *blind* search, governed by the suitable *selection* procedures. The self-testing programs are built upon the basic fact that VERIFYING A WITNESS IS EASIER THAN CONSTRUCTING IT. Blum's self-correcting programs implement and automatize the basic idea of fault tolerance by redundancy. Stipanovicz's model-integrated computing, dynamic reconfiguration is achieved using model-based generator traverses model libraries and synthesizes "executable models". He was designing Digital Signal Processing applications where on-the-fly adaptation is required to control the engine stress testing processes, too expensive to stop or allow to fail. (In his concrete applications, the various configurations which the system can assume are not unpredictable, and one could, in theory choose between them in a case statement; but there is a very large number of choices, and there are cases when it can be infinite. Here, the idea of self-adaptation does not arise by a qualitative leap, but as an implementation trick, allowing a large or infinite branching in a program. But in principle, models need not to be read from a library. Unknown to the programmer, they can be generated as suitable mathematical theories, or even by a genetic algorithm.) Specification carrying code| software module equipped with a formal specification of its functionality - comments in a generic specification language, or - Floyd-Hoare annotations - as first-class citizens of programs specification given with a (partial) implementation: - executable, or - abstract, but verifiable But the models integrated into Stipanowicz's software are represented as flow diagrams, formalized and implemented ad hoc. We are making the idea of embedding a model in software precise: a model is a spec. (The idea of a model is that it is an abstraction of reality, displaying what we care for, and abstracting away the rest. That is what software specs do in a formal and systematic way.) Accommodate the dynamics of ever changing interfaces, and the unpredictable interactions arising from the ever changing environments. The fact that the architectures never stop changing should not be taken as a nuisance, but recognized as the essence of the game of software, built into its foundation, and implemented as a design routine. Specification carrying code | is formalized in terms of "contracts": ( ) < p , l=, s) P = program s = specification 1= = model relation However, none of the existing mathematical formalisms of computer science accommodates in a uniform way both * structural specifications * and the executable code * satisfying it in some formal sense. On the other hand, if self-adaptive software is to be implemented, such a formalism will be necessary: semantics is the design backbone of implementation of adaptive modules, as well as the development and runtime environments for them. The reason for the terminology: contract is between a programmer and a customer. Refinement of contracts [ However, none of the existing mathematical formalisms of computer science accommodates in a uniform way both * structural specifications * and the executable code * satisfying it in some formal sense. On the other hand, if self-adaptive software is to be implemented, such a formalism will be necessary: semantics is the design backbone of implementation of adaptive modules, as well as the development and runtime environments for them. Refinement of contracts j Ul \r- h B = [p,> s J Ul Refinement can be viewed as a *game* between the customer, providing the specification, and the programmer, providing the program. The former controls the inputs, the latter the outputs. The rules of the game are given by the *contract*. Example: adaptive sorting | S : Divide and Conquer For simplicity, we assume that decompose and compose are functions, although they usually are not. This is not inessential for the algorithm, but the lost generality has no consequences on the point we are trying to make. In principle, I can be taken to be the datatype of bags, O the datatype of ordered lists or sequences. + is then the union at I, and merge at O. The diagram commutes. The commυtativity of the square captures Doug's soundness axiom: this is how decompose and compose determine one another. Example: adaptive sorting | states the values of i and o transitions i r→d k (i) s(d k (i)) f→ o(i) The states are the various values that can be found in i and o. Initially, o is undefined. From an initial state, D&C makes two recursive calls, for sorting shorter strings. Their outputs will be merged (or perhaps just concatenated). STATE = CLASS OF MODELS (assignments of values to variables) Example: adaptive sorting | Refine D&C: • Mergesort: d = + => c - merge • Quicksort: c = @ => d - qs-partition Refining the D&C spec means further specifying what d and c will be. Picking one determines the other (via the commutativity of the square in the D&C spec). Refinement of specs induces refinement of the programs: each of the states now refines to a whole automaton, with many rewrites/transitions. We get executable Quicksort and Mergesort. The actions of the refined programs are * silent * (hidden) in the original. Adaptive interpreter observes that the incoming inputs tend to lead to the worst case performance for quicksort. It notices that 98% of time is spent on the d(ecompose) function. So it instantiates d to multiset union. This could be achieved by building the choice into a program: maintain statistics, and make choice. But there may be an Infinity* of choices. Adaptive interpreter! generalized API (open for refinement and adaptation) Adaptive interpreter: - extends interpreter by the deductively supported adaptation machine - extends a design environment by an interpreter - tests results/monitors correctness/satisfaction, safety, and various aspects of performance (recognizes deadlocks...) Accommodate the dynamics of ever changing interfaces, and the unpredictable interactions arising from the ever changing environments. The fact that the architectures never stop changing should not be taken as a nuisance, but recognized as the essence of the game of software, bun: into its foundation, and implemented as a design routine. CO c CD CO m CO I m σ> m c math models of structure and behaviour m ro the design backbone when software synthesis, maintenance, or adaptation needs to be automated: - Implementation Is driven by a formal model (q, φ) ^ →(q φ, q ', φ') Denotationally, Accommodate the dynamics of ever changing interfaces, and the unpredictable interactions arising from the ever changing environments. The fact that the architectures never stop changing should not be taken as a nuisance, but recognized as the essence of the game of software, built into its foundation, and implemented as a design routine. Adaptive interpreter can be viewed of: generalized API (open for refinement and adaptation) Adaptive interpreter: extends interpreter by the deductively supported adaptation machine extends a design environment by an interpreter tests results/monitors correctness/satisfaction, safety, and various aspects of performance (recognizes deadlocks...) Semantics of adaptive interpreter! extends • a design tool by a runtime environment • an interpreter by a theorem prover • tests results/monitors correctness, safety, various aspects of performance, recognizes deadlocks... Denotationally, Accommodate the dynamics of ever changing interfaces, and the unpredictable interactions arising from the ever changing environments. The fact that the architectures never stop changing should not be taken as a nuisance, but recognized as the essence of the game of software, built into its foundation, and implemented as a design routine. Adaptive interpreter can be viewed of: generalized API (open for refinement and adaptation) Adaptive interpreter: extends interpreter by the deductively supported adaptation machine extends a design environment by an interpreter tests results/monitors correctness/satisfaction, safety, and various aspects of performance (recognizes deadlocks...) Ongoing work I Implement category of contracts as a design environment - develop adaptive software - refine and maintain specifications together with their implementation: combine top-down and bottom-up - support evolution and assured assembly of complex systems of modules, tagged by their specifications Implement adaptive interpreter - self-testing - runtime self-debugging - self-adaptation What Is self-adaptive software? Realization • Model integration: m ∞ - explicit assumptions about the environment CO - requirements and constraints included in software m CO I m O) • Adaptability: - automatically supported reasoning about • environment, inputs and outputs • software performance, with respect to requirements • deviations from the assumptions • to which extent does the given software satisfy the given requirements in the given environment - runtime monitoring, testing and debugging - automatic adjustments » on-the-fly reconfiguration » synthesis to appear in CalculaLional System Dtsign. Proceedings of tbe International Summer School Marktoberdorf, Ed. M. Broy, NATO ASI Series, IOS Press, Amsterdam. 1999. Mechanizing the Development of Software Douglas R. Smith Kestrel Institute 3260 Hillvie Avenue Palo Alto, California 94304 20 December 1998 Abstract. This paper presents a mechanizable framework for software development by refinement. The framework is based on a category of specifications. The key idea is representing knowledge about programming concepts, such as algorithm design, datatype refinement, and expression simplification, by means of taxonomies of specifications and morphisms. Examples are drawn from working systems Specware, Designware. and Plan- ware. Contents 1. Overview 3 2. Basic Concepts 4 2.1. Specifications 4 2.2. Morphisms 5 2.3. The Category of Specs 6 2.4. Diagrams 8 2.4.1. The Structuring of Specifications 8 2.4.2. Refinement and Diagrams 9 2.5. Logic Morphisms and Code Generation 10 3. Software Development by Refinement 10 3.1. Constructing Specifications 10 3.2. Constructing Refinements 11 3.3. Development or Sorting Algorithms 11 3.3.1. Sorting: Algorithm Design 12 3.3.2. Sorting: Datatype Refinement 21 3.3.3. Sorting: Expression Optimization 23 3.3.4. Sorting: Summary 26 4. Domain- Specific Software Development 29 4.1. Constructing a Requirement Specification 29 4.2. Datatype Refinement and Problem Reformulation 32 5. Scaling up 33 5.1. Design by Classification: Taxonomies of Refinements 34 5.2. Tactics 37 6. Summary 38 References 39 A Containers, Bags, and Sequences 40 B Specification for Sorting 48 1. Overview A software system can be viewed as a composition of information from a variety of sources, including • the application domain, • the requirements on the systenrs behavior. • software design knowledge about system architectures, algorithms, data structures, code optimization techniques, and • the run-time hardware/software/physical environment in which the software will execut e. This paper presents a mechanizable framework for representing these various sources of information, and for composing them in the context of a refinement process. The framework is founded on a category of specifications. Morphisms are used to structure and parameterize specifications, and to refine them. Colimits are used to compose specifications. Diagrams are used to express the structure of large specifications, the refinement of specificat ions to code, and the applicat ion of design knowledge to a specificat ion. The framework features a collection of t echniques for construct ing refinement s based on formal representations of programming knowledge. Abstract algorithmic concepts, datatype refinements, program optimization rules, software architectures, abstract user interfaces, and so on, are represented as diagrams of specifications and morphisms. We arrange these diagrams into taxonomies, which allow incremental access to and construction of refinements for particular requirement specifications. For example, a user may specify a scheduling problem and select a theory of global search algorithms from an algorithm library. The global search theory is used to construct a refinement of the scheduling problem specification into a specification containing a global search algorithm for the particular scheduling problem. The framework is partially implemented in the research systems Specware, Design- ware, and Planware. Specware provides basic support for composing specifications and refinements, and generating code. Code generation in Specware is supported by" inter- logic morphisms that translate between the specification language/logic and the logic of a particular programming language (e.g. CommonLisp or C++). Specware is intended to be general-purpose and has found use in industrial settings. Designware extends Specware with taxonomies of software design theories and support for constructing refinements from them. Planware provides highly automated support for requirements acquisition and synthesis of high-performance scheduling algorithms. The remainder of this paper covers basic concepts and the key ideas of our approach to software development by refinement, in particular the concept of design by classification [9]. A simple example is carried through algorithm design, datatype refinement, and expression simplification. We also discuss the application of these techniques to domain- specific refinement in Planware [1]. This paper assumes a rudimentary knowledge of logic and category theory, although details about the categories of interest are introduced and motivated by examples. spec Container is sorts E. Container op empty • — r Container op singleton : E — > ■ Container op . j oin- . Container. Container — > Container constructors {empty, singleton, join} construct Container axiom V(x : Cor2Ϊcπrce7)(:r. jozn empty = a; Λ empty join x — x) op _Ϊ7?_ . E. Container — >■ Boolean definition of in is axiom ?π empty — false axiom a- 7π sιngleton(y) = (x = y) axiom τ in U join V = (a m U V 1 in V) end-definition end-spec Figure ! • Specification for Containers 2. Basic Concepts 2.1. Specifications A specification is the finite presentation of a theory The signature of a specification provides the vocabulary for describing objects operations, and properties in some domain of interest and the axioms constrain the meaning of the symbols. The theory of the domain is the closure of the axioms under the rules of inference. Example Here is a specification for paitial orders, using notation adapted from Specwaie It mtioduces a sort E and an infix binary predicate on E. called It. which is constiained by the usual axioms. Although Specware allows higher-order specifications, fiist-oidei formulations are sufficient in this paper spec Par tial-Order is sort E op e_ : E. E → Boolean axiom reβtxiviLy is x It x axiom t ansitivity is x I y Λ y It z =- > j It z axiom antisymmetry is x le y Λ y le x = • x = z end-spec Example Containers are constructed by a binary join operator and they represent finite collections of elements of some sort E. The specification shown in Figure 1 includes a definition by means of axioms. Operators are required to be total. The constructor clause assei ts that the operators {tmpl y. ingl lon . join} construct the sort Container. providing the basis for induction on Container The geneiic term expression will be used to refer to a term, formula, oi sentence A model of a specification is a structure of sets and total functions that satisfy the axioms. However, for software development purposes we have a less well-defined notion of semantics in mind: each specification denotes a set of possible implementations in some computational model. Currently we regard these as functional programs. A denotational semantics maps these into classical models. 2.2. Morphisms A specification morphism translates the language of one specification into the language of another specification, preserving the property of provability. so that any theorem in the source specification remains a theorem under translation. A specification morphism m : T — T" is given by a map from the sort and operator symbols of the domain spec T to the symbols of the codomain spec 7". To be a specific at ion morphism it is also required that every axiom of T translates to a theorem of 7". It then follows that a specification morphism translates theorems of the domain specification to theorems of the codomain. Example: A specification morphism from Partial-Order to Integer is: morphism Partial -Order-to- Integer is {E (-* ■ Integer, le t-τ <} Translation of an expression by a morphism is by straightforward application of the symbol map. so. for example, the Partial-Order axiom x le x translates to x < x. The three axioms of a partial order remain provable in Integer theory after translation. Example: A parameterized specification can be treated as a morphism. The specification Bπq in Appendix A can be parameterized on a spec Triv with a single sort: sort E end-spec via the morphism morphism Bag-Pararrielerization is {E >→ E} The semant ics of a specification morphism is given by a contra variant funcl oi Mod t hat maps ea specification to a category of models and each specifi ation rriorphism m : T → I" to a reduct functor _| m : Mod{T) → Mod{T). Moiphisms come in a variety of flavois: heie we only use two. An extension or import is an inclusion between specs. Example. We can build up the theory of partial orders by importing the t heory of preorders. The import morphism is {E ι- E. le H- le}. spec PrtOrder sort E op Je_ : E. E - Boolean 01 12409 axiom lefltxiυdy is x le x axiom transitivity is x le y Λ y le z == x le z end-spec spec Partial-Order import PreOrder axiom antisymmetry is x le y Λ y le x = - x = z end-spec A definitional extension, written A —d→ B . is an import morphism in which any new symbol in B also has an axiom that defines it. Definitions have implicit axioms foi existence and uniqueness. Semantically. a definitional extension has the pioperty that each model of the domain has a unique expansion to a model of the codomain Example: Container can be formulated as a definitional extension of Pr t-Conlaintr . spec Pr e-Container is soi ts E. Container op empty : — Container op singleton : E — Container op -jorn_ : Container. Container -4 Container c Tuslni t ois {ernpl y. single! n. join} construct Container axiom V(z : Conlaιner)(x join empty = x A empty join x = ) end-spec spec Container is imports Pre-Container definition of in is axiom x in empty = false axiom x in smgleton(y) = (x = y) axiom x m U join V = (T n V V x in V) end-definit ion end-spec 2.3. The Category of Specs Specification morphisms compose in a straightforward way as the composition of finite maps. It is easily checked that specifications and specification morphisms form a category SPEC. Coliinitb exist in SPEC and die easily computed. Suppose that we want to compute the colimit of B-^~ A *-C . Fiist. form the disjoint union of all sort and operator symbols of A. B. and C. then define an equivalence relation on those symbols: .s « / iff (L(S) = I V /(/) = 6 V (.s) = 1 V j(l) = s). The signature of the colimit (also known as pushout in this case) is the collection of equivalence classes wrt «. The cocone morphisms take each symbol into its equivalence class. The axioms of the colimit are obtained by translating and collecting each axiom of A. B. and C. Example: Suppose that we want to build up the theory of partial orders by composing simpler theories. The pushout of Antisymmetry 4— Bm Rel -4 PreOrder is isomorphic to the specification for Partial-Order in Section 2.1 . In detail: the morphisms are { E H4 E. le H le.} from BinRel to both PreOrder and Antisymmetry. The equivalence classes are then {{ E. E. E}. {le. le. le}}. so the colimit spec has one sort (which we rename E). and one operator (which we rename le). Furthermore, the axioms of BinRel Antisymmetry, and Pre Order are each translated to become the axioms of the colimit. Thus we have Partial-Or r. In the category of specifications, colimit acts as a kind of union operator: the pushout collects the symbols of Antisymmetry and Pre Order where the morphisms from BinRel indicate which symbols are shared. The colimit has the universal property that it computes the simplest such specification. Example: The pushout operation is also used to instantiate the parameter in a parameterized specification. The binding of argument to parameter is represented by a morphism. To form a specification for bags of integers, we compute the pushout of Bag - Triv -4 Integer, where Bag <r- Triv is {E *- E}. and Triv -4 Integer is {E ι-4 Integer}. Example: A specification for sequences can be built up from Container, also via pushouts. We can regard Container as parameterized on a binary operator sort E op i>or>_ : E. E -4 E end-spec morphism Container-Parameterization : BinOp -4 Container is {E ι-4 E. bop H4 join} and we can define a refinement arrow that extends a binary operator to a semigroup: spec Associativity is import BinOp axiom Associativity is ({x join y) oin z) = (x join (y join z)) end-spec The pushout of Associativity - BinOp -4 Container, produces a collection specification with an associative join operator , which is Proto-Seq. the core of sequence theoiy in Appendix A. 2.4. Diagrams Roughly, a diagram is a graph morphism to a category, usually the category of specifications in this paper. For example, the pushout described above started with a diagram comprised of two arrows: BinRel * ■ PreOrder Antisymmetry and computing the pushout of that diagram produces another diagram: BinRel ■ PreOrdt Antisymmetry * ■ Partial-Order A diagram commutes if the composition of arrows along two paths with the same start and finish node yields equal arrows. 2.4.1. The Structuring of Specifications Colimits can be used to construct a large specification from a diagram of specs and morphisms. The morphisms express various relationships between specifications, including sharing of structure, inclusion of structure, and parametric structure. Several examples will appeal later. Example: The finest-grain way to compose Partial-Order is via the colimit of BinRel Reflexwity Transitivity Antisymmetry Example: Appendix B gives a structured specification for sorting bags over a linear order. 2.4.2. Refinement and Diagrams As described above, specification morphisms can be used to help structure a specification, but they can also be used to refine a specification. When a morphism is used as a refinement, the intended effect is to reduce the number of possible implementations when passing from the domain spec to the codomain. In this sense, a refinement can be viewed as embodying a particular design decision or property that corresponds to the subset of possible implementations of the domain spec which are also possible implementations of the codomain. Often in software refinement we want to preserve and extend the structure of a structured specification (versus flattening it out via colimit). When a specification is structured as a diagram, then the corresponding notion of structured refinement is a diagram morphism. A diagram morphism M from diagram D to diagram E consists of a set of specification morphisms. one from each node/spec in D to a node in E such that certain squares commute (a functor underlies each diagram and a natural transformation underlies each diagram morphism). We use the notation D = • E for diagram moiphisms. Example: A datatype refinement that lefines bags to sequences can be piesented as the diagram morphism BtoS : BAG =» BAG-AS-SEQ: where the domain and codomain of BtoS are shown in boxes, and the (one) square commutes. Here Bag-as-Seq is a definitional extension of Seq that provides an image for Bag theory. Specs for Bag. Seq and Bag-as-Stq and details of the refinement can be found in Appendix A. The interesting content is in spec morphism BIOS ' phisill BtoSβag Bag -4 Bag- as-Seq is {Baq H Bag-as-Seq. empty-bag H4 bag-empty. tmply-bagl ι-4 bag-tmplyl . nonemptyl 1-4 bag-nonemptyl . singltlon-bag H4 bag-singltlon. bingltlon-bagl ι-4 bag-bingletυnl. singl etonl . . Diagram morphisms compose in a straightforward way based on spec morphism composition. It is easily checked that diagrams and diagram morphisms form a category. Colimits in this category can be computed using colimits in SPEC. In the sequel we will generally use the term refinement to mean a diagram moiphism. 2.5. Logic Morphisms and Code Generation Inter-logic morphisms [2] are used to translate specifications from the specification logic to the logic of a programming language. See [10] for more details. They are also useful for translating between the specification logic and the logic supported by various theorem-provers and analysis tools. They are also useful for translating between the theory libraries of various systems. 3. Software Development by Refinement The development of correct-by-construction code via So a formal refinement process is shown to the left. The refinement process starts with a specification So of the requirements on a desired software artifact. Each Sl Si, i = 0.1. .... π represents a structured specification (diagram) and the arrows } are refinements (represented as diagram morphisms). The refinement from Si to S,- +1 embodies a design decision which cuts down the number of possible implementations. Finally an inter-logic morphism translates a low-level specification S n to code in a programming language. Semantically the effect is to narrow down the set of possible implementations of b n S„ to just one. so specification refinement can be viewed as a constructive process for proving the existence of Code an implementation of specification SQ (and proving its consistency). Clearly, two key issues in supporting software development by refinement are: (1) how to construct specifications, and (2) how to construct refinement . Most of the sequel treats mechanizable techniques for constructing refinements. 3.1. Constructing Specifications A specification-based development environment supplies tools for creating new specifications and morphisms, for structuring specs into diagrams, and for composing specifications via importation, parameterization, and colimit. In addition, a software development environment needs to support a large library of reusable specifications, typically including specs for (1 ) common datatypes, such as integer, sequences, finite sets, etc. and (2) common mathematical structures, such as partial orders, monoids, vector spaces, etc. In addition to these generic operations and libraries, the system may support specialized construction tools and libraries of domain-specific theories, such as resource theories, or generic theories about domains such as satellite control or transportation. 3.2. Constructing Refinements A refinement -based development environment supplies tools for creating new refinements. One of our innovations is showing how a library of abstract /reusable/generic- refinements can be applied to produce refinements for a given specification. In this paper we focus mainly on refinements that embody design knowledge about (1 ) algorithm design. (2) datatype refinement, and (3) expression optimization. We believe that other types of design knowledge can be similarly expressed and exploited, including interface design, software architectures, domain-specific requirements capture (see Section 4. on Planware). and others. In addition to these generic operations and libraries, the system may support specialized construction tools and libraries of domain-specific refinements. The key concept of this work is the following: abstract design knowledge about datatype refinement, algorithm design, software architectures, program optimization rules, visualization displays, and so on. can be expressed as refinements (i.e. diagram morphisms). The domain of one such refinement represents the abstract structure that is iequired in a user ' s specification in order to apply the embodied design knowledge. The lefinement itself embodies a design constraint - the effect is a reduction in the set of possible implementations. The codomain of the refinement contains new stiuchues and definitions that are composed with the user ' s requirement specification. The figure to the left shows the application of a librarv refine- A > S ment A = - B to a given (structured) specificat ion SQ. First the library refinement is selected. The applicability of the refinement to SQ is shown by constructing a classification arrow " '■ ' "^ϊ from A to So which classifies So as having .4-structure by making explicit how S 0 has at least the structure of A. Finally the refinement is applied by computing the pushout in the category of diagrams. The creative work lies in const ructing the classification arrow [7. 9]. 3.3. Development of Sorting Algorithms Our goal in this section is to show a simple example that refines a requirement spec to code bv applying ( I ) an algorithm design refinement. (2) a datatype refinement, and (3) an expression optimization refinement. We step through the refinement of a specification for sorting a bag over an arbitrary linear order. A full specification for the import to Sorting is given in Appendix B. spec Sorting is import Bag-Stq-oυer-LinOrd op sorted'? : Bag. Seq — )■ Boolean def sorltd?{x. z) = (υrdered1{z) A x = stq-to-bag(z)) op sorting : Bag -4 Seq axiom sorted? (x. sortιng(x)) end-spec The following diagram serves as a roadmap oϊ the design steps covered in the next three subsections. First a divide-and-conquer refinement is applied, then a datatype 2409 refinement (bags implemented as sequences), then an expression simplification refinement is applied, and so on. The squares represent the application of library refinements and the main line of refinement descends directly from Sorting. 3.3.1. Sorting: Algorithm Design Before algorithm design can be applying we must have a well-defined problem to solve. Problem- Theory in Figure 2 expresses the abstract structure of a problem: given input datum a: : D. find a feasible solution z : R satisfying the problem requirement constraint: O(x. z). We call the input datum x a problem instance: note that Problem-Theory is intended to specify a function. Suppose that the user first decides to apply a generic refinement for divide-and- conquer algorithms. The principle of divide-and-conquer is to solve small problem instances tπψty-seq t- ■ eτnpi y-seq by some direct means, and to solve fmpty- ieq' empty-seq larger problem instances by decomposing them, solving the pieces, and com¬ (> ■ - ■ 0 posing the resulting solutions. The figure to the left shows a mergesort as generalized kind of divide-and-conquer sιnqleton-seq{e) - ■ sm leton-seq(e) with three cases. The bottommost •ungleinn- iFq~ singleton- seq square shows the familiar divide-and- xά e conquer case: the input x 0 is decomposed into two subproblem instances. sorttng X] and x, 2 - which are solved to produce ΪO H • Zϋ z and 2 2 . which are in turn composed concat ' mergf to form solution z 0 . The decomposisorting X sorting tion operator here is loosely specified (XΛ . XΪ) (21 - 22) as the inverse of a constructor, concat. The first t wo cases also have the decompose-so] ve-compose pattern where the decomposition operator is the inverse of a constructor. The datum () is the 0-tuple and it is treated as the output of the inverse of a constant constructor: () is the sole element of 01 12409 the empty product sort, denoted Unit. The first two cases differ from the third in that the input decomposes into a subproblem that is easily solved, by the identity function on the appropriate sort. The domain spec of a simple divide-and-conquer refinement is given in Figure 2 that abstracts from the mergesort example. It supposes three cases (indexed by 0. 1 and 2). each based on the signature of a constructor, similar to a homomorphism between algebras with three constructors The abstraction lies in treating each operator in the example as a problem theory Subscripted O ' s denote problem requirement constraints- e.g. Oo f o po i e -i specifies the decomposition operator in case i. and so on. The discriminator p t represents the condition under which Or> fCom pose- t an decompose an input The Somιdness-2 axiom ielates 0. Or) fcomp0 e2 - and Oco m po ϊ - asserts that if (1 ) nonpπm- itive problem instance Q t n decompose into two subproblem instances x^ and .c 2 . (2) subproblem instances i and J 2 have feasible solutions zi and 22 lespective . (3) zi and 2 2 can compose to form z 0 . then z 0 is a feasible solution to input xo- Similar comments hold foi the othei Soundness axioms. The operator > is a well-founded order on D to assure termination (axioms are omitted for simplicity). A general scheme for problem reduction theories (including divide-and-conquer) is given in [6] The idea is to have a different first-order divide-and-conquer theory for each possible abstract constructor signature. The codomain spec of the ide-an -conquer refinement (Figure 3) contains a schemat ic definit ion for the top-level divide-and-conquer functions and schematic requirement specifications for subalgorithms Co C . and C 2 Given any morphism from Divide-and- Conηver-0- 1-2 and functions that satisfy the requirement specifications for the subalgorithms. the corresponding instance of the divide-and-conquer function satisfies its requirement specification [4]. There is one complication which needs explaining. Specware requires that all functions be total and deterministic: each operation in a spec has implicit axioms for existence and uniqueness of solutions. This simplifies the logic of Specware. but complicates the treatment of nondeterministic functions which arise naturally in refinement, due to the levels of abstraction Consider, for example, an operation that decomposes a bag b into an element a and the lema dei of the bdg b' i.e. dii inverse of ιnstrl-bag(a. b') If the bags die ultimately implemented as lists, then it is naturdl to use first and rest to per- foim the decomposition. Howevei consider two equal bags {1.1.2}= {2.1.1 }. If {1.1.2} is lepieseπted bj [1 .1 .2] then it will decompose into ] and {1.2}. If {2.1.1 } is represented by [2 1.1] then it will decompose into 2 and {1.1 }. Hence the decomposition is not functional on bags, even though it is functional on lists. There are vdrious approaches to this problem We observe however that any extra work requπed to make a naturally nondeterministic operator into a deterministic one is often unnecessary from the programming point of view, when the nondetermiinsm is of the don l-caie vaiiety - any solution will do. so any extia woik to foice one unique solution is a constraint posed by the programming system rather than by the software requirements. One solution to this problem is to treat operations that may need to be nondeterministic via existential quantification In place of decomposition operators in Figure 3we have existentially quantified requiiemen! constraints . The development of a divide-and-conquer algorithm for sorting begins with the con- 2409 spec Problem-Theory is sorts D.R op O : D.R -4 Boolean end-spec spec Dvιde-and-Conquer-OA-2 is import DRO sort E op F : -4 R op _ > _ : D. D — Boolean axioms > is a well-founded order ... op po ' D — > Boolean op Orjecom oseo ■ D. Unit -4 Boolean op Ocomposeo : R- c/ τ -4 Boolean axiom Sυundness-0 is 0(-C.2) a op pi : D -4 Boolean op OD < - compoβei : Zλ E -4- Boolean op 0 Comp „ se i : R.E → Boolean dxiom Sourιdness-1 is θDκo m pov l (z-e) Λ Oc 0 mposf l (z.e) = - O(x.z) axiom discriminator-of- Decompose] is p,(χ) = 3(e) O D mpose! i-l.e) op p 2 . D -^ Boolean op Oj rompo«2 : D.D.D -4 Boolean op 0 com ost 2 '■ R- R. R -4 Boolean ax Λ 0(-Tι.2ι) Λ 0(x 2 - Z 2 ) Λ coτn n «2(zo- l- Z 2 ) axiom discriminator -of- Decorrιpost2 is /> 2 (x) =» 3(xι.O , 2 ) OD from p 0 , f 2(x.Xl.l2) Λ X > ! Λ > X2 axiom V(τ : D) Po(τ) ror p \ (r) ror p ∑ (τ) end-spec Figure 2: A Simple Divide-and-Conquer Algorithm Theory spec Divide-and-Conquer-O-l-2-scheme is import Divide-and-C onquer-0-1-2 op Co :-4 R axiom Oc om pos eθ {Cθ-. {)) op d : E → R axiom Oc 0mpose i {Cι{ ), e) definition of F is axiom axiom axiom end-definition theorem 0(x. E(x)) end-spec Figure 3: Parameterized Divide-and-Conquer Algorithm struction of a morphism from Problem-Theory to Sorting theory: D t-4 Bag R 1-4 Seq 0 H4 sorted Since the morphism from Probl m -Theory to Divide-and-Conquer is an inclusion, we can use straightforward propagation to obtain translations for the components of Problem- Theory in Diυιdt-and-Conquer~. D H Bag R t-4 Seq 0 »-4 sorted? F ι-4 1 E ι-4 1 > - 1 P 1-4 1 C Decomposed ι-4 ? ^C m o ed ι-4 ? To complete the classification arrow we attempt to translate the remaining operat rs into expressions of Sorting. Alternative translations give rise to different sorting algorithms. There are several ways to proceed. One approach is based on the choice of a set of standard decomposition operators from a library. The tactic then uses unskolemization on the soundness axioms to derive specifications for the composition operators. This approach allows the derivation of insertion sort, mergesort. and various parallel sorting algorithms [4.8]. A dual approach is to choose a set of standard composition operators from a library and use the soundness axioms to derive the decomposition operators (leading to selections sort, heapsort. and quicksort). We present the first approach in detail, then sketch the second. Suppose that we choose the constructor set {empty-bag. singleton-bag. bag-union} as the basis foi the decomposition relation on the input domain Bag. This gives us the partial signature morphism D H4 Bag R 1-4 Seq 0 H4 sorted? F 1-4 sorting E 1-4 E > 1-4 bag-wfgl A> H4 empty-bag? U Decomposed 1-4 X(x) x = empty-bag ^Compos d 1-4 ? PΛ H-4 sirtgl lon-bag? J Decompose] 1-4 λ(x.e) x = sιngleton-bag(t) Gom sel t-4 1 PΪ v→ nonsinglelon-bag? ^ Decompose2 ι-4 λ(x Q ,x-i.x 2 ) Xo = bag-unιon(x^. x 2 ) v.'Compos<;2 ι-4 1 The soundness axiom V(r 0 . T J . Tj : D) V(z 0 . zι.z 2 : R) (0D«-rmp o «-2(z0- Il-Za) Λ 0(xl.Z\) A 0{x 2 .2 2 ) Λ Ocompt i∑O- ∑l ■ Zz) = 0(r Q .Z 0 )) cannot be tianslated into Sorting because has no translation yet. However. a technique tailed unskolemization allows us to use inference tools to deduce a suitable translation for 0c O m ose2 [7]. Unskolemizing operator symbol Oc <>m p <w2 replaces the occuπence of by a fresh existeπtially quantified variable in the scope of the quantifier 5; for ; 0 - Z \ .Z 2 : V(z u . z\.∑ι : R) 3(y : Boolean)V(x 0 .x 1 .x '• D) {θ co pos2{Xθ-Xl-X2) Λ 0(Xι- 2χ) Λ 0(x 2 .Z 2 ) A y =£ O(x 0 -Z 0 )). This formula has the same satisfiability properties as the original and it can be translated to Sorting via the partial morphism yielding: V(z -Z].z 2 : Seq) 3(y : Boolean)y(x 0 .xι.x 2 : Bag) (x 0 = bag-unιυn(x .x 2 ) A sorted?(x- \ .z Λ ) A sorttd?(x 2 -z ) A y = sorted?(xo.z Q )) A straightforward proof of this formula in Sorting finds a witness for y which results in a translation for Oc 0 mpose2'- sorted?(xQ. z 0 ) = by def of sorted? ordered(zu) A o = seq-to-bag(zo) = by assumption o = bag-υnion(xι, x 2 ) ordered(zo) A bag-υnion(x , X 2 ) = seq-io-bag(zo) = by assumption x; = seq-to-bag(z,). 7 = 1.2 ordered(z ) A bag-union(seq-to-bag{zι), seq-to-bag(z 2 )) — seq-to-bag(z ϋ ). which can then unify with the assumption y, which must be a term over variables z 0 . z, . z 2 . More generally, we pick up as part of the witness any assumptions that are expressed over the variables z . Zι, z 2 , in this case yielding { λ (z 0 , z , z 2 ) ordered(z ) A ordered(z 2 ) ==> ordered(z Q ) A bag-union(seq-to-bag(zι), seq-to-bag(z 2 )) = seq-to-bag(zo). This is. of course, a specification for a merge operation. If we take this as the translation of Oc om osf - then we know that the Soundness-2 axiom translates to a theorem in Sorting by construction. The remaining steps in constructing this classification arrow are similar. We un- skolemize Oco mp Λsfi in Soundness- 1 and then derive a witness: sorted? (x. z) = by def of sorted? ordered(z) A — seq-to-bag(z) = using assumption x = singleton-bag(e) ordered(z) A singleton-bag(e) = seq-to-bag(∑) = Bag-and-Seq-Conv axiom ordered(z ) A seq-to-bag(singi 'eton-seq(e)) — seq-to-bag(z) < = Leibnitz: equality is a congruence order ed[z) A singleton-seq(e) — z = simplifying using ordered(singlet on-seq(e)) = true sιngleton-seg(e) = z. yielding the translation Oc mp sei i→ \{z. e) si rιgItlon-stq{e) = z. We can unskolemize Oco pos e in Soundness-0 and then derive a witness: sorted?(x. z) = def of sorted? ordered(z) A x = seq-io-bag(z) = assumption x = empty-bag 01 12409 ordered(z) A empty-bag = seq-to-bag(z) = Bag-and-Seq-Conv axiom ordered(z) A seq-to-bag(empiy-seq) = seq-io-bag(z) • = Leibnitz: equality is a congruence ordered(z) A empty-seq = 2 = simplifying using Sorting axiom order ed(empty-seq) = true ernply-seq — z. yielding the translation Ocomposeo >→ z ) empty-seq = z. The classification arrow is now complete: D 1-4 Bag R H4 Seq 0 H4 sorted? F 1-4 sorting E 1-4 E > 1-4 bag-wfgt 7>o 1-4 empty-bag? _D <"= mm p 0 *t e 0 1-4 λ(x) x = empty-bag ^C m osed 1- λ(z) z = empty-seq H4 singleton-bag? ^Decomposel H4 λ(x. e) x — singleton-bag(e) ( JC pos l H4 λ(z. e) z = smgleton-seq(e) V2 H4 n on si let on -bag? U De ompoif2 H4 λ(x 0 - 'ι - X2) . x 2 ) UCom o$e -4 X (z . z 1 . z 2 ) ordertd(zι) A υrdered(z 2 ) = ordertd(zo) A seq-to-bag(zo) — bag-umon(seq-to-bag(zι). seq-to-bag(z-ι)) Note that we have a classification arrow that translates a symbol to an expression (rather than a symbol). This commonly occurring situation is treated by extending the codomain diagram with definitional extensions as necessary. In the sequel we will assume this treatment whenver the need for a symbol-to-expression translation arises. We can compute the pushout Dιvιde-and-Conqver-0-1-2 ■ Sorting D tυide-and-C onquer-Q-\-2-scheme ■ ■- > Sυrlιng-Alg to obtain a refinement of Sorting that contains a definition for a mergesort algorithm, shown in Figure 4. Example - sketch of a derivation for Quicksort: spec Sortιrιg-Alg is import Bag-Stq-oυtr-LinOrd op sorted? : Bag. Seq -4- Boolean definition sorled?(x,z) = (ordered?(z) A x = seq-to-bag(z)) op sorting : Bag -4 Seq theorem sorted?(x. sortιng(x)) definition of sorting is axiom empty-bag?(x) == ■ (x = empty-bag A sorting(x) = Co) axiom singleton-bag? (x) = 3(e)(x = smgleton-bag(e) A sortmg(x) = C-(e)) axiom n ι gleton-bag?(x) = - 3(x'ι. x 2 )(x = bag-uniυn(xι.x 2 ) A sortιng(x) = C 2 (sorlιng(xι), orli ng(x 2 ))) end-definition op o : -4 Seq axiom Co = empty-seq op C'ι : £ -4 Seq axiom Cj(f) = sιngleton(e) op C 2 : Seg. Seq -4 Seq axiom ordered?(zι) A ordered?(z 2 ) == (ordered?(C 2 [z \ .z 2 )) A seq-lυ-bag(C 2 (zι. z 2 )) = bag-union(seq-lo-bag(zι).seq-lo-bag(z 2 ))) end-spec Figure 4: Divide-and-Conquei Sorting Algorithm T U 01/12409 The derivation of a variant of quicksort is dual to the derivation for mergesort: rathei than select a simple set of constructors as the basis for the decomposition operations, we select a simple set of constructors as a basis for the composition operators. In particular if we return to the stage of filling in the partial morphism 0 H4 sorted? F H4 sorting E 1-4 E > 1-4 bag-wfgt and we choose the set of sequence constructors {empty-seq. singl eton-seq . concat} as a basis for composition, we get D H- Bag R 1-4 Seq 0 1-4 sor ted? F ι-4 sorting Po 1-4 ? 'DecomposeO ι→ ? UCo poifO 1-4 X(z) z = empty-seq Pi 1-4 1 C-' Decompose] 1-4 1 JCom osti ι-4 X(z. e) z = si gleton-seq(e) ?2 ι-4 ? ~'Decompose2 ι-4 ? ( JCompose2 1-4 X(ZQ. ∑ι- z 2 ) z 0 = concat(zι, z 2 ) Now we use the Soundness axioms to deduce translations for the decomposition requirement constraints. The most interesting case is unskolemizing operator symbol Ooecompos in axiom Soundness-2: V(x 0 This formula can be translated via the partial morphism yielding: V(x 0 ..τ 1 . x 2 : Bag) 3(y : Boolean) V(z Q . z- , z 2 : Seq) [y A sυrled?(x \ . ∑- ) A sorted? (x 2 . z 2 ) A z a — concat(z . z 2 ) = sorled?(xo. zo)) T U 01/12409 A straightforward proof of this formula in Sorting: sorted? (XQ. ZQ) = by def of sorted? ordered(zo) Λ o = seq-to-b = by assumption zo = z 2 ) ordered(concat(zχ. z 2 )) Λ x 0 = seq-tυ-bag(corιcai(zι. z 2 )) = distributing using axioms from Sorting and Bag-and-Seq-Conv ordered(z ) A all-le(z x . z 2 ) A ordered(z 2 ) A o = bag-union(seq-to-bag(zι). seq-to-bag(z 2 )) = simplifying using seq-to-bag(z,) = .τ t . ? = 1.2 all-le(xι, x 2 ) λ xo = bag-unιon(xι . x 2 ) iesulls in a translation for ODec m ie2-' 0 Dec „ mPosc2 ι→ λ(x 0 . xι : x 2 ) α /- e(xι . x 2 ) Λ o = bag-unιυn(xι , x 2 ) That is. each element of bag i is less-than-or-equal to each element of bag x 2 and x 0 decomposes into two exhaustive subbags of itself. This is. of course, a specification for a partition operation in a quicksort. If we take this as the translation of O eco po λ- then we know that the soundness axiom translates to a theorem in Sori ng-lheory by constiuction. The other two cases are similar to the base cases in mergesort. The divide-and-conquer theory allows a pleasing derivational symmetry between algorithms with simple decomposition operations and complex composition operations and dually, algorithms with simple composition operations and complex decomposition oper- ai ioiis. 3.3.2. Sorting: Datatype Refinement Next, suppose that the usei decides to bring the mergesort algorithm closer to implementation by Br G == = ' ■ ■ Sorting refining bags into sequences. This datatype refinement was discussed in Section 2.4.2. and il is j* e l m presented in detail in Appendix A. The classifi- ^ cation arrow fiom BAG to Sorting is essentially an inclusion, comprised of the identity morphisms from BAG specs to corresponding specs in the diagram of the Sorting spec. The classification arrow together with tbe pushout is shown in Figure 5. The two cocone airows in the figure are shown with dashes to help distinguish them. Also, to save space the figure abbreviates Bag-as-Seq to BaS. The pushout gives us a refinement of Sorting, shown in Figure 6. The effect on the mergesort definition arises mainly from the translations of the BtoSβ ag morphism in Sec- t ion 2.4.2.. Note that not all bag operators are directly in the image of the classification arrow. In particular, the operators all-le and seq-to-bag are extensions to bag theory added during the construction of sorting theory. The fact that they are defined however Figure 5: Refining Bags to Seqs in Sorting means that the pushout will translate the definitions too. so they remain defined after translation. If some of the auxiliary sorts or operators on the datatype are not in the image of the classification arrow, then the pushout will simply translate whatever constraints there are on such sorts and operators. If the constraints are definitions, then the translation will be definitional, otherwise the symbols will be constrained but not necessarih' uniquely, and further refinement will be required later. This datatype refinement mainly serves to bring the spec closer to the programming language level. Other datatype refinements have a more dramatic effect by refining to a complicated data structure that efficiently implements the more abstract type. The same machinery applies. Leverage on the programming task will be most apparent when a relatively small domain specification refines to a relatively large codomain spec - the extra bit s of information in the codomain are added to the user's spec essent ially for free once the classification arrow has been constructed. Some examples would include Boolean expressions refining to BDDs, and sets over a fixed finite universe refining to bit vectors or hash tables, and finite relations refining to B-trees. P T/US01/12409 spec Sorting-Alg 2 is import Bag-as- Seq-Seq-over-LinOrd op sorted? : Bag-as-Seq, Seq -4 Boolean def sorted? (x, z) = (ordered?(∑) A x = seq-to-bag(z)) op sorting : Bag-as-Seq -4 Seq theorem sorted? (x, sorting(x)) definition of sorting is axiom ba g-empty? (x) == (x = bag-empty A soriing(x) = Co) axiom bag-sin gleton?(x) =» 3(e)(x — bag-sin gleton(e) A sorting(x) = Cχ(e)) axiom bag-vonsingleton?(x) = 3(xι, 2 )(x = bag-union (xχ, x 2 ) A sorling(xo) = C 2 (sσrting(xι), sorting(x 2 ))) end-definition op o :— Seq axiom Co = empty-seq op C'ι : E -4 Seq axiom Cι (e) = single on(e) op C 2 : Seq. Seq -4 Seq axiom ordered.? (z x ) A ordered? (z 2 ) =» (order d?(C 2 (zι, z 2 )) A seq-to-bag(C (z-i , z 2 )) = bag-union(seq-lo-bag(z- ), seq-to-bag(z 2 ))) end-spec Figure 6: Sorting Algorithm after Datatype Refinement 3.3.3. Sorting: Expression Optimization After performing algorithm design and datatype refinement, there are often opportunities for optimizing various subexpressions. Transformation rules and optimization tactics, such as equational rewriting, context-dependent simplification [5]. finite differencing [3], partial evaluation, and other more specialized techniques can result in dramatic improvements in the complexity of the final implementation. We show how at least some optimization techniques can be captured as refinements, and can be applied in the same manner as refinements that express algorithm design and datatype refinement knowledge. The general approach to representing expression optimization techniques is a two stage process: first, an equation is deduced with the expression-to-be-simplified on the left-hand-side, and second, the spec S is reformulated to exploit the new theorem. The applicable metatheorem here is: if spec Si has theorem el = e2 and S2 is the same spec as Si except that some occurrences of el are replaced by e2. then SI and S2 are isomorphic specifications. The metatheorem justifies a substitution operator on specs that creates a spec and the isomorphism between them. Equational rewriting is a basic optimization technique that uses equational infeience in the user ' s spec to deduce an equal term. However, to determine which side is simpler requires a metric. The approach used in KIDS [5] and in Specware/Designware is to assign a weight to each operator, measuring the complexity of an expression b\ summing the weights of each operator occurrence. Heie is a refinement to express equational simplification: spec Expression is sorts D. Q op expr : D -4- Q end-spec spec New- Expression is import Expression op new-expr : D -4 Q axiom expr(x) = new-expr(x) end-spec spec Equational-Simphficat on is import N ew-Expre sion theorem expr(x) = new-expr(x) end-spec Equation al— Simplification (ES) is a degenerate case of our approach to representing expression optimization techniques. The presence of the theorem in ES is required in order that it be included in the pushout spec when the refinement is applied. Context-dependent simplification is a generalization of equational simplification. Suppose that we wish to simplify expression expr in some context C within spec S. To simplify expr. we want to use not only the equations and other theorems of the spec S. but also whatever contextual properties hold when expr is evaluated. For example, when simplifying the else-branch of a conditional, we can assume the negation of the test of the conditional. spec Expression-and-C'ontext is import Expression op C : D -4 Boolean op new-expr : D -4 Q axiom C(x) == expr(x) = new-expr(x) end-spec spec- Context-Dependent-Simplification is import Expression-and-C onlexl theorem C(x) = expr(x) = new-expr(x) end-spec P T/US01/12409 Finite Differencing [3] is a less basic optimization and it has nondegenerate structuie in the codomain of its represention as a refinement. The idea in finite differencing is to replace an expensive expression in a loop by incremental computation, making use of some extra variables. The following refinement expresses the correspondence between these concepts and the symbols in the refinement: e is the expensive expression, which may occur in the definition of operator /: h is the body of the definition of /. g is a new operatoi which is an abstraction of with an extra parameter c that is maintained to satisfy the invariant c = e(a ). The theorem asserts equality between / and g when g is invoked with an argument that satisfies the invariant. After developing the classification arrow and pushing out. the substitution (meta)operator replaces occurrences of / with g. To realize the implicit incremental benefits of this change, context-dependent simplification must be applied to exploit the local invariant c = e(x). spec Eτpressi on-and- Function is import Expression sort R op / : D 4 Λ op h : D -4 R def /(.τ) = /ι(.τ) end-spec spec Abstracted-Op is import Expression-and-F unction sort DQ = (D. Q \ X(x. c) c = e(x)) op g : DQ -4 R def g(x. c) = h(x) theorem f(x) — g(x. e(x)) end-spec W r e show how the equational simplification refinement works by simplifying the clauses of the definition of sorting in Sortmq-Alq . The simplification often serves a larger goal. Following algorithm design there are usually opportunities to simplify expressions due to the juxtaposition of subexpressions from different sources that have been assembled b\ means of a definitional scheme. More pertinent here, when datatype refinement is peifoimed and the codomain has a constructed sort, then simplification can reexpress definitions in terms of the base sorts of the construction. The definitional axioms of sorting in Sortιng-Alg 2 are expressed in terms of a quotient sort construction sort-axiom Bag-as-Seq = Seq/perm? Simplificat ion will have the effect of reexpressing the axioms in terms of sequences, using coeicion functions. We show the simplification of the singleton case since it most clearly exemplifies the essential reasoning pattern. The classification arrow can be partially filled in once a user has selected an expression to simplify: D H4 Unit Q 1-4 Boolean expr ι-4 V(x : Bag-as-Seq)(bag-singleton?(x) = 3(e)(x = bag-singleton(e) A sorting(x) — C (e))) new-expr H4 1 To obtain a translation for new-expr. we unskolemize it in the New-Expression axiom, then translate the result and find a witness in Sorting theory as shown in Figure 7. The proof draws on axioms from Bag-as-Seg and Seq and the quantifier change theorems: (V(x : Dj ≡) A(x)) = (V(y : D) A(q(y))) where q = quotient(≡) (V(x : D\p?) A(r(x))) = (V(y : D)(p?(y) => A(y))) where r = rtlax(p?) The completed the classification arrow is: D ι-4 Unit Q H4 Bool an expr 1-4 V(x : Bag-as-Stq)(bag-singleion?(x) =Φ- 3(e)(x = bag-singleton(e) A sorting(x) = C \ (e))) C 1-4 X(x) singleton-seq?(x) new-expr ι-4 V(y : Seq) singlel.on-seq?(y) = > sorl.ing(q(y)) = y The effect of computing the pushout of the refinement and the classification arrow is to add the theorem asserting equality of these two expressions in context. We then apply a substitution (meta)operator that creates an isomorphic spec. After simplifying all three axioms of sorting, these refinements have the effect shown in Figure 8. The special form restrict coerces a value to a subsort, provided that the subsort predicate holds in context. Sorting-Alg 4 . also shown in Figure 8 shows the effect of aggregating the clauses of the sorting definition into a conditional using another refinement. 3.3.4. Sorting: Summary One remaining step is to design a definition for C 2 . This spec can be refined either by applying divide-and-conquer to create one of several possible simple merge operators: the straightforward one is the usual linear time sequential merge [4]: an alternate derivation yields the log time parallel merge underlying Batcher ' s Sort [8]. Another way to synthesize a definition for C 2 is to reduce it to "legacy code' ' : that is. an existing library routine that can satisfy the C 2 requirement specification. This reduction process uses a general mechanism called connections betweeen theories [7]. The code generation process uses library refinements from the specification language to a programing language (current ly in Specware t hese are CommonLISP and C++). V(x : Bag-as-S q)(bag-singleton?(x) = > 3(e)(x = bag-singleton(e) A soriinq(x) = Cι(e))) = quantifier change via q : Seq -4 Bag-as-Seq = quotieni(perm?) V(y : Seq)(bag-singleton?(q(y)) == 3(e)(q(y) = bag-singlelon(e) A sorting(q(y)) — Cι(e))) = simplifying, using the defs of bag-singleton? and bag-singleton V(y : Seq)(singlelon-seq?(y) == • 3(e)(q(y) = q(singleion-seq(e)) A sorling(q(y)) = Cι (e))) = quantifier change via the subsort coercion rl : 1-Seq -4 Seq — relax(singleton-seq?) V(z : l-Seq)3(e)(q(rl( 2 )) = Λ sσrlhιg(q(rl(z))) = C t (e)) = unifying with q(r\(ιυ)) = q(singleton-seq(singlet.on-seq-inv(ιυ))) which is inferred from the inverse axiom rl(w) = singlelon-seq(sin gl lon-seq-inv(w)) V(z : 1-Seq) soriing(q(rl(z))) = Cι (singletυn-stq-inv(z)) — unfolding the def of Ci V(z : 1-Seq) sorting(q(rl (z))) = sin gleton-se.q(sin gl eton-seq-inv(z)) = simplifying, again using the inverse axiom V(z : 1-Seq) sorting(q(r\ (z))) = r (z) = quantifier change in reverse, using subsort coercion rl : 1-Seq — Seq V(y : Seq) singltton-stq?(y) =4> sorling(q(y)) = y Figure 7: Simplification of a sorting axiom spec Sorting-Algz is import Bag-as-S eq-S eq-over-LinOrd op sorting : Bag-as-Seq — Seq theorem sorted? (x. sorting(x)) definition of sorting is axiom ernptg-seq?(x) =$> sυrling(q(x)) = x axiom singl elon-seq?(x) => sortιng(q(x)) = x axiom nonsingleton-seq?(x) ==> let (xl = q(left-split(restrict(x))). x2 = q(right-split(restrict(x)))) sorting(q(x)) = C 2 (sort777(7(xl). sorr!7n<7(.τ2))) end-definition end-spec spec Sorting- Al ^ is import Bag-as-S eq-S eq-over-LinOrd op sorting : Bag-as-Seq — >• Seq theorem sorted? (x. sorting(x)) def sort ng(q(x)) = (if empty-seq? (x) then x elseif sr gle on-seq?(x) then x else (let (xl = q(left-split(restrιct(x))). x2 — q(right-spht(restrιct(x)))) C 2 (sortirιg(xl ). sorling(x2))) end-spec Figure S: Sorting Algorithm after Simplifications 4. Domain-Specific Software Development The sorting example shows how to exploit domain-independent design knowledge that is uniformly represented by refinements. In this section the Planware system provides examples of using refinements to represent domain-specific design knowledge. Planware [1] is a synthesis system that is specialized to the production of scheduling algorithms. It extends Specware/Designware with libraries of theories and refinements about scheduling and with a specialized tactic for controlling the application of such design knowledge. Planware applies library refinements to transform information from the user into a formal requirement specification, which is typically thousands of line of specification text. The refinement to code via algorithm design, datatype refinement , and expression optimization is completely automatic. 4.1. Constructing a Requirement Specification Planware provides an answer to the question of how to help automate the acquisition of requirements from the user and to assemble a formal requirement specification for the user. The key idea is to focus on a narrow well-defined class of programs and to prebuild an abstract specification that covers the class. Interaction with the user is only required in order to obtain the refinement from the abstract spec to a specification of the requirements of the user ' s particular problem. The scheduling problem in general is to compute a set of reservations on a given set of resources in order to accomplish a given set of tasks subject to certain constraints. Optionally we may want to optimize an objective function. A sketch of an abstract scheduling specification follows, where the import Scheduling-Import combines specifications for Time. Quantity, Task. Resource, Reservation, Set-of- Task, Set-of-Resource, Sel-of-Reserυalιon: spec Task sort Task end-spec spec Resource sort Resource end-spec spec Abstract-Scheduling is imports Scheduling-Import sorts Reservation = Re source. Task. Time Schedule = Sel-oJ -Reservation op scheduler : Set-of-Task, Set-of-Resource -4 Schedule op all -tasks-scheduled? : Set-of-Task, Schedule -4 Boolean def all -la sks-schtd ul ed? (tasks, schedule) = V(lsk)( tsk in tasks = 3\(rsvn)(rsvn in schedule A iask(rsvn) = tsk)) axiom all-tasks-schedultd? (tasks, scheduler (tasks . resources)) T U 01/12409 Resource Consumable Asynchronously Reusable Synchronously Reusable Transportation ExactCapacity • i UnitCapσcity Figure 9: Taxonomy of Resource Theories op only-given-resources-used? : Set-of-Resource. Schedule -4 Boolean def only-grven-resources-used?(resources. schedule) = (rsυn)( rsυn in schedule = resource(rsυn) in resources) axiom only-gιven~resources-used?(resources. scheduler(tasks. resources)) end-spec This specifies an abstract scheduler whose input is a set of tasks (each with unspecified structure) and a set of resources (each with unspecified structure). Two generic problem constraints are imposed on the output of the Scheduler operator: all-tasks-schedultd? asserts that il must schedule all (and only) the input tasks, and only-giυen-resυurcts-ustd? asserts that it uses only the supplied resources. So. for example, if the input is 100 cargo items to fly on 10 aircraft, then the schedule generated by Scheduler must schedule all 100 items (and no more) and use only the 10 given aircraft. We briefly describe two domain-specific refinements that are applied to refine this abstract scheduling spec to a spec for a particular problem, such as a transportation problem, a power plant maintenance problem, or a processoi scheduling problem. Figure 9 depicts a taxonomy of resource theories that covers many commonly occurring types of resources. Abstract Resource theory refines into Consumable resources (e.g. fuel, crew duty time) and Reusable resources (e.g. aircraft, trucks, parking lots, printers). Reusable resources further refine into Synchronously Reusable resources (e.g. ships, aircraft) where all reservations of a resource instance must be synchronized (e.g. passengei leservations on a particular airline flight are all for the same start and finish time). Synchronously Reusable resources further refine into Transportation resources (which have spatial translation between starting and ending points), and Unit Capacity resources, which can handle at most one reservation at a time. The axiomatic formulation has several benefits over an object-oriented classification hierarchy. Both provide the signature of the relevant operations on the resource. A specification for a resource type also provides the axiomatic characterization of all properties T U 01/12409 spec Semilattice-Struciured- Attribute is sorts Task, Attr-sort op attribute : Task -4 Attr-sort op rsvn-attr : Reservation -4 Attr-sort op Je_ : Attr-sort. Attr-sort -4 Boolean axioms ... le is a bounded semilattice ... end-spec spec Create-C on stra nt-from-T ask- Attribute is import Semilattice-strυctυred-attribute op con si stent-attr : Schedule -4 Boolean def co sιstent-attr(sched) = /(rsvn. Isk)(rsυn 6 sched A tsk in lasks(rsvn) =>• attr(tsk) le rsvn-attr(rsvn)) axiom consιstent-attr(scheduler(tasks. resources)) end-spec Figure 10: Refinement for Constructing Requirement Constraints on a Schedulei shared by instances of the resource class. Furthermore, important theorems that prov ide the basis for important algorithmic concepts can be recorded in the resource spec. To particularize the abstract scheduling specification, Planware requests the user to select a resource theory from the taxonomy. The composed arrow from Resource theory to the selected theory is the arrow used for refinement. In the next step, the user interacts with a spreadsheet-like interface that simultaneously gathers information about task attributes and how they are ordered as a semilattice - this information is critical for algorithm design later. We skip the details of this process in order to focus on how task attributes are lifted into requirement constraints on Scheduler (see [1]). As an example, consider the release-time attribute on Task which we suppose the user has selected as a lower bound on the start-time attribute which is required on reservations of a transportation resource. A domain-specific refinement is used to create problem requirement constraints is shown in Figure 10: The codoπidin of the refinement creates a new predicate which is asserted as a requirement constraint on Scheduler. The axiom asserts that the scheduler must produce an output thai satisfies this new constraint. From context. Planware automatically constructs the classification arrow: Task 1-4 Task attribute i- release-time rsvn-attr l→ start-time Ie H4 < The pushout then automatically adds the new structure, including the requirement constraint on Scheduler, to the Scheduling specification: 01 12409 spec Abstract-S chedulingi is imports Scheduling-Import sorts Reservation = Resource. Task. Time Schedule = Set-of -Reservation op scheduler : Set-of-Task. set-of -Resource -4 Schedule op all-iasks-scheduled? : Set-of-Task. Schedule -4 Boolean def all-tasks-scheduled? (tasks, schedule) = (tsk)(isk in tasks = 3\(rsvn)(r sυn in schedule A task(rsvn) = iόfc)) axiom all -tasks-scheduled? (tasks. schedυler(tasks. resources)) op only-given-resources-used? : Set-of-Resource. Schedule -4 Boolean def υnly-gtυen-resources-used?(resourcts. schedule) = V(rsυn)( son m schedule => rtsourct(rsvn) in resources) axiom only-gιven-resources-used?(resources.5 'cheduler (tasks, resources)) op rsvn-start-time : Reservation -4 Time op co i tent-start-time : Schedule -4 Boolean def con sr stevt-start-tιme(sched) = V(rsvn. tsk)(rsvn in sched A tsk in tasks(rsvn) = ■ release-tιme(tsk) < rsvn-starf- i me(rsvn)) axiom consιstent-start-t me(scheduler(tasks. resources)) end-spec This new requirement constraint on the scheduler asserts that the actual start time of a reservation must be no earlier than the release date of any task in the reservation. Applying this domain-specific yet abstract refinement to all attributes of Task gathered from dialog with the user allows Planware to automatically generate the requirements constraints on Scheduler (without the user having to know. see. or write advanced mathematics). 4.2. Datatype Refinement and Problem Reformulation Datatype refinement plays several important roles in Planware. We briefly mention one use at the problem formulation stage that is critical to the development of good algorithms at a latei stage. Sei(Resource x Tasl x Time) Map( Resource. Set,( Resource x Task x Time)) Map(Resource. Ordered-Seq(Resource x Task x Time)) At the most abstract level in Planware. a schedule is presented as a set of reservations (which in turn are effectively tuples). This abstraction is well-suited for gathering and formalizing user requirements, but quite poor if implemented in a straightforward or naive way (e.g. as lists). Planware exploits refinements shown above, where the second refinement exploits the linear ordering of time to refine a set to a sorted sequence (ordered by increasing time). The effect is to Tefine schedule-as-a-relatioπ into schedule-as-a-Gantt- chart (or time-function) - a common representation in scheduling algorithms. The main benefit is that several of the output constraints on the scheduler function now simplify tremendously. Put another way, the refined data structures satisfy some of the constraints implicitly, they are built-in, so they need not be explicit. A clear example of this phenomenon arises in the ά-queens problem. A straightforward specification of this problem would define d - k x k bit matrix to represent placement of queens on a k x k chessboard. In this representation we have four constraints to satisfy: no-two-queens-per-row. no-two-queens-per-column ; no-two-queens-per-ascending- diagonal. no-two-queeiis-per-descending-diagonal. The constraint that there is at most one queen per column allows us to refine the bit matrix to a sequence where the i th entry represents the row number of the queen in that column, if there is one. The effect is to cut down the space of possible solutions by building in the constraint no-two-queens-per- column. Furthermore, the specification of the queens problem can be condensed since the no-two-queens-per-column constraint simplifies away in the refined representation. This phenomenon points out another benefit of a refinement approach: it supports users in exploring alternative formulations of their problem. A complicated (i.e. lots of information) but efficient (i.e. tighter representation of the problem so that there's less junk in the search space) representation can be derived from a simpler, more understandable formulation. Oftentimes in current software development approaches the inertia of the lots-of-bits formulation may discourage users from backing up and rethinking/reformulating their approach. The refinement approach encourages a carefully staged design process wherein the user first expresses the most abstract essential requirements on the desired software, and only then begins to explore decisions about formulation, algorithms, architectures, data structures, and so on. 5. Scaling up The process of refining specification So described above has three basic steps: 1. select a refinement A == B from a library. 2. construct a classification arrow A =Φ- So, and 3. compute the pushout Si of B <S= A = S 0 . The resulting refinement is the cocone arrow S 0 ==Φ- Si . This basic refinement process is repeated until the relevant sorts and operators of the spec have sufficiently explicit definitions that they can be easily translated to a programming language, and then compiled. In this section we address the issue of how this basic process can be further developed in order to scale up as the size and complexity of the library of specs and refinements 01 12409 Container Proto-Seq Proto-Bag Seq Bag Proto-Set Figure 11: Taxonomy of Container Datatypes Expression New- Expression Expression-and-Context Expression-and- Function Figure 12: Taxonomy of Expression Optimization Refinements grows. The first key idea is to organize libraries of specs and refinements into taxonomies. The second key idea is to support tactics at two levels: theory-specific tactics for constructing classification arrows, and task-specific tactics that compose common sequences of the basic refinement process into a larger refinement step. 5.1. Design by Classification: Taxonomies of Refinements A productive software development environment will have a large library of reusable refinements, letting the user (or a tactic) select refinements and decide where to apply them. The need arises for a way to organize such a library, to support access, and to support efficient construction of classification arrows. A library of refinements can be organized into taxonomies where refinements are indexed on the nodes of the taxonomies, and the nodes include the domains of various refinements in the library. The taxonomic links are refinements, indicating how one refinement applies in a stronger setting than another. Figure 1 1 sketches a taxonomy of abstract datatypes for collections. Details are given in Appendix A. The arrows between nodes express the refinement relationship: e.g. the morphism from Proto-Seq to Proto-Bag is an extension with the axiom of commutativity applied to the join constructor of Proto-Seqs. Datatype refinements are indexed by the specifications in the taxonomy; e.g. a refinement from (finite) bags to (finite) sequences is indexed at the node specifying (finite) bag theory. Figure 12 shows the beginnings of a taxonomy of expression optimization refinements that includes the ones presented in Section 3.3.3.. Figure 13 shows a taxonomy of algorithm design theories. The refinements indexed at each node correspond to (families of) program schemes. The algorithm theory associated with a scheme is sufficient to prove the consistency of any instance of the scheme. Nodes that are deeper in a taxonomy correspond to specifications that have more structure than those at shallower levels. Generally, we wish to select refinements that are indexed as deeply in the taxonomy as possible, since the maximal amount of structure in the requirement specification will be exploited. In the algorithm taxonoπn. the deeper the node, the more structure that can be exploited in the problem, and the more problem- solving powei t hat can be brought to bear. Roughly speaking, narrowly scoped but fast ei algorithms die deeper in the taxonomy, whereas widely applicable general algorithms are at shallo ei nodes. Two problems arise in using a library of refinements: (1 ) selecting an appropriate refinement, and (2) constructing a classification arrow. If we organize a libraij of refinements into a taxonomy, then the following ladder construction process provides incremental access to applicable refinements, and simultaneously, incremental construction of classification arrows. The process of incrementally constructing a refine¬ A Sp cu ment is illustrated in the ladder construction diagram to the left. The left side of the ladder is a path in a taxonomy starting at the root . The lad¬ A] > Specj der is constructed a rung at a time from the top down. The initial interpretation from AQ to Spec 0 is often simple to construct. The rungs of the ladder Spec 2 are constructed by a constraint solving process that involves user choices, the propagation of consistency constraints, calculation of colimits. and constructive theorem proving [7. 9]. Generally, the rung construc¬ In tion is stronger than a colimit - even though a co¬ A n Spec„ cone is being constructed. The intent in contracting I, : = Spec t is that Spec t has sufficient defined symbols to serve as the codomain. In other words, the implicitly defined symbols in A t aie translated lo explicitly defined symbols in Spec t . A n B n Again, rung construction is not simply a matter of computing a colimit. For example. Problem Theory (D→ R\0) Assignment Problem GS-Horn-CSP Hungarian method (Definite constraints) constraint propagation Figure 13: Taxonomy of Algorithm Theories 01 12409 there are at least two distinct arrows from Divide-and-Conquer to Sorting, corresponding to a mergesort and a quicksort - these are distinct cocones and there is no universal sorting algorithm corresponding to the colimit. However, applying the refinement that we select at a node in the taxonomy is a simple matter of computing the pushout. For algorithm design the pushout simply instantiates some definition schemes and ot her axiom schemes. It is unlikely that a general automated method exists for constructing rungs of the ladder, since it is here that creative decisions can be made. For general-purpose design it seems that users must be involved in guiding the rung construction process. However in domain-specific settings and under certain conditions it will possible to automate rung const ruction (as discussed in the next sect ion). Our goal in Designware is to build an interface providing the user with various general automated operations and libraries of standard components. The user applies various operators with the goal of filling out partial morphisms and specifications until the rung is complete. After each user-directed operation, constraint propagation rules are automatically invoked to perform souiid extensions to the partial morphisms and specifications in the rung diagram. Constructive theorem-proving provides the basis for several important techniques for constructing classification arrows [7, 9j- 5.2. Tactics The design process described so far uses primitive operations such as (1 ) selecting a spec or refinement from a library, (2) computing the pushout/colimit of (a diagram of) diagram morphisms. and (3) unskolemizing and translating a formula along a morphism, (4) witness-finding to derive symbol translations during the construction of classification arrows, and so on. These and other operations can be made accessible through a GUI. but inevitably, users will notice certain patterns of such operations arising, and will wish to have macros or parameterized procedures for them, which we call tactics. They provide higher level (semiautomatic) operations for the user. The need for at least two kinds of tactics can be discerned. 1. Classifi d tion tactics control operations for constructing classification arrows: The divide-and-conquer theory admits at least two common tactics for constructing a classification arrow which we illustrated in Section 3.3.1.. One tactic can be procedurally described as follows: (1 ) the user selects a operator symbol with a DRO requirement spec, (2) the system analyzes the spec to obtain the translations of the DRO symbols. (3) the user is prompted to supply a standard set of constructors on the input domain D, (4) the tact ic performs unskolemization on the composition relation in each Soundness axiom to derive a translations for Oa, and so on. This tactic was followed in the mergesort derivation. The ot her tactic is similar except t hat t he tactic: selects constructors foτ the composition relations on R (versus D) in step (3), and then uses unskolemization to solve for decomposition relations in step (4). This tactic was followed in the quicksort derivation. A classification tactic for context-dependent simplification provides another example. Procedurally: (1) user selects an expression expr to simplify. (2) type analysis is used to infer translations for the input and output sorts of expr. (3) a context analysis routine is called to obtain contextual properties of expr (yielding the translation for C). (4) unskolemization and witness-finding are used to derive a translation for new-expr. 2. Refinement tactics control the application of a collection of refinements: they may compose a common sequence of refinements into a larger refinement step. Planware has a code-generation tactic for automatically applying spec-to-code interlogic moiphisms. Another example is a refinement tactic for context-dependent simplification: procedurally. (1) use the classification tactic to construct the classification aiiow. (2) compute the pushout. (3) apply a substitution operation on the spec to replace expr with its simplified form and to create an isomorphism. Finite Differencing requires a more complex tactic that applies the tactic for context-dependent simplification repeatedly in order to make incremental the expressions set up by applying the Expressiυn-and-F unction ~¥ Abstracted-Op refinement. We can also envision the possibility of metatactics that can construct tactics for a given class of tasks. For example, given an algorithm theory, there may be ways to analyze the sorts, ops and axioms to determine various orders in constructing the translations of classification arrows. The two tactics for divide-and-conquer mentioned in Section 3.3.1. are an example. 6. Summary Perhaps the main message of this paper is that a formal software refinement process can be supported by automated tools, and in particular that libraries of design knowledge can be brought to bear in constructing refinements for a given requirement specification. One goal of this paper has been to show that diagram morphisms are adequate to cap- tuie design knowledge about algoiitluns. data structures, and expiession optimization techniques, as well as the lefinement piocess itself. We showed how to apply a library refinement to a requirement specification by constructing a classification arrow and computing the pushout. We discussed how a library of refinements can be organized into taxonomies and presented techniques for constructing classification arrows incrementally. The examples and most concepts described are working in the Specware. Designware. and Planware systems. Acknowledgements: The work reported here is the result of extended collaboration with my colleagues at Kestrel Institute. I would particularly like to acknowledge the contribut ions of David Espinosa. iMei Gilham. Junbo Liu. Dusko Pavlovic. and Stephen Westfold. I would also like to thank Lambert Meertens for his suggestions on treating nondeterministic functions in the Specware context. References [1] B AINE, L., GILHAM, L., LIU , J., SM ITH , D., AND WESTFOLD . S. Planware - domain-specific synthesis of high-performance schedulers. In Proceedings of the Thirteenth Automated Software Engineering Conference (October 1998), IEEE Computer Society Press, pp. 270-280. [2] MESEGUER. J. General logics. In Logic Colloquium 87, H. Ehrig et al., Ed. North Holland, Amsterdam, 1989, pp. 275-329. [3] PAIGE, R., AND KOENIG, S. Finite differencing of computable expressions. ACM Transactions on Programming Languages arid Systems 4-. (July 1982), 402-454. [4] SMITH . D. R. Top-down synthesis of divide-and-conquer algorithms. Artificial. Intelligence 27. 1 (September 1985), 43-96. (Reprinted in Readings in Artificial Intelligence and Software Engineering, C. Rich and R. W r alers, Eds., Los Altos, CA, Morgan Kaufmann. 1986.). [5] SM ITH . D. R. KIDS - a semi-automatic program development, system. IEEE Transactions on Software Engineering Special Issue on Formal Methods in Software. Engineering 16, 9 (September 1990), 1024-1043. [6] SM ITH . D. R. Structure and design of problem reduction generators. In Constructing Programs from Specifications, B. Mόller, Ed. North-Holland, Amsterdam, 1991 , pp. 91-124. [7] SM ITH , D. R. Constructing specification morphisms. Journal of Symbolic Computation. Special Issue on Automatic Programming 15, 5-6 (May-June 1993). 571-606. [8] SM ITH . D. R. Derivation of parallel sorting algorithms. In Parallel Algorithm Derivation and Program Transformation, R. Paige. J. Reif. and R. W r achter, Eds. Kluwer Academic Publishers, New York, 1993, pp. 55-69. [9] SM ITH . D. R. Toward a classification approach to design. In Proceedings of the Fifth International Conference on Algebraic Methodology and. Software Technology, A MAST'96 (1996), vol. ' LNCS 1101 , Springer- Verlag, pp. 62-84. [10] SRINIVAS. Y. V.. AND JULLIG. R. Specware: Formal support for composing software. In Proceedings of the Conference on Mathematics of Program Construction, B. Moeller, Ed. LNCS 947, Springer- Verlag, Berlin, 1995, pp. 399-422. A Containers, Bags, and Sequences In this section we present parts of the following taxonomy of Container structures and their refinements. This taxonomy is adapted from the Specware/Designware library. One refinement is given in detail: Bag =≠> Seq. Container Proto-Seq Proto-Bag Seq spec CONTAINER is sorts E, Container op empty : -> Container op singleton : E -> Container op _j oin_ : Container , Container -> Container axiom unit is x j oin empty = x & empty j oin x = x constructors {empty , singleton, j oin} construct Container op empty? : Container -> Boolean def inition of empty? is axiom empty? (empty) = true axiom empty? (singleton (y) ) = false axiom empty? (U j oin V) = (empty? (U) k empty? (V) ) end-de inition op nonempty? : Container -> Boolean def nonempty?(b) = "empty?(b) op _in_ : E, Container -> Boolean definition of in is axiom x in empty = false axiom x in singleton(y) = (x = y) axiom x in U join V = (x in U or x in V) end-de ini ion end-spec spec PROTO-SEQ is translate colimit of diagram nodes ASSOCIATIVE, CONTAINER, BIN-OP arcs BIN-OP -> ASSOCIATIVE : {} , BIN-OP -> CONTAINER : {E -> Container , binop -> j oin} end-diagram by {Container -> Seq, empty -> empty-seq, empty? -> empty-seq? , singleton -> singleton-seq, join -> concat} The definit ion of PROTO-SEQ given above evaluates to the following "flat' ' spec- spec PR0T0-SEQUENCE is sorts E , Seq const empty-seq : -> Seq op singleton-seq : E -> Seq op _concat_ : Seq, Seq -> Seq constructors {empty-seq, singleton-seq, concat} construct Seq axiom unit is forall(x:Seq) (x concat empty-seq = x & empty-seq concat x = x) axiom associativity is x concat (y concat 2) = (x concat y) concat 2 op empty-seq? : Seq -> Boolean de inition of empty-seq? is axiom empty-seq? (empty) = true axiom empty-seq?(singleton(y)) = false axiom empty-seq?(U concat V) = (empty-seq?(U) & empty-seq?(V)) end-definition op nonempty-seq? : Seq -> Boolean def nonempty-seq?(b) = "empty-seq?(b) op _in_ : E, Seq -> Boolean de inition of in is axiom x in empty-seq = false axiom x in singleton-seq(y) = (x = y) axiom x in U concat V = (x in U or x in V) end-definition end-spec spec PROTO-BAG is translate colimit of diagram nodes BIN-OP , COMMUTATIVE, PROTO-SEQ arcs BIN-OP -> COMMUTATIVE : {} , BIN-OP -> PROTO-SEQ : {E -> Seq, binop -> concat} end-diagram by {Seq -> Bag, empty-seq -> empty-bag , empty-seq? -> empty-bag? , singleton-seq -> singleton-bag, concat -> bag-union} spec PRQTO-SET is translate colimit of diagram nodes BIN-OP , IDEMPOTENT , PROTO-BAG arcs BIN-OP -> IDEMPOTENT : {} , BIN-OP -> PROTO-BAG : {E -> Bag, binop -> bag-union} end-diagram by {Bag -> Set , empty-bag -> empty-set , singleton-bag -> singleton-set , bag-union -> union} We must extend the PROTO- specs with additional sorts and operations to get useful theories. In practice there would be several extensions to Proto-SEQ yielding a subtax- onomy of Sequence theories, each useful in different contexts. Note the use of subsort definitions here. Subsorts are necessitated by the requirement that all operators be total. For example, NE-seq (short for Non-Empty sequences) is defined as sequences restricted to nonempty sequences. The selectors head, and tail can then be introduced as with signature NE-seq ->■ Seq. A subsort construction E\p? comes with an injective coercion function relax(p?) : E\p? — >■ E that takes subsort elements into the parent sort. The function r is introduced as an abbreviation of relax (nontmply-stq? ) which coerces from the subsort (NE-seq) to the parent sort (seq). Note also how SEQ introduces destructors as inverses of constructors. These play a key role in divide-and- conquer-style algorithms. Unit denotes the empty product construction - the sort with one element, (). spec SEQ is import PROTO-SEQ sort NE-Seq sort-axiom NE-Seq = Seq | nonempty-seq? op r: NE-seq -> Seq def r = relax(nonempty-seq?) op prepend : E , Seq -> Seq op head : NE-Seq -> E op tail : NE-Seq -> Seq constructors {empty-seq, prepend} construct Seq axiom prepend(a,S) = singleton(a) concat S op size: Seq -> Nat definition of size is axiom size(empty-seq) = 0 axiom size(prepend(a,S)) = 1 + size(S) end-definition sort 0-Seq sort-axiom 0-Seq - Seq I empty-seq? op rO: 0-seq -> Seq def rO = rela (empty-seq?) op empty-seq-inv : 0-seq -> Unit inversion of constructor empty-seq is discriminator empty-seq? destructors [empty-seq-inv] axiom fa(x: 0-seq) empty-seq(empty-seq-in (x)) = rθ(x) sort 1-Seq sort-axiom 1-Seq = Seq I singleton-seq? op singleton-seq? : Seq -> Boolean def singleton-seq? (x) = (size(x)=l) op rl : 1-seq -> Seq def rl = relax(singleton-seq?) op singleton-seq-inv : 1-Seq -> E inversion of constructor singleton-seq is discriminator singleton-seq? destructors [singleton-seq-inv] axiom fa(x: 1-Seq) singleton-seq(singleton-seq-inv(x)) = rl(x) sort 2-Seq sort-axiom 2-Seq = Seq I nonsingleton-seq? op nonsingleton-seq? : Seq -> Boolean def nonsingleton-seq?(x) = (size(x)>l) op r2: 2-seq -> Seq def r2 = relax(nonsingleton-seq?) op left-split : 2-Seq -> Seq op right-split : 2-Seq -> Seq inversion of constructor concat is discriminator nonsingleton-seq? destructors [left-split, right-split] axiom fa(x:2-Seq) concat(left-split(x) , right-split (x)) = r2(x) end-spec spec BAG is import PROTO-BAG op size: Bag -> Nat definition of size is axiom size(empty-bag) = 0 axiom size(singleton-bag(e)) = 1 axiom size(bag-unio (bl, b2)) = size(bl) + size(b2) end-definition op bag-wflgt : Bag, Bag -> Boolean def bag-wfgt (bl, b2) = (size(bl) > size(b2)) op singleton-bag? : Bag -> Boolean def singleton-bag?(b) = (size(b)=l) op nonsingleton-bag? : Bag -> Boolean def nonsingleton-bag?(b) = (size(b)>l) end-spec The following text gives the refinement of bags to sequences, as described in Section 2.4.3. Note the use of a quotient sort definition here. The sort Bag-as-Seq is defined as sequences quotiented by the permutation relation. The purpose of Bag-as-Seq is to serve as the translation of Bag. The quotienting captures the commutativity of bags: we say two sequences are equal as bags exactly when t hey are permutations of one another. When we finally translate down to code, equality on bags will translate to the permutal ion relation on the implementing sequences. A quotient sort construction E\ = comes with a surjective coercion function quotient^) : E -> Ej = that takes E elements to their equivalence class. The function q is introduced as an abbreviation of quotient (perm?). spec BAG-AS-SEQ is import SEQ op remove- 1 : E , Seq -> Seq definition of remove-1 is axiom empty-seq? (S) => remove- l (x S)=S axiom x=head(S) => remove-l (x , r(S) )=tail(S) axiom ~ (x=head(S) ) => remove-l(x, r(S))=prepend(head(S) , remove-l(x, tail(S))) end-definition op perm? : Seq, Seq -> Boolean definition of perm? is axiom empty-seq? (Sl)=> (perm?(SI S2) = empty-seq?(S2)) axiom perm?(r(SI) , S2) => head(Sl) in S2 & perm?(tail(SI) , remove-1(head(SI) , S2)) end-definition sort Bag-as-Seq sort-axiom Bag-as-Seq = Seq/perm? constructors {bag-empty, bag-singleton, bag-union} construct Bag-as-Seq op q : Seq -> Bag-as-Seq def q = quotient (perm?) op bag-empty : Bag-as-Seq def bag-empty = q(empty-seq) op bag-empty? : Bag-as-Seq -> Boolean def bag-empty?(q(s)) = empty-seq?(s) op bag-nonempty? : Bag-as-Seq -> Boolean def bag-nonempty?(q(s)) = "empty-seq?(s) op bag-singleton : E -> Bag-as-Seq def bag- singlet on (x) = q(singleton-seq(x)) op bag- singlet on? : Bag-as-Seq -> Boolean def bag-singleton? (q(s) ) - singleton-seq? (s) op bag-nonsingleton? : E -> Bag-as-Seq def bag-nonsingleton(q(s) ) = nonsingleton-seq? (s) op bag- in : E, Bag-as-Seq -> Boolean def bag-in(x, q(S) ) = x in S op bag-union : Bag-as-Seq, Bag-as-Seq -> Bag-as-Seq def bag-union(q(Sl) , q(S2) ) = q(concat (Sl , S2) ) op bag-size : Bag-as-Seq -> Nat definition of bag-size is axiom bag-size(q(s) ) = size(s) end-def inition op bag-wflgt : Bag-as-Seq, Bag-as-Seq -> Boolean de inition of bag-wflgt is axiom bag-w lgt (bl , b2) = gt (bag-size (bl) , bag-size(b2) ) end-def init ion end-spec Finally the refinement from BAG to SEQ-AS-BAG is diagram-morphism BtoS is {BtoS-Triv : Triv -> Triv , BtoS-Bag : Bag -> Seq-as-Bag} diagram BAG is nodes Bag , Triv arcs Triv -> Bag: {E-> E} end-diagram diagram BAG-AS-SEQ is nodes Seq, Bag-As-Seq, Triv arcs Triv -> Bag-As-Seq : {E -> E} , Triv -> Seq : {E -> E} , Seq -> Bag-As-Seq: import-morphism end-diagram morphism BtoS-Bag : Bag -> Bag-As-Seq is {Bag -> Bag-As-Seq, empty-bag -> bag-empty, empty-bag? -> bag-empty? , nonempty? -> bag-nonempty? , singleton-bag -> bag- singleton , singleton-bag? -> bag- singlet on? , nonsingleton-bag? -> bag-nonsingleton? , in -> bag- in, bag-union -> bag-union, bag-wfgt -> bag-wfgt , size -> bag-size} B Specification for Sorting Listed below are the components of the structured specification for the problem of sorting a bag whose elements are drawn from a linearly ordered set. The specification is parameterized on the linear order. LINEAR-ORDER BAG-SEQ-LinOrd BAG-SEQ-over-LinOrd SORTI G spec BAG-AND-SEQ is colimit of diagram nodes TRIV, BAG, SEQ arcs TRIV -> BAG: {}, TRIV -> SEQ: {} end-diagram spec BAG-AND-SEQ-CONV is import BAG-AND-SEQ op seq-to-bag: Seq -> Bag definition of seq-to-bag is axiom seq-to-bag(empty-seq) = empty-bag axiom seq-to-bag(smgleton-seq(e) ) = smgleton-bag(e) axiom seq-to-bag(concat (Sl , S2) ) = bag-unιon(seq-to-bag(Sl) , seq-to-bag(S2)) end-def mit ion end- spec spec BAG-SEQ-LmOrd is colimit of diagram nodes TRIV, LINEAR-ORDER, BAG-AND-SEQ-CONV arcs TRIV -> LINEAR-ORDER: {} , 112 TRIV -> BAG-AND-SEQ-CONV: {} end-diagram spec BAG-SEQ-over-LinOrd is import BAG-SEQ-LinOrd op all-le: Bag, Bag -> Boolean de inition of all-le is axiom all-le(empty-bag, empty-bag) = true axiom all-le(B, empty-bag) = true axiom all-le(singleton-bag(e) , empty-bag) = true axiom all-le(singleton-bag(d) , singleton-bag(e)) = (d le e) axiom all-le(singleton-bag(d) , bag-union(Bl, B2)) = all-le(singleton-bag(d) , Bl) & all-le(singleton-bag(d) , B2) axiom all-le(bag-union(Bl, B2) , B) = all-le(Bl, B) & all-le(B2, B) end-de inition op ordered? : Seq -> Boolean de inition of ordered? is axiom ordered?(empty-seq) = true axiom ordered?(singleton(a)) = true axiom ordered? (prepend(a, y)) = all-le(singleton(a) , seq-to-bag(y)) axiom ordered? (concat(y, z)) = ordered?(y) & all-le(seq-to-bag(y) , seq-to-bag(z)) & ordered?(z) end-definition end-spec spec SORTING is import BAG-SEQ-over-LinOrd op sorted? : Bag, Seq -> Boolean def sorted?(x,2) = (ordered?(z) & x = seq-to-bag(z)) op sorting : Bag -> Seq axiom sorted? (x, sorting(x)) end-spec

Claims: What is claimed is: 1. A method of developing a self-modifying computer program, comprising: providing a set of partial software code implementations; providing a set of specifications describing functionality of a software program, certain specifications in the set of specifications being paired with certain partial software program code implementations in the set of partial software program code implementations; generating a software program that includes the set of partial software program code implementations and having functionality described by the set of specifications; redefining the functionality of the software program when the generated software program does not comport to a minimum functionality standard; and regenerating the software program so that it still includes the set of partial software program code implementations but has the redefined functionality. 2. The method of claim 1, where the redefined functionality is mandated by a change in an external interface of the software program. 3. The method of claim 1, where the redefined functionality is mandated by a change in an interface within the software program. 4. A method of developing a self-modifying computer program, comprising: providing a set of partial software program code implementations; providing a set of specifications describing functionality of a software program, certain specifications in the set of specifications being paired with certain partial software program code implementations in the set of partial software program code implementations; generating a software program that includes the set of partial software program code implementations and having functionality described by the set of specifications; redefining the set of partial software program code implementations; and regenerating the software program so that it includes the redefined set of partial software program code implementations but still has the functionality described by the set of specifications. 5. The method of claim 4, where the redefined partial software program code implementations make the regenerated software program more efficient. 6. The method of claim 4, where the redefined partial software program code implementations make the regenerated software program faster executing. 7. The method of claim 4, where the redefined partial software program code implementations are mandated by a version change. 8. The method of claim 4, where the redefined partial software program code implementations are mandated by a change in an interface of the software program. 9. The method of claim 4, where the redefined partial software program code implementations are mandated by a change in an interface within the software program. 10. A method of developing a self-modifying computer program, comprising: providing a set of partial software program code implementations; providing a set of functional comments describing functionality of a software program, certain functional comments in the set of functional comments being paired with certain partial software program code implementations in the set of partial software program code implementations; generating a software program that includes the set of partial software program code implementations and having functionality described by the functional comments; redefining the set of functional comments; and regenerating the software program so that it still includes the set of partial software program code implementations but has redefined functionality in accordance with the set of functional comments.

Inventors: FITZPATRICK STEPHEN, PAVLOVIC DUSKO, SMITH DOUGLAS R, FITZPATRICK, STEPHEN, PAVLOVIC, DUSKO, SMITH, DOUGLAS, R.
Applicants: KESTREL INST, KESTREL INSTITUTE
Application year: 2001
Main classification: G06F9
IPC codes: G06F9