DANCE/MULTITUDE CONCURRENT COMPUTATION

Abstract: This invention computes by constructing a lattice of states (figure 34). Every lattice of states (figure 6) corresponding to correct execution satisfies the temporal logic formula comprising a DANCE program (33). The invention integrates into a state-lattice (figure 34) computational model: a polymorphic strong type system; visibility-limiting domains; first-order assertions; and logic for providing a program correctness. This invention includes special hardware means (figure 24) for elimination of cache coherency among other things, necessary to construct state-lattices (figure 6) concurrently. The model of computation for the DANCE language consisting of four, interrelated, logical systems describing state-lattices (figure 6), types, domains (figure 42) and assertions. A fifth logical system (BPL) interrelates the other four systems allowing proofs of program correctness. The method of the present invention teaches programs as temporal formulas satisfied by lattices of states (figure 6) corresponding to correct execution. State-lattices (figure 6) that are short and bushy allow application of many processors simultaneously, thus reducing execution time.

Description: DANCE/Multitude Concurrent Computation Brian Ralph Larson, inventor. BACKGROUND-FIELD OF INVENTION This invention relates to execution of computer programs, specifically to performing multiple state transitions simultaneously. BACKGROUND-DISCUSSION OF PRIOR ART 1 New Parallel Language Needed I surveyed ways to program parallel machines. Despite their manufacturers claims, I do not consider SIMD (single-instruction, multiple- datastream) machines like the Connection Machine (CM-2) to be parallel processors. SIMD machines are restricted to performing the same operation in many cells and thus can perform a narrow range of applications. Most applications require much data-dependent branching that kills performance on SIMD machines such as the Cray YMP-8 and CM-2 alike. Good old FORTRAN with a vectorizing compiler is just fine for SIMD machines. Every attempt to wring efficient, massive parallelism from dusty-deck FORTRAN programs has failed. Only programs expressed with a language embodying an inherently parallel model of computation can execute efficiently in parallel. The real trick is to coordinate activities in processing nodes that are doing different things. Modifications to special-purpose languages like Lisp and Prolog with futures and guards respectively are problematic, addressing restricted application domains of largely academic interest. Then there are the many extensions of sequential languages. Such extensions fall into two classes: processes and "doall." Ada is the best example of a process based language. Each Ada task (i.e. process) represents a single thread of control that cooperates with other tasks through message passing. Hoare's Communicating Sequential Processes ["Communicating Sequential Processes," C.A.R. Hoare, Communications of the ACM, Vol. 21, No. 8, August 1978] forms the underlying model of computation for message-passing, process-based languages. However, deciding whether an arbitrary collection of processes is deadlock-free is undecidable and proving that a particular collection of processes is deadlock-free is onerous. Furthermore, such pairwise communication and synchronization is unnecessarily restrictive thus limiting the degree of parallelism. Finally, a programmer writes (essentially) a bunch of sequential programs rather than a single, parallel program. The "doall" is just a FORTRAN do-loop in which all the iterations are instead performed concurrently; again a collection of sequential programs. 2 Current, Conventional, Concurrent Computing Traditional, sequential computing causes a sequence of states to be created so that the last state holds the data desired. Figure 1 illustrates sequential computing starting at an initial state ( 101), proceeding through intermediate states ( 103), and terminating in a final state ( 102). The values of program variables at a particular instant of time comprise a state, depicted as circles. In state diagrams, time always flows down the page, that is, the state at the top of the figure occurs first followed by the state(s) connected by a line. Broken lines indicate repetition or "more of the same." In "parallel" execution several sequences of state transitions occur concurrently. Figure 2 depicts parallel execution in doall style. The state diagram shows state transitions occurring as "parallel" sequences. The conception of parallel processing as a collection of sequential executions characterizes the prior art this invention supersedes. Two mechanisms allow communication between sequential processes: message-passing or shared-memory. Load balancing and problem partitioning have produced many academic papers, for either message- passing or shared-memory paradigms— nearly all yielding poor performance. Mutual exclusion (MutEx) via message passing (i.e. for distributed systems, no shared state) is horribly inefficient: if the application needs MutEx often, performance will suffer. Ricart & Agrawala ["An Algorithm for Mutual Exclusion in Computer Networks," Glenn Ricart & Ashok K. Agrawala, Communications of the ACM. Vol. 24, No. 1, January 1981] published the obvious O(N) request/reply algorithm, which really needs 0(N ) messages when everyone wants the lock. Maekawa ["A */N Algorithm for Mutual Exclusion in Decentralized Systems," Mamoru Maekawa, ACM Transactions on Computer Systems. Vol. 3, No. 2, May 1985] reduces messages to 0(VrT ) with nifty multidimensional geometry. Sanders' generalized MutEx algorithm is even more convoluted ["The Information Structure of Distributed Mutual Exclusion Algorithms," Beverly A. Sanders, ACM Tran.sactions on Computer Systems. Vol. 5 , No. 1, August 1987]. Sanders considers deadlock freedom (absence of possibility that computation ceases because every process is waiting to acquire a lock held by a different process), but the basic model needs further augmentation to detect and recover from deadlock! Of course, only a wordy explanation substitutes for correctness proof. Therefore the cherished interference freedom provided by MutEx is built on mush. Raymond ["A Tree-Based Algorithm for Distributed Mutual Exclusion," ACM Transactions on Computer Systems. Vol. 7, No. 1, February 1989] further reduced the number of messages to O(logN) by increasing the latency, which could easily become a significant burden if processes need the lock briefly. Controlling access to shared variables is so difficult that designers have resorted to the message-passing model in which all state variables are local to some process and processes interact via exchange of messages exclusively. Inevitably, two different kinds of system-wide operations are required by the algorithm or application that are notoriously inefficient: barrier synchronization and remote state acquisition. Barrier synchronization ensures that all the parallel computations in one phase are completed before the next phase is begun. Remote state acquisition occurs when the application's data cannot be partitioned disjointly so that all processes use values exclusively from its partition held in local memory. For example, advanced magnetic resonance imaging (MRI) research seeks to analyze brain scans and call to the attention of the radiologist any anomalies. Three-dimensional fast Fourier transform (FFT) is necessary for filtering and extraction of interesting features. The data consists of a Ik by Ik by Ik array of floating point intensities. The first phase of the algorithm maps nicely by putting a Ik by Ik plane in each of Ik processors. However integrating the third dimension finds the data in the worst possible distribution—every computation needs values that are possessed by each of the Ik processors! Barriers are an artifact of the popular, yet problematic process paradigm. A static collection of sequential processes alternately work independently and wait for laggards. Figure 3 depicts four processes (201 ) synchronized with barriers (204). Each process is actively computing (202) and then wait idly (203) until all other processes reach the barrier. The best (known) algorithms for barrier-synchronization take O(logP) time (with a large constant) for both message-passing and shared-memory models. This corresponds with the depth of a tree using either messages or flags. A large body of academic work has been produced for both message-passing and parallel, random-access machine (PRAM) models (i.e. shared memory). Barrier synchronization in a message-passing model requires O(logP) time (P is number of processors), with a big constant proportional to message transmission latency ["Algorithms for Scalable Synchronization on Shared-Memory Multiprocessors," John M. Mellor-Crummey and Michael L. Scott, ACM Transactions on Computer Systems. Vol. 9, No. 1, February 1991]. Consider an ideally parallelizable application that requires synchronization. A single processor would require N steps (time, T s ) to perform the application with L iterations of a single loop each requiring N/L steps. When parallelized, barrier synchronization is required at the end of each iteration taking logP steps (best case-real systems take many steps for each level of the synchronization tree). So the parallel time for each iteration, T 4 , is N/(L*P) + logP and the whole application Tp, is Ti*L or N/P + L*logP steps. Speedup, S, is the ratio of sequential time to parallel time, T s /T p . Efficiency E, is the speedup obtained divided by the number of processors used, S/P. In this case E = T S /T P *P = N/(N+P*L*logP). When the number of processors is small, N»P*L*logP, the efficiency can be quite good, close to 1. But as more processors are used (N«P) the barrier synchronization overhead begins to dominate even when only a single barrier is needed. Therefore, an application needing barrier synchronization on any conceivable message-passing architecture (N«P), the minimum time of execution will be O(logP), the best speedup possible is 0(P/logP), the efficiency can be no better than 0( l/logP). Little wonder that efficiencies of even 10% are hard to achieve on parallel processors like CM-2, CM-5, NCube, Paragon, and T3D. They can perform no better than their theoretical model. Parallel Random-Access Machines (PRAMs) suffer similar fate. When most people think of "shared-memory" they imagine a PRAM—a bunch of sequential processors that can read and write the same memory locations. Many PRAM variations exist. Whether the memory locations can be accessed concurrently or exclusively and what happens for concurrent accesses constitute most of the PRAM model differences. Even unrealistic operations like storing the median or geometric mean of all concurrent writes still inflicts an O(logP) penalty for barrier synchronization, same as message-passing. Sequential algorithms have been traditionally analyzed using Turing machines. Both the time (number of steps) and the space (number of tape squares) are used to measure the complexity of a computation. Additionally whether the Turing machine has a unique next action (deterministic) or a set of possible next actions (nondeterministic) substantially affects the complexity measure of a computation. The following paragraphs summarize three chapters of a recent book on parallel algorithms [Synthesis of Parallel Algorithms, John H. Reif, editor, Morgan Kaufmann, 1993] that I believe is state-of-the-art computer science theory for shared-memory computers and my disappointment with the current state of computer science "theory." The following sections quote heavily those authors (Faith E. Fich, Raymond Greenlaw, and Phillip B. Gibbons) and will be indented to indicate quotation. PRAMs attempt to model the computational behavior of a shared-memory parallel processor. The alternative is shared-nothing, message-passing parallel processors, which are popular these days. PRAMs have many variations and have a large body of published scholarship. Fich, Greenlaw, and Gibbons cover the main kinds of PRAMs and are deemed (by me) to represent the conventional wisdom about how to think about parallel computation. This paradigm has, so far, not yielded efficient, general- purpose, parallel processing. Thus the motive for this invention. Fich's chapter begins with a concise definition of a "synchronous" PRAM. The PRAM is a synchronous model of parallel computation in which processors communicate via shared memory. It consists of m shared memory cells M], . . . , M m and p processors Pi, . . ., Pp. Each processor is a random access machine (RAM) with a private local memory. During every step of a computation, a processor may read from one shared memory cell, perform a local operation, and write to one memory cell. Reads, local operations, and writes are viewed as occurring during three separate phases. This simplifies analysis and only changes the running time of an algorithm by a constant factor. (Underline mine.) Throughout Fich's paper she assumes a synchronous PRAM: single- instruction, multiple-datastream (SIMD). This allows her to "cheat" and ignore synchronization costs and hassles in her analyses. The synchronous PRAM cannot model MIMD machines with their independent, data-dependent branching and uncertain timing. Having all processors read-execute-write each step sidesteps sticky issues like what happens when a memory cell is read and written simultaneously. Furthermore, the execute part of a step can perform any calculation whatsoever! So the lower bounds derived for various algorithms concentrate solely on shared- memory accesses, while totally ignoring the real computation— data transformation in the execute part of the step. Fich describes "forking" PRAM in which new processors are created with a fork operation (as in Unix). A forking PRAM can dynamically describe parallel activities at run-time. (Regular PRAMs have a fixed set of processes/processors.) But a forking PRAM still is a collection of sequential execution streams, not a single, parallel program. A forking PRAM is a PRAM in which a new processor is created when an existing processor executes a fork operation. In addition to creating the new processor, this operation specifies the task that the new processor is to perform (starting at the next time step). After defining a forking PRAM Fich never mentions it again. But she does make claims for PRAMs about "programmer's view," "ignoring lower level architectural details such as . . . synchronization," and "performance of algorithms on the PRAM can be a good predictor of their relative performance on real machines," that I completely disagree with. Since PRAMs exemplify the process paradigm (collection of sequential executions) they may adequately predict performance of algorithms and architectures similarly afflicted (all machines built so far), but certainly not what's possible. Most distinctions between PRAMs involve memory access restrictions. The most restrictive is called exclusive-read exclusive-write (EREW). Ensuring that two processors don't write to the same memory cell simultaneously is trivial with a synchronous PRAM since all operations are under central control. The other four memory access restrictions are: concurrent-read exclusive write (CREW), COMMON, ARBITRARY, and PRIORITY. The last three restrictions differ in how concurrent writes are handled. With COMMON, values written at the same time to the same cell must have the same value. With ARBITRARY, any value written in the same step may be stored in the memory cell. With PRIORITY, the value written by the highest priority process (lowest process number) is stored in the memory cell. Obviously an EREW program would run just fine on a CREW machine; the concurrent read hardware would not be used. The five memory restrictions Fich uses nest nicely in a power hierarchy: PRIORITY > ARBITRARY > COMMON > CREW > EREW Fich then spends much of her chapter showing how many steps it takes one PRAM to simulate a single step of another. Proving that EREW can simulate a step of PRIORITY using 0(log p) steps and ρ*m memory cells takes Fich two pages of wordy prose. Essentially the simulation takes three phases: Phase 1 : Each processor that wants to write marks its dedicated cell from the p EREW cells used for each PRIORITY cell. Phase 2: By evaluating (in parallel) a binary tree of all processors that want to write, the highest priority processor is found. Phase 3: The winning processor writes the "real" memory cell. Since the height of the binary tree is 0(log p) it takes 0(log p) steps for an EREW PRAM to emulate a PRIORITY PRAM. Much of Fich's chapter is devoted to convoluted schemes bv which one model simulates another without mentioning any applications that benefit from a more powerful and expensive model. Gibbons finally provides a reasonable PRAM model in the very last chapter of this 1000 page book. None of the other authors constrain themselves to reasonable models thus obviating any value of their work. In Gibbons' PRAM model: Existing MIMD machines are asynchronous, i.e., the processors are not constrained to operate in lock-step. Each processor can proceed through its program at its own speed, constrained by the progress of other processors only at explicit synchronization points. However in the (synchronous) PRAM model, all processors execute a PRAM program in lock-step with one another. Thus in order to safely execute a PRAM program on an asynchronous machine, there must be a synchronization point after each PRAM instruction. Synchronizing after each instruction is inherently inefficient since the ability of the machine to run asynchronously is not fully exploited and there is a (potentially large) overhead in performing the synchronization. Therefore our first modification to the PRAM model will be to permit the processors to run asynchronously and then charge for any needed synchronization, (underline mine) Additionally shared memory accesses are charged more than local accesses: To keep the model simple, we will use a single parameter d to quantify the communication delay to memory. This parameter is intended to capture the ratio of the median time for global memory accesses to the median time for a local operation. I invented the Layered class of multistage interconnection networks [U.S. patent 4,833,468, LAYERED NETWORK, Larson et al., May 23, 1989] specifically to keep d small. For the Multitude architecture (interconnected by a Layered network) the ratio of time for global memory accesses to local memory accesses is about 3 to 4. This compares with a ratio of local memory access time to cache hit rime of 20 to 30. Gibbons' asynchronous PRAM model uses four types of instructions: global read, global write, local operation, and synchronization step. The cost measures are: The parameter B = B(p), the time to synchronize all the processors, is a nondecreasing function of the number of processors, p, used by the program. In the Asynchronous PRAM model, the parameters are assumed to obey the following constraints: 2 ≤ d < B ≤ p. However a reasonable assumption for modeling most machines is that B(p) e 0(dlog p) or B(p) e O(olog p/log d). Here again is evidence of a logP performance hit for barrier synchronization. A synchronous EREW PRAM can be directly adapted to run on an asynchronous PRAM by inserting two synchronization barriers, one after the read phase and write phase of the synchronous PRAM time step. Thus a single step of a synchronous PRAM can be simulated in IB + 2d+ 1 time on an asynchronous PRAM. Therefore a synchronous EREW PRAM algorithm running in f time using p processors will take t(2B + 2d+ 1) time on an asynchronous PRAM-a significant penalty. Gibbons shows how to maintain the 0(J?r) running time but use only p/B processors by bunching the operations of B synchronous processors into a single asynchronous processor, thus getting more work done for each barrier synchronization. Gibbons transforms an EREW algorithm for the venerable "all prefix sums" operation for an asynchronous PRAM yielding an 0(£? log n/log B) time algorithm which isn't too bad. Gibbons presents asynchronous versions of descaling lemmas (performing the same application with fewer processors) which tack on a term for the barrier synchronizations. Still he uses many barriers in order to obtain behavior similar to synchronous PRAMs. Only when applications are synchronized just when they need it. and those needed synchronizations are really fast, B ~ d ~ 1, will scaled-up parallel processing be possible, which is the contribution of this invention and Layered networks. Gibbons considers several algorithms that run in 0(B log n/log B) time. In particular, he proves an interesting theorem about summing numbers: Given n numbers, stored one per global memory location, and the following four types of instructions: L := G, L := L + L, G := L and "barrier," where L is a local cell and G is a global cell, then the sum of n numbers on an Asynchronous PRAM with this instructions set requires Ω(5 log n/\o~ B) regardless of the number of processors. This is interesting because Multitude can sum n integers in 0( 1) time! Gibbons concludes with "subset synchronization" in which only a subset of processors in the machine need synchronize. Since the barriers involve fewer processors, and thus are shorter, summing numbers runs marginally faster. So the best PRAM model I could find charges PRAM algorithms for their "cheating," but still offers no insights about how to efficiently compute on an MIMD machine. 3 Temporal Logic Moszkowski in his Executing Temnoral Logic Programs [Cambridge University Press, 1986] presented a temporal-logic programming language. His language, Tempura, allows description of parallel execution (unsatisfactorily) within a single program expression. The foundation for Tempura is interval temporal logic. A Tempura interval is a non-empty sequence of states. A state is determined by the values of program variables at a particular instant of time. The sequence of states that form an interval provide a discrete notion of time. The semantics of Tempura were much different than any I had previously encountered. Instead of programs being a step-by-step recipe for how to do the computation, they were logical formulas that are "satisfied" by constructing a "model" that makes the formula "true." Unfortunately the models constructed are simply sequences of states~the process model underlying PRAMs. The semantics of DANCE programs, although also temporal logic formula based, are satisfied by constructing models with a different and superior mathematical structure that allows efficient execution on a plurality of computational engines. It is this mathematical structure that forms the core of this invention. BACKGROUND-MATHEMATICAL FOUNDATIONS In this chapter we explain the notation used throughout this paper and some necessary elementary notions from mathematical logic. Although these notations are fairly standard, this presentation is adapted from Apt and Olderog's book .Verification of Sequential and Concurrent Programs. Krzysztof R. Apt, and Ernst-Rϋdiger Olderog, Springer- Verlag, 1991]. 1 Sets I assume the reader is familiar with the notion of a set, a collection of elements. Finite sets may be specified by enumerating their elements between curly brackets. For example, {T, Fj denotes the set consisting of the Boolean constants T (true) and F (false). When enumerating the elements of a set, I sometimes use ". . ." as a notation. For example, ( 1, . . ., nj denotes the set consisting of the natural numbers from 1 to n where the upper bound, n, is a natural number that is not further specified. More generally, sets are specified by referring to some property of their elements: { x I P } denotes the set consisting of all elements x that satisfy the property P. The bar, I, can be read "such that." For example, { x I x is an integer and x is divisible by 2 } denotes the infinite set of all even integers. We write a e A to denote that a is an element of the set A, and b φ A to denote that b is not an element of A. Often it is convenient to refer to a given set when defining a new set. We write { xeAIPj as an abbreviation for { x I x e A and P ]. Some sets have standard names: 0 denotes the empty set, Z denotes the set of all integers, and No denotes the set of all natural numbers including zero. In a set one does not distinguish repetitions of elements. Thus {T, F} and {T, T, F] are the same set. Similarly, the order of elements is irrelevant. In general, two sets A and B are equal if-and-only-if the have the same elements; in symbols: A = B. Let A and B be sets. Then A c B (and B D A) denotes that A is a subser of B, A Cl B denotes the intersection of A and B, A U B denotes the union of A and B, and A - B denotes the set difference of A and B. Symbolically, A c B if a e B for every a e A, Af|B= {alaeAandaeB J, AUB=(alaeAoraeBj, A-B= { laeAandaeB }. Note that A = B if both A c B and B c A. A and B are disjoint if they have no element in common, that is, A f B = ø. The definitions of intersection and union can be generalized to more than two sets. Let A; be a set for every element i of some other set J. Then C\ ] ej Aj = { a I a e Aj for all i 6 J } Pi j € j Aj - { a I a G Aj for some i e J ] For a finite set A, card(A) denotes the cardinality , or the number of elements of A. For a non-empty, finite set A C Z, min(A) denotes the minimum of all integers in A. 2 Tuples In sets the repetition of elements and their order is irrelevant. If ordering matters, we use ordered pairs and tuples. For elements a and b, not necessarily distinct, (a,b) is an ordered pair or simply pair. Then a and b are called components of (a,b). Two pairs (a,b) and (c,d) are identical, (a,b) = (c,d), if-and-only-if a=c and b=d. More generally, let n be any natural number, n e No. Then if ai , . . ., a n are any n elements, then (ai, . . ., a n ) is an n-fupie. The element ai, where i e jl, . . .,n} is called the i-th component of (ai, . . ., a n ). An n-tuple (a_, . . ., a n ) is equal to an m-tuple (b], . . ., bm) if-and-only-if n=m and for all i e { 1, . . .,n}. Note that 2-tuples are pairs. Additionally, a 0-tuple is written as ! ), and a 1 -tuple as (a) for any element a. The Cartesian product A xB of sets A and B consists of all pairs (a,b) with a e A and b e B. The n-fold Cartesian product A-χ. . .χA n of sets Ai,. . .,A n consists of all n-tuples (ai, . . ., a**) with aj e Aj for i e {1, . . .,n|. If all Aj are the same set A, then the n-fold Cartesian product Ax. . .xA is also written A n . 3 Relations A binary relation R between sets A and B is a subset of the Cartesian product A x B, that is, R A x B. If A=B then R is called a relation on A. For example, i(a,l),(b,2),(c,2)} is a binary relation between {a,b,c} and { 1,2J. More generally, for any natural number n an n-ary relation R between Ai,. . .,A n is a subset of the n-fold Cartesian product A*χ. . .x A n , that is, R Aix. . ,χA π . Note that 2-ary relations are the same as binary relations. Instead of 1-ary and 3-ary relations one usually uses unary and ternary instead. Consider a relation R on a set A. R is called reflexive if (a,a) e R for all a e A; it is called irreflexive if (a,a) £ R for all a e A. R is called symmetric if for all a,b e A whenever (a,b) e R then also (b,a) e R; it is called antisymmetric if for all a,b e A whenever (a,b) e R and (b,a) e R, then a*-=b. R is called transitive if for all a,b,c e A whenever (a,b) e R and (b,c) E R, then also (a,c) e R. The transitive, reflexive closure R * of a relation R on a set A is the smallest transitive and reflexive relation on A that contains R as a subset. The transitive, irreflexive closure R + of a relation R on a set A is the smallest transitive and irreflexive relation on A that contains R as a subset. The transitive, irreflexive closure is necessary because the graphs considered later do not have self-loops. The "plus" superscript may not be standard. The relational composition R- o R 2 of relations Ri and R 2 on a set A is defined as follows: Ri o R 2 = {(a,c) I there exists b e A with (a,b) e Ri and (b,c) e R 2 ). For any natural number n the n-fold relational composition R n of of a relation R on a set A is defined inductively as follows: R 0 = {(a,a) l a ε A!, R n+1 = R n o R. Note that R^ U n eNo R n and that R+ = n_No-{0} R " = R " R where R 1 = R. Membership of pairs in a binary relation R is mostly written in infix notation, so instead of (a,b) e R one usually writes aRb. Any binary relation R AxB has an inverse R "1 c B x A defined as: bR^aiff aRb. 4 Functions Let A and B be sets. A function or mapping from A to B is a binary relation between A and B with the following special property: for each element a e A there is exactly one element b e B with a/b. Mostly we use prefix notation for function application and write (a) = b instead of ab. For some functions, however, we use postfix notation and write a/= b. In both cases b is called the value of/ applied to the argument a. To indicate that/ is a function from A to B we write /:A→B. The set A is called the domain off and the set B is the co-domain off. Consider a function/ : A -* B and some set X c A. Then the restriction off to X is denoted byf[X] and defined as the intersection of/ (which is a subset of A x B) with X x B: irχ]=/n(χχB). We are sometimes interested in functions with special properties. A function/: A → B is called one-to-one or injective if sti ) ≠fiai) for any two distinct elements aι,a 2 e A; it is called onto or surjective if for every element b e B there exists an element a e A with/; a) = b; it is called bijective if it is both injective and surjective. Consider a function whose domain is a Cartesian product, say / : A* x . . . x A n -* B. Then it is customary to drop one pair of parentheses when applying/ to an element (ai, . . .,a n ) ≡ A- . . . x A n . We also say/ is an n-ary function. Consider a function whose domain and co-domain coincide, say/ : A → A. An element a e A is called a fixed point off if /(a) = a. 5 Sequences In the following let A be a set. A sequence of elements from A of length n _* 0 is a function /: {1, . . .,n} → A. We write a sequence/ by listing the values off without punctuation in the order of ascending arguments, that is, as aι . . . a n where ai = (l), . . .,a n = (n). Then aj, i e (1, . . .,n} is referred to as the i-th element in the sequence ai. . . a n . A finite sequence is a sequence of any length n ≥ 0. A sequence of length 0 is called the empty sequence and is denoted by ε. We also allow countably infinite sequences. An infinite sequence of elements from A is a function | : No -> A. To exhibit the general form of an infinite sequence ξ we typically write I : ao ai a 2 . . . if aj =ξ(i) for all i e NQ. Then i is called the index of the element j. Consider now relations Rj, R 2 , . . . on a set A. For any finite sequence a υ . . . a n of elements from A with ao Ri ai, ai R 2 a 2 a n -ι R n a n we write a finite chain ao Ri a- R 2 a 2 . . . a n -ι R n a n . For example, using the relations = and < on Z, we may write We apply this notion also to infinite sequences. Thus for any infinite sequence ao ai a . . . of elements of A with ao Ri ai, ai R 2 a , a 2 R3 3, . . . we write an infinite chain a 0 Ri a- R 2 a? R3 Α3 . . . Such a chain stabilizes if from some index n onwards all elements aj with i ≥ n are identical ao Ri ai R 2 a . . . a n -ι R n a n = an + i = an +2 Apt and Olderog use chains to describe the computations of programs. This invention does not. Chains represent the sequential paradigm of computation that permeates computer science. It is the fundamental limiting factor for both the kind of programs Apt and Olderog can handle, and the efficiency with which those programs can be executed. 6 String s A set of symbols is often called an alphabet. A string over an alphabet A is a finite sequence of symbols from A. For example 1+2 is a string over the alphabet {1,2.+ }. The syntactic objects considered in this treatise are strings. We shall introduce several classes of strings: expressions. assertions, temporal predicates, programs, and correctness formulas. We use ■ for the syntactic identity of strings. For example 1+2 -1+2 but not 1+2 ■ 2+1. The symbol = will be used for the semantic equality of objects. Thus 1+2 = 2+1. The concatenation of strings si and s 2 yields the string sιs formed by first writing si and then s? without intervening spaces. A string t is called a substring of a string s if there exist strings Sj and s? such that s β sιts 2 . Since si and S 2 may be empty, s is always a substring of itself. 7 Partial Orders A partial order is a pair ( A,c ) consisting of a set A and a reflexive, antisymmetric, and transitive relation c on A. If x - y for some x,y e A, we say that x is less than or equal to y or y is greater than or equal to x. Since c is reflexive, x . Sometimes we consider irreflexive partial orders. These are pairs (A, - ) consisting of a set A and an irreflexive and transitive relation c on A. Consider now a partial order (A, c ). Let a e A and X c A. Then a is called the least element of X if a ε X and a c x for all x ε X. The element a is called an upper bound of X if x - a for all x e X. Note that upper bound of X need not be elements of X. Let U be the set of all upper bounds of X. Then a is called the least upper bound of X if a is the least element of U. Similarly for greatest lower bound. A partial order is called complete if A contains a least element and if for every ascending chain ao c- i ~~ a.2 . . . of elements from A the set has a least upper bound. 8 Graphs A graph is a pair (V, E) where V is a finite set of vertices {v* , v 2 , . . ., v n ] and E is a finite set of edges where each edge is a pair of vertices { (V l , V 2 ) , (Vi, V 3 ), . . „ (Vj, Vj) } . All graphs we consider are directed in that the order of vertices within the pair describing an edge is significant. The set of edges E forms a relation on the set of vertices: E C V x V. The transitive, irreflexive closure of E, namely E + , is especially important. In the temporal logic created for DANCE (i.e. this invention) E + is a partial order that corresponds to the concept of precedes. A graph (V, E) is a lattice if E + is an irreflexive partial order, ~, it has a least element 1 e V, and an upper bound u e V. All executions of DANCE programs will create a lattice of states. Lattices may be combined to form new lattices in two ways. Consider two lattices (V- , Ei ) and (V , E 2 ) that have no vertices in common, Vi ) V 2 = 0, least elements e Vj and 1 2 e V , and upper bounds ui e Vj and u e V 2 . Their sequential combination may be performed as follows: substitute ui for 1? in V 2 and E? and form the union of the vertices and edges, (Vj U V 2 , Ei U E?). Their insertion combination may be performed as follows: choose an edge e e E-, e = (Vj. Vj), remove it from i. substitute VJ for 1 2 and Vj for u? in V? and E? and form the union of the vertices and edges, (Vj U V 2 , Ei \J E ). We assert, without proof, that sequential and insertion combination of lattices results in a lattice. 9 Proofs A mathematical proof is a sequence of statements, formulas, or expressions, each of which is an axiom of the logical system used, a given assumption, or a theorem derived by an inference rule from statements earlier in the sequence. The last theorem is usually the desired truth to be established such as the program meets its specifications and therefore is correct. We will exhibit proofs in the following form: ( 1) theorem 1 => [explanation why theorem 2 is derived from theorem 1] (2) theorem 2 (n-1) theorem n-1 => [explanation why theorem n is derived from theorems 1 to n-1] ( n ) theorem n Sometimes the parenthetical theorem numbers or obvious explanations may be omitted. The symbol iff will be used for if-and-only-if. The symbol 3 will be used for there exists. The symbol V will be used for for all. 10 Induction Proofs frequently rely on induction. Usually this involves proving some property P that has a natural number parameter, say n. To prove that P holds for all n ε No, it suffices to proceed by induction on n, organizing the proof as follows: Induction basis: Prove that P holds for n=0. Induction step: Prove that P holds for n+1 from the induction hypothesis that P holds for n. The induction principle for natural numbers is based on the fact that the natural numbers can be constructed by beginning with the number 0 and repeatedly adding 1. By allowing more general construction methods, one obtains the principle of structural induction, enabling the use of more than one case at the induction basis and the induction step. 11 Grammars To facilitate parsing as well as defining attributes to be synthesized or inherited, the grammar for DANCE has been written in Larson normal form or LNF. LNF production rules: Non-terminals are enclosed in angle brackets: <module>. A production consists of a single non-terminal followed by ::= and then an expression in LNF describing what can be substituted for the non-terminal. Production expressions have have only three forms: 1) A single terminal. <command> ::= "skip" 2) A pair of non-terminals. <expression> ::= <function_name><arguments> 3) A single nonterminal. <declaration> ::= <package> Literals are strings enclosed by quotation marks, e.g. "MODULE"; the quote character itself is represented by """. For convenience, the language subset that is used for explaining this invention (simple DANCE) will use a more liberal notation. Productions within <> Literals within "" One or More + Zero or more * Choice I Grouping [] Zero or One () <expression> ::= <function_name> "(" <expression> + " ) " <anything_n me> is just an identifier grammatically, but is helpful to know when a "function" name is expected, that must have a fucntion of that name delcared somewhere. 12 Data, States, and Meaning A model is a triple (T, S, «/) containing a set of data types T, a set of domains containing states S, and an interpretation W giving meaning to every symbol used in the language. For now, consider the data type domain, T, to be the integers. A srafe is a function mapping variables to values in T. For a state s in S and a variable X, let W S [[X]] denote X's value in state s. Each k- place function symbol f has an interpretation W s [[f]] which is a function mapping k elements in T to a single value: What this means is that every function symbol (like "plus') has a "meaning" (like W[[plus]]) that is a mapping from zero or more numbers (in this case two) to a single number. W s llplus]] ( (T 2 → T) 1 The line above doesn't say how "plus" works, just that it takes two numbers and returns one. Meanings aren't rigorously defined in this application for common arithmetic functions. Rather than develop axioms for number theory, the meanings of function symbols rely on intuition. Moszkowski uses temporal predicates to affect side effects (cause state change). This may cause confusion since for correctness proofs we need assertions made with first-order predicate calculus. Keep the distinction clear between the text that represents executable program (temporal predicates) and text used to assert some property of state variables (assertions). Interpretations of predicate symbols are similar but map to truth values: W„[p]] [ (T k → {true, false}) Commands in the DANCE language are temporal predicates. Program execution tries to construct a model that makes all the temporal predicates "true." A model is a lattice of states that satisfies the temporal formula represented by program text. OBJECTS AND ADVANTAGES Several objects and advantages of my invention, the model of computation devised for the Definitive, Axiomatic Notation for Concurrent Execution (DANCE) computer programming language are: Execution of many parts of the same program simultaneously, such that, • hundreds, thousands, or more processors may be efficiently used; • program correctness may be proved; • special data structures may be created that allow many elements within the data structure to be simultaneously accessed without locking, blocking, or semaphores; • the meaning of every language construct is precisely and mathematically defined; and • during program execution, processors may simultaneously self-schedule or post work for other processors to do. Expanding and explaining the objects and advantages: "Execution of many parts of the same program simultaneously" The fundamental concern of this invention is reducing the time of program execution by using many processors (i.e. speedup). Existing computer architectures and computational models underlying parallel programming languages encounter fundamental, theoretical limitations of speedup. I invented the "Layered" class of multistage interconnection networks to permit construction of parallel computers comprising thousands of processors connected with a Layered network to create a fundamentally different computer architecture I called "Multitude." Unfortunately, the models of computation embodied explicitly or implicitly in programming languages adapted for parallel computers (e.g. Ada) would not allow full exploitation of the Multitude architecture computer's power and efficiency. Therefore, I invented a new computational model that allows a programmer to describe, and the machine to utilize, maximal opportunities for simultaneous execution of many parts of the same program. I endeavored to create an embodiment, the DANCE language, whose form is familiar (patterned after Ada) but whose semantics (i.e. meaning or computational model) are radically different. "hundreds, thousands, or more processors may be efficiently used" Techniques that allow a dozen or two processors to cooperate effectively generally do not work for hundreds or thousands of processors. That is, the techniques don't "scale." Efficiency is of utmost importance; keeping the processors busy (almost) always doing something useful economically employs expensive equipment. Machines that use 512 processors to deliver speedup around 25 have not been, and will not be, purchased by savvy, private-sector users. Speedup greater than one thousand for a broad range of applications requires all aspects of an application to scale from description in an inherently-parallel language down to the topology of the network physically interconnecting the processors. "program correctness may be proved" Poor quality software is endemic today. No other product's flaws are so tolerated by society. Buggy programs inevitably result from the "group hack" methodology: code-test-fix-test-fix-release-get bug reports from users-fix-test-release-get more bug reports-etc.-etc.-etc. Testing can only show the presence of errors, not their absence. Only when a proof that a program meets its specification has been exhibited and confirmed can we be sure that a program does what we say it does and nothing more. Correctness proofs are even more important for parallel programs; debuggers that rely on a single locus of control to step through program executions cannot be extended for debugging of parallel programs. Furthermore, parallel programs must assure noninterference— one part of the execution doesn't adversely modify the values of variables needed by another part of the execution. This invention is precise, mathematical semantics for parallel programming languages which necessarily permits formal correctness proofs. "special data structures may be created that allow many elements within the data structure to be simultaneously accessed without locking, blocking, or semaph ores " Achieving efficiency requires existing applications to be reprogrammed using data structures that can be accessed in parallel. Constructs that ensure exclusive use of some object by forcing other users to wait inevitably impede execution thus destroying efficiency. Many such constructs have been proposed or tried: critical sections, monitors, semaphores, locks, spin waiting, snoopy caches, message passing, etc. This invention incorporates semantics for combinable operations in which many or all of the processors can access the same variable at the same clock cycle yielding results as if the accesses occurred one-at-a-time. Provision by Layered networks of combinable operations for Multitude architecture computers is crucial for efficient program execution. Automatic conversion of existing, sequential programs for parallel execution is futile. Efficient parallel execution demands data structures, and their method of access, be radically different from their sequential counterparts. "the meaning of every language construct is precisely and mathematically defined" The semantics of most programming languages are defined in a "natural" language, usually English. Therefore a program compiled by different compilers for the same machine may not behave the same, though both compilers are "correct." In reality the compiler defines language semantics; compiler bugs are language features. Only when language and program semantics are precisely (formally, mathematically) defined can an implementation be deemed "correct." Furthermore, proof of program correctness demands formal language semantics. "during program execution, processors may simultaneously self- schedule or post work for other processors to do" When an idle processor can fetch its own work to do simultaneously with many other processors, processors can be kept busy doing useful work. Self-scheduling is essential for good efficiency. In addition, anonymous processor scheduling greatly facilitates fault-tolerance; permanent failure of a processor, switch, or memory requires checkpoint/restart of the program and on-the-fly hardware reconfiguration to allow graceful degradation. Clever, but brittle, mapping schemes of applications onto architectures generally implode with the loss of a single processor or link. Additional objects and advantages such as elimination of cache coherency, concurrent memory allocation, cache write-back, and others will become apparent from the specification and drawings. DESCRIPTION OF DRAWINGS Figure 1 Sequential Execution: The graph representing sequential execution has one edge from every node except the final state, and one edge to every node except the initial state; the final state is reachable from any state in the graph and any node is reachable from the initial state. Figure 2 Parallel Execution: Traditional parallel execution is merely several (or many) sequential executions. Figure 3 Barrier Synchronization: Barrier synchronization ensures that every process has completed a particular computation before any process may start its next computation. Figure 4 A Simple Execution: The values of b and a may be computed concurrently. Figure 5 Sequential Intervals: 
, <sttus>, and <ttt>. Sequential intervals may be represented with either text or a graph. Figure 6 Simple State Lattice: Every execution of a DANCE program forms a lattice of states. Figure 7 An Arbitrary Interval: The interval, i, has multiple, concurrent computations. Figure 8 Sequential Composition: Two intervals are composed sequentially by fusing the "end" of one interval with the "start" of another. Figure 9 Simple Parallel Composition: Two intervals may be composed in parallel by fusing their "start" states together and fusing their "end" states together. Figure 10 Universal Quantification: Universal, temporal quantification creates many concurrent computations, each with a different value for the quantified variable, in this case X. Figure 11 Assignment Interval: An assignment interval evaluates an expression using the values contained in its "start" node; the end node has the new value as the state of the assigned variable, in this case X. Figure 12 Concurrently accessible queue: Using fetchadd to determine the array index for either get or put allows many intervals to access the queue concurrently. Figure 13 Inserting records into a linked list: Two records can be inserted into the same place at the same time using swap. Figure 14 Linked-List Queue: Using swap to manipulate pointers allows concurrent access to a dynamically-sized queue. Figure 15 Initial graph is a single node: The "first" node contains the values for variables at the beginning of the program, before any computation occurs. Figure 16 Initial state of procedure "sort": Because the actual parameter names (what the caller calls the variables) and the formal parameter names (used within a procedure) usually differ, we show two nodes rather than just one. Since there is no state change, the two nodes can be collapsed into one. Figure 17 Interval to satisfy "sort": Four subintervals are inserted into the lattice for "sort" to first partiton the data (51), followed by sorts on the upper (54) and lower (57) halves, concurrent with storing the pivot (60). Figure 18 Interval to satisfy "partition" Figure 19 Concurrent intervals satisfying the alternative command within "partition" Figure 20 Sorting of Swedish Bikini Team ( 1 of 3) Figure 21 Sorting of Swedish Bikini Team (2 of 3) Figure 22 Sorting of Swedish Bikini Team (3 of 3) Figure 23 Multitude architecture connects processors with a Layered network Figure 24 Processing node in a Multitude architecture computer Figure 25 Physical address comprised of node number and memory address Figure 26 Memory of each node partitioned into global, local, and spread. Figure 27 Main memory words are 84 bits wide Figure 28 Translation of segmented address to effective virtual address Figure 29 Translation of effective virtual address to physical addresses Figure 30 Free-space list for blocks of 2 Λ n bytes Figure 31 Segment capability bit assignment Figure 32 Page descriptor fields Figure 33 Static Linking with link record Figure 34 Lattice of states satisfying a DANCE temporal formula Figure 35 Hidden Registers Figure 36 MMU translates segmented address to physical address Figure 37 UST defines Segmented-to-EVA translation Figure 38 TLB hit address translation Figure 39 TLB hit address translation with page descriptor Figure 40 Cache line holding four words needs 335 bits Figure 41 Cache hits determined by encoding virtual address to select cache lines, and then compare that user, segment, and offset match. Figure 42 Nesting of domains allows proof of noninterference. Figure 43 Count faces. ISA Table An instruction-set architecture table for Multitude architecture processors describes the functionality of every instruction and defines it in terms of temporal logic. LIST OF REFERENCE NUMERALS SUMMARY Efficient, massively-parallel computing requires work scheduling that keeps every processor busy almost all the time. The dismal 3% to 5% efficiency found by the Army High-Performance Computing Research Center at the University of Minnesota (AHPCRC) across a broad range of applications results from inherent limitations in currently-available, massively-parallel computers. This is not for lack of trying. While employed by Unisys and its precursor Sperry-Univac, I invented a new class of multistage interconnection networks to provide rapid, voluminous communication between RISC processors [U.S. patent 4,833,468 LAYERED NETWORK, Brian R. Larson, Donald B. Bennett, and Steven A. Murphy, May 23, 1989]. The shared-memory, multiprocessor architecture based on my "Layered" class of networks was dubbed Multitude. Although Multitude nicely supports Ada [ANSI/MIL-STD-1815A] tasking and rendezvous are both difficult to use and retain an inherent sequential paradigm. Therefore, I tried to define a new language, Samantha, that would allow programmers to define applications without circumscribing opportunities for parallel execution. Negotiations with Unisys management for title to Samantha to use as thesis material in exchange for a license foundered under the onerous demands of a Unisys technocrat. Leaving all trade secret material safely locked in Unisys filing cabinets, I started afresh with DANCE. The objective of efficient computing with 1000s of processors has daunted computer architects for two decades. Achieving the objective requires major changes in the operation of virtually every aspect of hardware and software, with the important exception of fabrication technology. Sub- micron CMOS from any of a dozen silicon foundries will do fine. The trick to making a faster machine with slower parts is to buy more of those parts, for less money, and then actually use a much bigger fraction of the parts doing something useful. Additionally, as fabrication technology improves, it can readily be incorporated without changing the architecture or the software. This invention relies on my earlier invention of [U.S. patent 4,833,468 LAYERED NETWORK, Brian R. Larson, Donald B. Bennett, and Steven A. Murphy, May 23, 1989]. The architectural innovations for the Multitude architecture and the new mathematical systems I devised for the computational model of DANCE comprise the rest of the means to achieve efficient parallel computing. To be clear, I am not claiming the mathematics per se, but a huge class of computations— actual transformations of data for paying customers— performed using some or all of the innovations in Multitude and DANCE. The sequential paradigm is so powerful that it is difficult to think otherwise. It is natural to think of parallel computing as a bunch of sequential execution streams. Sometimes they cooperate by exchanging messages; sometimes they cooperate by sharing state variables; sometimes they wait until all other processors have finished a phase before proceeding to the next phase. Sequential languages have been augmented with means to fork off nearly identical copies of themselves, with hints how to use the data in parallel, and with libraries of pre-compiled procedures for message passing or shared-memory access. In all of these cases the paradigm is still replicated sequential control flow. Virtually all of computer science theory deals with sequentiality or replicated sequentiality. Huge tomes of algorithms have been published and studied that are either sequential control flows or replicated sequential control flows. Regular expressions are recognized by finite-state machines; programs are parsed using a push-down automaton coupled with a handle-recognizing finite-state machine. The compiler-construction a handle-recognizing finite-state machine. The compiler-construction tool, yacc, known and loathed by computer science students, can only parse a subset of context-free grammars: LALR( 1). The very definition of the LALR( l) class of languages has sequentiality built into it— those languages which can be parsed by push-down automata coupled with a handle- recognizing finite-state machines. The pull of sequentiality is so strong, many people will not be able set it aside to consider a different way of computing. DESCRIPTION OF INVENTION This section describes my invention. The operation of the invention is explained in a later section. A mathematical model of computation is presented with the salient advantage that the computation may be performed by hundreds or thousands of microprocessors simultaneously and efficiently. Five mathematical systems interrelate in special ways. The five systems are: temporal logic, domains, type theory, assertions and proofs. Representing programs as special temporal logic formulae that are satisfied (executed) by lattices of state transitions form the core of this invention and provide the simultaneity and efficiency desired. Domains contain variable-value bindings that restrict visibility of variables allowing necessary proofs of non-interference, but without the locks, semaphores, or messages that hobble the prior art attempts. The f pe theory adapts an elegant, lambda- calculus-based, polymorphic, object-oriented type system for the temporal logic execution espoused here. Assertions in first-order predicate calculus define specifications and allowable intermediate states for programs. Since only proofs can assure program correctness, a proof system called BPL associates inference rules and/or weakest-precondition predicate transformers with each language production. A proof of soundness for BPL is presented; a logic is sound if it infers true fact from true facts. 13 Computational Model A model of computation provides a mathematical foundation for the meaning of programs and their execution. Hopcraft and Ullman ■Introduction to Automata Theory. Languages, and Comnutation. Addison-Wesley, 1979] present a series of increasingly complex sequential machines and their models. Hopcraft and Ullman show correspondences between the machines and classes of languages recognized by those machines. State transitions are defined for an input character, the current state, and perhaps some internal data structure such as a stack. All the machines are sequential. Hoare ["Communicating Sequential Processes," C.A.R. Hoare, Communications of the ACM, Vol. 21, No. 8, August 1978] expands the sequential model by allowing multiple sequential machines. Others use models such as dataflow (Petri nets), logic programming (Horn clauses), or functional (lambda calculus) without much greater utility. None of these models of computation adequately expresses parallel execution. Therefore I invented the state-lattice temporal logic model for DANCE. 14 Temporal Logic Moszkowski in his Executing Temporal Logic Programs treats programs as temporal logic formulae. His language, Tempura, allows description oi traditional, process-based, parallel execution within a single program expression. The foundation for Tempura is interval temporal logic. A Tempura interval is a non-empty sequence of states. A state is determined by the values of program variables at a particular instant of time. The sequence of states that form an interval provide a discrete notion of time. The formal model for DANCE presented here differs significantly from the formal model for Tempura: States form a lattice not a sequence. The domains visible at a particular state are both rest ή cted and explicit. Later a sophisticated, polymorphic type system will be substituted for the simple integer types presented here. Again, later a first-order predicate calculus for describing sets of states very similar to those used by Apt/Dijkstra/Gries/Hoare for states in a guarded command language program is presented. A logical system for reasoning about the behavior of programs in terms of interval temporal logic lattices forms another significant, original contribution of this exposition. Each DANCE interval may be thought of as a non-empty lattice of states, partially ordered by their occurrence in time. A state is the values of a set of variables at an instant in time. The set of variables visible by a particular state is its domain. An interval can have many intervening states between any pair of states. An interval may also be split, but the subintervals must have a starting and ending state in common (i.e. every subinterval is itself a lattice, combinations of lattice must form lattices). Intervals are strung end-to-end by sequential composition invoked by the semicolons in the above program. Intervals are strung side-by-side by parallel composition invoked the ampersand. Consider the following DANCE program fragment: a := 10 b := a+a & a := a*a a := b+a First 'a' is assigned the value ten. Then the values of 'b' and 'a' are computed in parallel. Finally 'a' is assigned the sum of 'a' and 'b'. In this paper, interval lattices will be depicted as a set of circles (or stretched circles) connected by undirected edges. However those edges do have direction in that time flows down the page. Figure 4 illustrates the lattice corresponding to the previous example. In state 301 both a and b have no value. State 302 has a assigned 10. States 303 and 304 are independently determined, and may occur simultaneously. State 305 is the synchronization point for the independent computations of 303 and 304. Finally state 306 is computed. The heavy lines between states like 307 are state transitions. The actual computation occurs during state transitions like 307 and are shown as arcs between the nodes holding state. Please note that this model explicitly rejects the notion of some "universal" sequence of states comprised of states of individual processes in "some" sequential order. Such models, although popular, provide poor paradigms for programming parallel processors. Commands in the DANCE language are really temporal predicates. Program execution tries to construct a model that makes all the temporal predicates true. A model is a lattice of states that satisfies the temporal formula represented by program text. The use of the word "model" here refers to creation of a lattice of states that satisfies a particular temporal logic formula embodied in a DANCE program. The "model" of computation for DANCE refers to the general method of constructing lattices of states concurrently. 15 Expressions and Predicates This section starts to use all the mathematical machinery in the Background-Mathematical Foundations section to define the meaning of DANCE programs, which is the model of execution that forms the core of this invention. Obviously an infringer could merely change the grammar and symbols used in the DANCE language, but if the underlying meaning of such languages and programs is based on temporal logic formulae satisfied by lattices of states, then the infringer can expect my rights to this invention to be vigorously defended. Expressions in DANCE are built inductively as follows: Variable names or constants: X, Y, Z, 5, 1 509, 0 Functions: f(e* e^), where f is a function symbol defined by W, ks.0, and ei , ..., ej- are expressions. Moszkowski (and I) take quite a bit of liberty assuming "syntactic sugar" allowing traditional infix representations of X + Y rather than plus(X, Y). DANCE grammar for expressions: <expression> ::= <function_name> "(" <expression> + " ) " <expression> ::= <expression> <function_symbol> <expression> <expression> ::= "(" <expression> ")" <expression> ::= <variable_name> <expression> ::= <literal> The following are expressions: X X+Y (X+Y)/Z somefunction(X Y Z) Formulas (constructions of temporal predicates) are built inductively as follows: Predicates: p(eι , ..., ej , where p is a predicate symbol defined by W, k>= 0, and ei , ..., e^ are expressions. Like functions, we assume a basic set of relations complete with syntactic sugar. Logical connectives: wi & w and wi ; w 2 where wi, and w are predicates. The ampersand corresponds to "and." The semicolon corresponds to "after." One more logical connective (bounded universal quantification) will be added in a following section. Parentheses may be used for grouping expressions. Predicates may be grouped with BEGIN-END. DANCE programs are temporal logic formulas. Execution of a program seeks to construct an interval (e.g. model) that makes the formula true. DANCE grammar for temporal logic formulas: <command> ::= <procedure_name> "(" <expression> + " ) " <command> ::= <command> ";" <command> <command> ::= <command> "&" <command> <command> ::= "BEGIN" <command> "END" The following are examples of formulas: someprocedure(X Y Z) dothis ; beforethat dothis & together BEGIN dothis ; beforethat END & someprocedure(X Y Z) 16 Interval s From the states in S, intervals are constructed from S + , the set of all nonempty, finite lattices of states. A stare is the values assigned to variables at a particular point in time. Figure 4 depicts a lattice of states of which 305 is a state. For example suppose the state variables are X and Y and states s, t , and u are: s __ (χ=ι, γ=2) t = (X=0, Y=3) u = (X=0, Y-6) Sequential intervals can be represented as a string of state names enclosed in o. Some examples are 
 = <(X=l,Y=2)> <sttus> = <(X=1, Y=2) (X-=0, Y=3) (X=0, Y=3) (X=0, Y=6) (X=l, Y=2)> <ttt> = <(X=0, Y=3) (X=0, Y=3) (X=0, Y=3)> Figure 5 shows these sequential intervals in graph form. DANCE expands the definition of an interval to a finite lattice instead of a sequence of states. This provides the mathematical machinery upon which correct, efficient, parallel execution will be based. Figure 6 depicts a simple lattice interval containing states s, t, u, v, w, and x. A textual representation could be: tu / \ <s x> \ / vw but notation good for sequential intervals becomes cumbersome for DANCE lattices so a graphical representation will be used instead. A lattice of states, L. is a graph whose nodes are states, S , and arcs are state transitions, T , that must have the following properties: An irreflexive partial order relation, c L , such that a ~ι b * 3 n>0 : 3 tι,...,t n ε T L : [ ti = (a,sι) and t π = (Sj-.i ,b) and A least upper bound (lub) and greatest lower bound (gib) functions are defined for any pair of states in SL as lub(a,b)=c - a -i c and b c L c and not 3 d ε SL : glb(a,b)=c * c -I a and c r_ L b and not 3 d ε SL : There exists a distinguished states "start" and "end" defined as start(L) - V s ε L : glb(s,start) = start, end(L) ■ V s ε L : lub(s,end) = end. Since time flows downward in interval graphs, the start state (which is the greatest lower bound of every state in the interval) is actually at the top of the graph. Conversely, the end state is at the bottom of the graph. Construction of a lattice state graph is the act of computing. At the end, the values bound to variables are the results. State lattices may be hierarchically structured and composed. Figure 6 is a simple temporal lattice. In this case the lattice, L, is composed of states SL = { s,t,u,v,w,x }, (reference numbers 310, 311, 312, 313, 3 14, and 315 respectively) and state transitions T L = { (s,t), (s,v), (t,u), (v,w), (u,x), (w,x) }, (reference numbers 316, 317, 318, 319, 320, and 321). The partial order relation CL corresponds to the notion of before. State s, 310, occurs before state w, 314, s ~ι w, because state transitions tj=(s,v) 317 and t?=(v,w) 319 are in TL. The transitions form a chain from s 310 to w 3 14. Least upper bound (lub) and greatest lower bound (gib) are defined for all pairs of nodes. For example, the lub of u 312, and v 313 is node x 315. Similarly, the gib of u 312, and v 313 is node s 310. The distinguished states start(L) and end(L) are s 310 and x 315. Being a "partial" order provides opportunities for parallelism. Since neither u c L w nor w c L u, the transitions (t,u) 318, and (v,w) 319 may occur independently, even simultaneously. Good DANCE programs create execution lattices that are short and broad. 17 Temporal Meaning Let 1 denote the set of all possible intervals. Consider an arbitrary interval i c I, composed of a lattice of states with distinguished states start(i) 101 and end(i) 102 with intermediate states 103 and state transitions 307 as depicted in Figure 7. In defining meaning for DANCE language constructs over interval will be denoted by a capital W subscripted with an interval or state designator, the language construct in double brackets, [[ ]], the equality-by-definition symbol, ■, followed by some expression as to the meaning of the construct: Wi er va i[[construct]] - definition 17.1 Meaning of Variables The meaning of variable names is defined by: W;[[X]] ■ W S [[X]] where X is a variable name, X ε s, and s ε Sj. This says "the meaning of X for the interval, i, is defined as the value of X in each state, s, in the interval." If X is not a state variable of s, or s is not a state in the interval i, then X's meaning is undefined. 17.2 Meaning of Functions Where f is a function name and ei, ..., ej- are expressions or variable names, the meaning of function symbol, f, is defined by: Wj[[f(eι , ..., e k )]] _ Wj[[f]] (Wj[[eι]], ..., WitfejJ]) This says "the meaning of an expression in the interval (i) is the meaning of the function symbol (f) applied to the meaning of each of the subexpressions f uses as its operands." DANCE grammar for functions: <expression> ::= <function_name> "(" <express >n> + " ) " <expression> ::= <expression> <binary_function_name> <expression> <declaration> ::= "FUNCTION" <function_name> "(" [<variable_name> + ":" <type>] + ")" "RETURN" <type> ( "IS" <command> ) ";" 17.3 Meaning of Temporal Predicates Where p is a temporal predicate name (i.e. a procedure name) and ei, ..., e k are expressions: rø-[[p(eι- *•*, <*)]] - Wj[[p]] (W_[[e_]], ..., Wi[[e_J_) This says "the meaning of a temporal predicate in the interval (i) is the meaning of the predicate symbol (p) in the interval applied to the meaning of each of the subexpressions p uses as its operands." Simple DANCE grammar for procedures: <command> ::= <procedure_name> "(" <expression> + " ) " <declaration> ::= "PROCEDURE" <procedure_name> "(" [<variable_name> + ":" <type>] + ")" ( "IS" <command> ) ";" 17.4 Temporal Formulas Formulas are compositions of temporal predicates. Such compositions will be covered in the later sections. For now consider that a formula is a simple temporal predicate. A formula w is satisfied by an interval i iff the meaning of w on i is true: Wj[[w]] = true. Execution of a program constructs an interval so that the interval satisfies the formula that expresses the program. If all intervals satisfy a formula that formula is deemed valid. However, valid formulas (tautologies) are rather uninteresting programs. Constructing an interval, a lattice of states, is the execution of the program. The predicates in interval temporal logic become commands in DANCE; they affect side effects. The simplest temporal formula (and the first one to be defined) is "skip." The predicate "skip" is always true. Wj[[SKIP]] _ true. Simple DANCE grammar for skip temporal formula: <command> ::- "SKIP" 17.5 Sequential Composition Moszkowski makes a big deal about his "chop" operator. It is nothing more than sequential composition indicated by semicolon: w- ; 2 where Wj and w are formulas means 3 j,k : jci and kci : (Wj[[wι]] = true) and (W k [[w ]] = true) and (end(j) = start(k) ) This says "the meaning of w- ; is true in the interval i if-and-only-if there exists some subintervals of i, namely j and k, such that the meaning of w- is true during j and the meaning of W 2 is true during k, and k starts at the end of j." Figure 8 depicts sequential composition of intervals. Note that the two subintervals of i share the state end(j) = start(k), and w- is true on the first part of the interval and 2 is true on the last part. Simple DANCE grammar for sequential composition: <command> ::-= <command> ";" <command> 17.6 Parallel Composition Conjunction of formulae provides the parallelism of DANCE: W- & where w- and w 2 are formulas means there exist two subintervals of i, j c i and k c i, such that røj[[w- ]] = true, W k [[w 2 ]] = true, start(j) = start(k) = start(i), and end(j) = end(k) = end(i). This says "the meaning of w* & w 2 over the interval i is true if-and-only-if the meaning of w- is true over some path of states, j, from start(i) to end(i) in the interval i and the meaning of W 2 is concurrently true over some different path, k, of states from start(i) to end(i) in the interval i." Where w- is true on <svwx> and w 2 is true on <stux> as in Figure 9. Therefore wi & w 2 is true over the whole interval. Moszkowski doesn't recognize such bifurcation. Rather he uses the more traditional arbitrary interleaving of states. Though he makes reference to parallel computing, his intervals are always sequences, it's just that several predicates must be true over the same sequence of states. In this example the states v, and w contain variables seen only by w* . Similarly the states t and u contain variables seen only by w 2 . This is the essential, paradigmatic difference in DANCE from other models. Thus the importance of local variables and limited scoping in DANCE provided by domains. Simple DANCE grammar for parallel composition: <command> ::= <command> "&" <command> 17.7 Temporal Existential Quantification For many computations additional, local variables are needed. Temporal Existential quantification allows introduction of such local variables whose scope is limited. The keyword DECLARE (instead of 3) introduces such variables as in: DECLARE X=c begin w end where X is any variable, c is a constant, and w is a formula in which X occurs free. Wj[[DECLARE X=c begin w end]] _ true iff W s t nd ) [[X]] = c, and Wj[[w]] = true. The definition of "free variable" is formally quite hairy. Suffice to say that variable X is free in formula w iff X occurs in w and nowhere else. The scoping rules of DANCE are more permissive however. Even if X is used outside of the existential quantifier, within w all uses of X refer to the local variable. DANCE uses a syntax similar to Ada; the predicate above comes from the Ada production <block>. Existential quantification provides scoping within the model of computation. Variables introduced in the declaration part of blocks, procedures, and packages (implicitly) invoke existential quantification over their scope. Existential quantification readily generalizes to many variables. Later, we will require such existentially quantified variables to have type declarations. Simple DANCE grammar for existential quantification: <command> ::= "DECLARE" [<variable_name> ":" <type> ( "=" <expression>)] + "BEGIN" <command> "END" 17.8 Temporal Bounded Universal Quantification Bounded universal quantification allows a group of formulae to be executed together, but with different parameters. The keyword FORALL is used instead of V. FORALL X IN b DO w where X is any variable, b is a boolean expression or range, and w is a formula means: (DECLARE X=x- begin w end) & (DECLARE X=x 2 begin w end) & ... & (DECLARE X=x π begin w end) where x*, x , ..., x n are values for X that make b true. For now consider b to be a subrange of integers such as 0 *_X<12, which would initiate twelve copies of w each having a local constant X ranging from 0 to 11. Wi[[F0RALL X IN b DO w]] - true iff 3 n : n > 0 : (V x k : l - k n : b x xk ) and (V y :: b x y =_> y ε {x*. x 2 , .... x n }) and (3j l ...jn : jl c i ... jn c i : W j i [[DECLARE X=x- begin w end]] = true, and W j2 [[DECLARE X=x 2 begin w end]] - true, and ... and W jn [[DECLARE X=x n begin w end]] - true) Bounded universal quantification (FORALL) is the source in DANCE for massively parallel processing opportunities; each subinterval can be executed concurrently. Figure 10 depicts universal quantification. All program variables must be introduced with either existential or universal quantification, A range such as [1..10] may substitute for the boolean expression, b, with the understanding that [j..k] means j < X < k; X takes on all the descrete values fron j to k inclusive. Simple DANCE grammar for universal quantification: <command> ::= "FORALL" <variable_name> ":" <type> "IN" [ <range> I <boolean_expression> ] "DO" <command> 17.9 Assignment So far we have no way to modify the values of state variables. Such a language has limited utility! Therefore temporal assignment is needed: X := e where X is a variable name and e is an expression. The value of X at the end of the subinterval becomes the value of e evaluated in the state at the beginning of the subinterval. Formally: Wi[[X:=e]] - Wend<i) [[X_] - Wstartu) -le]]- This says "the meaning of X (its value) at the end of the interval i is the meaning of e evaluated at the beginning of the interval i." Figure 11 shows an interval satisfying an assignment formula. Simple DANCE grammar for assignment: <command> ::= <variable_name> ":=" <expression> 17.10 Alternative Command The alternative command provides data dependent branching. Wi[[IF Bl -> Sl D B2 => S2 D* . *D Bn => Sn FI]] . true iff (Ws t ar t (i)[[Bl]] or W start(i) [[B2]] or ... or W sta r t( i)[[Bn]]) and (Ws t ar t (i)[[Bl]]-*Wi[[S1 ]]) and (Ws t ar . (_) [[B2]]~Wi[[S2]]) and ... and (W st _r t (i)[[Bn]]-»Wi[[Sn]]) Simple DANCE grammar for alternative command: <command> ::= "IF" <boolean_expression> "=>" <command> [ "[]" <boolean_expression> "=>" <command ] "FI" 17.11 Iterative command: The iterative command provides data dependent looping: Wj[[DO Bl => Sl D B2 => S2 Q ... Q Bn => Sn OD]] _ true iff 3 jl.-.jc : jlci ... jcci : ( u jk = i) and k e(i..c] (start(i)=start(j1 ) and endOO=start(j2) and ...and endQc)=end(i) ) and V k : 0<k<«n : ( (Wstart(jk) [[Bl]] - Wj k [[S1 ]]) and (W start(jk ) [[B2]] -* W jk [[S2]]) and ... and (Wstaπ(j k ) [[Bn]] -» W jk [[Sn]]) ) Simple DANCE grammar for iterative command: <command> :.= "DO" <boolean_expression> "=->" <command> [ "[]" <boolean_expression> "=>" <command> ] "OD" 17.1 2 Combinable Operations Combinable operations are crucial to obtaining high efficiency in parallel processing. Combinable operations allow access to a broad range of data structures in parallel without mutual exclusion. The meaning of a fetch¬ add (other combinable operations are similar) is that the value of a shared variable is fetched (read) and its retained value is indivisibly changed to the sum of its previous value and the parameter provided in the fetch-add. If two (or more) processors fetch-add the same variable at the same time, they receive values as if the fetch-adds occur in some sequential order, even though the fetch-adds really occur at the same instant. Previous language constructs were defined over an interval with some unspecified number of intervening states between the initial state and the final state (e.g. assignment). Originally I had not used the "next" (O) temporal operator, but definition of combinable operations demands it. The meaning of the "next" operator over an interval i is for some state sεi, 0( expression) is true if and only if the expression is true in state tεi and no other state occurs between s and t. W s [[θ(e)]] = true iff W t _[-]] = true and glb(s,t)=s and lub(s,t)=t and not (3 uεi : glb(s,u)=s and lub(u,t)=t). A single fetch-add where X is the shared variable, e is the expression to be added (usually an integer constant), and Y is the variable to receive the fetched value is defined as follows: W s [[FETCH_ADD(X,e,Y)]] = true iff røs[[o(Y=X)]] = true and W s [[θ(X-=X+e)]] = true. Multiple combinable operations are more difficult to define. The new value held by the shared variable is straightforward: the new value is the sum (and, or, xor) of the old value and the sum (and, or, xor) of the values contained in the combinable operations. The returned values are more problematic; they are the result of some, nondeterministic ordering as if they occurred one at a time. The Layered network combines requests and decombines responses for combinable operations. Since which processor executes a part of an interval is unspecified, the apparent sequential order cannot be known. Fortunately it doesn't need to be. Let ndperm(eι ,..., e ) be a nondeterministic permutation of expressions resulting in some scrambling of their order: Wj[[ndperm(eι ,..., e k )]] - (eι s ,..., e^) where V j : l<=j<=k : V n : l<=n<=k, n ≠) : not(e m = e π ). Now with ndperm, fetch-add can be defined for simultaneous execution: W7 ik [[FETCH_ADD(X,e* ,Y ) & FETCH_ADD(X,e 2 ,Y 2 ) & ... & FETCH_ADD(X,e k ,Y k )]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., β k s) such that = true and and ... and Wik[_o< Yks= +ei +e2s+ - +€k-is)l] = true. What the above gobbledygook says is that the shared variable X gets all the values contained in the fetch-ads added to it and the values returned are the original X value plus some subset of other values from other fetch-adds. They important is that the behavior is as if the fetch-adds occurred in some sequential order when in fact they occurred at the same instant. Combinable operations are why "interleaving" of states are inappropriate. Any languages or architectures for which interleaving provides an adequate model are doomed to poor efficiency. The fetch-and, fetch-or, and fetch-xor are defined similarly: rø ik [[FETCH_AND(X,e,Y)]] - true iff W ik [[θ(Y=X)]] = true and W ik [[o(X-_X AND e)]] = true. W ik [[FETCH_AND(X,eι ) Yι) & FETCH_AND(X,e 2 ,Y 2 ) & ... & FETCH_AND(X,e k ,Y k )]] - true iff there exists some ndperm(eι ,..., e ) = (eι s ,..., e^) such that W ik [[0(X=X AND ei AND e 2 AND ... AND etJ] = true and Wik[[θ(Yis=X)]] = true and W ik [[o(Y 2s =X AND eι s )]] = true and Wi k [[θ( Y 3s =X AND eis AND e 2s )]] = true and ... and Wi k .[0( Y ks =X AND e ls AND e 2s AND ... AND e k -i s )]]. W/i k [[FETCH_OR(X,e,Y)]] - true iff Wa [[θ(Y=X)]] = true and Wij.[[0(X=X OR e)]] = true. Wi k [[FETCH_OR(X,e- ,Yι) & FETCH_OR(X,e 2 ,Y 2 ) & ... & FETCH_OR(X,e k ,Y k )]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., e^) such that W/ ik [[0(X=X OR ei OR e 2 OR ... OR ej] - true and Wi k _[0(Y_s-X)]] = true and W ik [[0(Y 2s =X OR eι s )]] = true and Wi k [[θ( Y3s=X OR eis OR e 2s )]] = true and ... and Wflj[0( Y ks =X OR ei s OR e 2s OR ... OR e k -ι s )]] = true. rø ik [[FETCH_XOR(X,e,Y)]] - true iff Wi [[0(Y=X)]] = true and W ik [[0(X=X XOR e)]] = true. W ik [[FETCH_XOR(X,eι ,Yι) & FETCH_XOR(X,e 2 ,Y 2 ) & ... & FETCH_XOR(X,e k ,Y k )]] _ true iff there exists some ndperm(eι ,..., e ) = (eι s ,..., e^) such that W ik [[0(X=X XOR ei XOR e 2 XOR ... XOR ej] = true and Wik[[θ(Yι s =X)]] = true and W ik [[θ(Y 2s =XXOReι s )]] = true and W ik [[θ( Y 3s =X XOR ei s XOR e 2s )]] = true and ... and Wik-[0(Y k s=X XOReis XOR e 2s XOR ... XOR e^)]] = true. The swap operation behaves differently. When a single swap occurs at a time, the value of the shared variable becomes the parameter of the swap and the value returned is the previous shared variable value. W ik [[SWAP(X,e,Y)]] - true iff Wi k [[θ(Y=X)]] = true and W ik [[0(X=e)]] = true. When many swaps occur simultaneously, one processor will get the value of the shared variable, the others will get values from some other swap, and the shared variable will get the swap value no processor receives. Wi k [[SWAP(X,eι ,Yι) & SWAP(X,e 2 ,Y 2 ) & ... & SWAP(X,e k ,Y k )]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., ej-s) such that Wi k [[θ(X=e ks ]] = true and Wii-[[0( Y ls =X)]] = true and Wik-[0(Y 2s =eis)]] = true and W ik [[0(Y 3s =e 2s )]] = true and ... and Wa-[[0( Y s =e k -i s )]] = true. The swap combinable operation is good for manipulating pointers in dynamic data structures. The fetch-add is good for indexing into a shared array. The fetch-and, fetch-or, and fetch-xor are good for clearing, setting, and toggling flags respectively. Virtually all of the DANCE programs use combinable operations to manipulate shared variables to access data structures in parallel without mutual exclusion. Hardware support for combinable operations is necessary for high efficiency. In addition to combining in the Layered network, I'd like combinable operations in the processor's ISA and memory management unit. Although combinable operations are expected to represent a small percentage of all instructions executed, they (can) eliminate parallelism-destroying mutual exclusion and blocking. 18 Temporal Domains Temporal domains hold bindings between identifiers (variables) and values. All state variables occur in some domain, and at most one domain. Domains hold not only changeable states, but complete packages which contain procedures and functions as well as state variables. Domains are essential for proving noninterference— that no other processor is modifying the variables any particular processor is currently using. Noninterference is crucial to showing that parallel activities can occur together correctly ["An Axiomatic Proof Technique for Parallel Programs," Susan Owicki and David Gries, Acta Informatica 6. Springer- Verlag 1976]. 18.1 Domain Trees All domains for a single parallel program running on a single Multitude computer (which is comprised of many processors) form a rooted, directed tree. Intervals being executed (causing state change) exist in some leaf domain. The temporal predicates (commands) that control the interval can use any variable-value bindings in any domain below it in the tree. The domain tree will dynamically expand an contract as the program is executed. A new domain is created when a block is entered that declares local variables and is deleted when the block is exited. Domain names are formed by a string of identifiers separated by periods representing the path in the domain tree from the root to the leaf. The local variables immediately declared within a subprogram are contained in a domain whose name is the subprograms identifier prepended (with a period) with the domain name in which it was invoked. Within subprograms, blocks that declare new variables may be encountered; a new domain leaf is grown. Non-nested blocks are assigned numbers starting with 1 , in the order in which they are encountered. Nested blocks grow new domain leaves. Automatically numbered blocks cannot be confused with either named domains or subprogram domains since DANCE identifiers cannot start with digits. However domain identifiers and subprogram identifiers may be the same. However, an identifier's position in the name make clear whether it's a domain name or subprogram name. In the following, d, dl, d2, etc. stand for domains and N(d) stands for the name of domain d. Define the domain operator 2), by: -2>[[d]] - { (n,v) I n ε identifier, v ε Type(n) } which means "the domain of d is a set of pairs in which the first element, n, is a program identifier and the second element, v, is a value in the type of n." Types are sets of values. A domain contains the domains of all its ancestors in the domain tree. Ancestor domain names are formed by truncating their descendent's names: -2>[[dl]] C2>[[d2]] iff 3 s : string I N(dl) && "." && s = N(d2) where && is string concatenation. That is that domain d2 can "see" all the values in every domain dl whose name is a shortened version of d2. All the values visible in domain d2 are either declared in the DANCE program text corresponding to d2, or are declared in some ancestor of d2. Domains are neighbors if they have some variables they both can see: neighbors(dl,d2 ) - 2>[[dl]] D2)[[d2]] ≠ 0. Remote domains have no common variables: remote(dl,d2) - 2>[[dl]] fl2)[[d2]] = 0. Remote domains are needed to represent programs that span machine boundaries, i.e. distributed processing. Simple DANCE program fragment: package body ex:example is declare A:float; procedure p is declare W:integer; begin commandl end; procedure is declare X:float; begin declare Y:bool; begin command2 end » command5 declare Z:char; begin command3 » declare V:string; begin command4 end end end; end package In the above program fragment: command 1 sees W and A; command 2 sees A, X, and Y; command 3 sees A, X, and Z; command 4 sees A, X, Z, and V; command 5 sees X and A. Figure 42 shows the domains for the example above. Figure 42 is not a set diagram in the usual sense of a subset drawn inside its superset. The set of nameable and therefore usable variables for a particular domain are all the variables declared within the domain, union variables declared in surrounding domains. Domains are necessary for proof of noninterference. The easiest way to assure noninterference is by restricting access to critical varaibles. Semaphores protect variables by serializing access to critical sections. DANCE protects variables with domains. By relying on the fact that no other interval can "see" critical, local variables, noninterference is assured. 18.2 Remote Procedure Call Distributed systems have domains that do not form a tree—no single domain exists within which all other domains are contained. No shared address space, no combinable operations, just messages, usually point-to-point sans confirmation. Remote procedure call (RPC) works just fine for much of what one would like to do at disparate geographical locations. (Note: I explicitly reject message passing paradigms for parallel processing that have one machine out of many microprocessors, in a single location. Only geography demands distribution.) However there are worthwhile applications that demand geographical distribution that cannot utilize nested RPCs such as databases. RPCs are great for computation where the duration or lifetime of the domain which contains all accessed variables coincides with the duration of the invoked procedure. Such applications abound, i.e. photorealistic, 3D, animated rendering. Sometimes the lifetime of state variables must exceed the lifetime of any procedure that uses it. Databases are like this. The traditional means of ensuring noninterference (and thus partial correctness) is with locks, semaphores, monitors or some other form of mutual exclusion which may lead to deadlock. This has caused a colorful consideration of resource allocation and deadlock detection starting with Dijkstra's dining philosophers and ending with global snapshots. Total correctness proofs cannot tolerate even the possibility of a deadlock occurring— the program musr terminate! How to ensure noninterference between transactions that may wish to exclusively use the same values without risking deadlock? Must have noninterference, cannot permit deadlock. Noninterference without deadlock. But how? 1) Divide the database into a large number of domains which may reside on machines across the country. 2) Impose a total ordering on the domains. 3) Allow only one RPC to be active in a domain— queue other received RPC invocations first-in-first-out. 4) Make database accesses that need to maintain consistency between values in different domains use an RPC chain in the order of the domains used. The proof by induction of totally correct programs that invoke only other (already proven) totally correct subprograms with RPC is direct. Anything that is totally correct and is invoked with RPC will eventually terminate and return with the right value, and the caller will then terminate (the inductive hypothesis). The chains of RPCs that may occur must be finite so that the deepest invocation will terminate without making further invocations (the basis). An RPC chain is a finite number of nested RPC calls, each of which can be on a different machine. During the invocation phase, values of the database domains of interest are "read." During the return phase, values of the database are "written." Therefore any mapping from previous database values to new values is possible, and provide some initiating transaction the answer to, and confirmation of, its query. The total ordering of domains makes it work. Suppose a single transaction occurs. It will create a chain through the database, complete its last RPC and eventually terminate. If a transaction is stalled because its deepest RPC is stalled in a queue at a domain already in use, it will eventually be given access since the currently active RPC (and all RPCs ahead in the queue) will eventually terminate and release the domain (the inductive hypothesis). There is some "deepest" chain that occupies (has an active RPC chain) in the highest numbered domain. It will eventually terminate and release all its domains therefore possibly unblocking other RPC chains (the basis). Therefore all RPC chains will eventually terminate and total correctness can be proven. This method only works efficiently when the transactions require domains evenly— each processor must know in which domain the value of interest lies. Tree-structured directories save much space, and may work well for sequential accesses, but they cause "hot spots" that would substantially degrade the performance when distributed. On the other hand, if the number of domains were large enough so that the probability of any two transactions needing the same domain were small, the system would exhibit latency largely independent of the number of transactions— a blessing to travel agents worldwide. 19 Type Theory Type systems are important. Most languages such as Pascal and C only allow types that are basic and constructed. Basic types generally correspond to types supported directly by the hardware like integers, floating point number and characters. Constructed types are built from the basic types to form arrays, lists, and records. Some languages such as Ada allow forms of polymorphism (many types) through operator overloading. I hate C. Whenever I wrote a C program I was always mucking with the lowest level of representation whenever 1 used a complex type. Programs such as Lint and interactive debuggers alleviated only slightly my frustration in getting my programs to run properly. A common error of mine occurred when I dereference a pointer only to use it with the wrong type. Run-time error messages of "segmentation fault" are decidedly unhelpful in determining what my error was. I believe that strong, but not necessarily static types, are essential for program development. As a shareholder representative to the Microelectronics and Computer Technology Corp. (MCC), I encountered a new way to think about types. Abstract data typing seemed to answer my desire for powerful, reusable, and encapsulated types. A paper by Cardelli and Wegner ["On Understanding Types, Data Abstraction, and Polymorphism," Luca Cardelli and Peter Wegner, Computing Surveys. Vol. 17, No. 4, December 1985.] gave new vistas to my thinking about types. The type system of DANCE has been adapted from Cardelli and Wegner' s type expression language Fun. Existentially and universally quantified types are realized through "packages" described in a following section. 19.1 Basic Types The basic types of DANCE are: Integer, Bool, Rat, Float, and Char for integer, boolean, rational, floating point, and character. I include rational because exactness cannot be obtained with floating point numbers. A special value, NIL, is a member of each of the basic types representing no value. Uninitialized variables have the value NIL. Attempts to use variables whose values are NIL in expressions will result in error messages. However NIL can be assigned to variables. Simple DANCE grammar for simple types: <type> : "INTEGER" I "BOOL" I "RAT" I "FLOAT" I "CHAR" 19.2 Constructed Types Types may be constructed from basic types by the use of type expressions. Arrays, function spaces, records, and variant types may be constructed. Arrays are integer indexed and behave hke functions as in IThe Science of Programming. David Gries, Springer- Verlag, 1981]. Accessing an array with an index out of bounds results in NIL. Functions are first-class and so may be passed as parameters. Record and variant types have standard usages. Simple DANCE grammar for constructed types: <type> ::= "(" [<identifier>+ I <literal>+] ")" <type> ::= "ARRAY" "[" <range>+ "]" "OF" <type> <type> ::= <type> "=>" <type> <type> ::= "RECORD" <label> ":" <type> ["," <label> ":" <type>] * "END" "RECORD" <type> ::- "VARIANT" <discriminant> : <type> ";" <label> ":" <type> ["," <labei> ":" <type>f "END" "VARIANT" 19.3 Parametric Types Parametric types are type constructors that take type parameters to build a new type. For example, a generic list could be defined and then specialized as with: TYPE List[ltem] IS -declare a generic list VARIANT c : empty, cons; empty : NIL, cons : RECORD head : Item, tail : List[ltem] END RECORD END VARIANT; TYPE Intϋst : List[lnteger]; -bind Integer to Item to get IntLst Simple DANCE grammar for parametric types: <declaration> ::= "TYPE" <type_name> ( "[" <type_name> + "]" ) "IS" <type> ";" <type> ::= <type_name> "[" <type_name> + "]" 19.4 Packages are Abstract Data Types Packages provide program structuring and information hiding. The packages in DANCE are adapted from those in fAlgebraic Specifications in Software Engineering. Ivo Van Horebeek & Johan Lewi, Springer- Verlag, 1 89] and inspired by packages in Ada ΓANSI/MIL-STD-18 ISA- 1983 Reference Manual for the Ada Programming Language. American National Standard Institute, 1983.]. Packages have two parts: a visible, declaration section, and a hidden body. The body is not really hidden or secret, just that it is not within the scope that can be seen by other packages. All that a programmer should need to know about the services of a package should be visible in the declaration section. Those familiar with object oriented languages like Smalltalk will recognize that packages define classes and that these classes are instantiated as objects. Such considerations will be finessed for the time being. The conception of executing a package body is adequate. Packages have names. The items within a package can be named from other packages similar to records by prefacing the name of the item with the package's name and a period. Packages group together objects and operations that have useful purpose. Packages require lots of declarations. C hackers don't want to be bothered with scoping more sophisticated than "local" and "global." DANCE adds some powerful, yet esoteric declarations to Ada's. These declarations allow generic (polymorphic) programs that are parameterized with type (universal type quantification) and certain requirements for the awaited type (inheritance via bounded universal quantification). Ada's private type declarations serve as existential type quantification. DANCE'S type system (including pointers or access types) allows definition of every type in either Ada or Fun, plus exotic types not found in either. A requirement clause demands passing of a type to the package. Often this type will be restricted to some form. Cardelli and Wegner's bounded universal quantification is realized through the use of requirement clauses. An import clause specifies the use of another package's services. Such packages must be visible at compile time. The private type declaration declares types that are defined within the package but may be used by other packages. The implementation of the types (and operations using such types) are hidden within the body of the package. This corresponds to Cardelli & Wegner's existential quantification. The private type states that a type exists, but does not tell how it works. Other packages may only manipulate existentially quantified types with operations declared by the package. The package body contains the implementations of the operations. The relationships between each section will become clearer with later examples. Simple DANCE grammar for packages: <declaratk>n> ::= <package> I <package_body> <package> ::= "PACKAGE" <package_name> ( "[" <name>+ "]" ) "IS" # <requirement_clause> <import_clause> <declaration> + ( "PRIVATE" <declaration> + ) "END" "PACKAGE" <package_body> ::= "PACKAGE BODY" <body_name> ":" <package_name> "IS" <declaration> + "END" "PACKAGE" <declaration> :- "PROCEDURE" <procedure_name> "(" [<variable_name> + ":" <type>] + ")" ( "IS" <command> ) ";" <declaration> ::- "FUNCTION" <function_name> "(" [<variable_name> + ":" <type>] + ")" "RETURN" <type> ( "IS" <command> ) ";" <import_clause> ::= "IMPORT" [ "ALL" I <name> + ] "FROM" (<body_name> ":") <package_name> ( "[" [ formal-parameter "=>" actuaLparameter ] + "]" ) "END" "IMPORT" 19.5 Polymorphism There are two major kinds of universal polymorphism, that is, two major ways in which a value can have many types. In parametric polymorphism, a polymorphic package has an explicit type parameter. In inclusion polymorphism, a package can be viewed as belonging to many different types, which need not be disjoint. So far, I haven't imposed limitations on overloading to avoid ad hoc polymorphism; for example, a programer may define a new * +' function on a new type that is not associative. Types are set of values. The universe of all values, V, contains all simple values like integers and strings, and all data structures like pairs, records, variants, and records. A type is a set of elements of V. Moreover, V when ordered by set inclusion forms a lattice (of types not states). The top of this lattice is the type Top, the set of all values or V itself. The bottom of this lattice is, essentially, the empty set. The types used by any given programming language is only a small subset of V. A particular type system is a collection of sets (ideals) of V, which is usually defined by presenting a language of type expressions and a mapping from type expressions to sets of values. Since types are sets, subtypes are subsets. Moreover, the semantic assertion "Tl is a subtype of T2" corresponds to the mathematical condition T1CT2 in V. This is the foundation for type checking. For simple types (like integer subranges) this subtype relationship is intuitive. But Cardelli and Wegner add types such as: type B - VT.T=>T This means that type B is all functions whose result type is the same as it operand type. Not just some function, but all such functions-universal quantification. For example, the trigonometric function cosine has floating-point valued operand and result: type float_functions = float= f loat cosine(x) ε float_.functions float_functions B Similarly for existential quantification, type C = 3c.t(c) means, for some type c, C has type t(c) where t(c) is a type expression using c. Every value has type 3a.a because for every value there exists some type (set) for which the value is a member. Thus the type 3a.a denotes the set of all values or Top. Universal quantification yeilds generic types, whereas existential quantification yeilds abstract data types. Combining these two notions with subtyping yeilds parametric data abstractions. A package declaration, package p[t] is required t : r end required d —declarations end package creates a type of the form type p =* v tc r : 3b:d(b) Which says that package p is a type, such that for any type t which is a subtype of r, there exists some body, b, that implements the declaration s), d. There may be more than one body b that upholds the declaration(s), d, and the package will operate with any supplied package t, so long as it behaves like r. 19.6 Type Checking Types have been called "armor" protecting data representations from inappropriate manipulation. By requiring explicit declaration of all program variables, the DANCE language supplies redundant information used for checking types. In general, the type of a thing must match the use of that thing. Using separate information to determine thing-type and use- type, allows them to be compared. Like balancing a double-entry bookeeping system, type checking gives confidence. Type checking involves (somehow) determining types of an object and its use. That determination is called type inferencing. These type inference rules are adapted from Cardelli and Wegner. The DANCE type system can be formalized as a set of type inference rules that prescribe how to establish the type of an expression form the type of its subexpressions. These rules can be intended as the specification of a type-checking algorithm. The inference rules are given in two groups. The first group is for deducing that two types are in the inclusion relation. The second group is for deducing that an expression has a type. Type expressions are denoted by s, t, and u, type variables by a and b, type constants (e.g. float) by k, expressions by e and f, variables by x, and labels by L. 19.7 Type Inclusion Rules: t£β C is a set of inclusion constraints for type variables. C.a : is the set C extended with the constraint that the type variable a is a subtype of the type t. C t__s is an assertion meaning that from C we can infer t__s. A horizontal bar is logical implication; if we can infer what is above it, then we can infer what is below it. [TOP] C - t__Top [VAR] C.a_-t r. a t [BAS1] C - a__a [BAS2] C - k__k [ARROW] C i- s t £s and C - tQ* C *- s— t s-→t* (RECD] C * s- -t! and . . . and C *- s-Ω n C - RECORD L* :Si L n :s n , ... , L-,:.-- END RECORD RECORD L* is* L n :s n END RECORD [VART] C - siQ- and . . . and C - _n_-t n C - VARIANT L:(L, ... _<-); L-:s* L^s*, END VARIANT Q VARIANT L:(L* ... L-): -s* L n :sn L m :_m END VARIANT [U i] c.3__s - tQ* a not free in C [EXI] C.a__s - t * a not free in C C - (3___s.t) (3a_-S.t 1 ) [TRANS] C - s-≡t and C - u__u C *=s 19.8 Type Rules for Expressions: e A is a set of type assumptions for free program variables. A.x:t is the set A extended with the assumption that variable x has type t. C.A e:t means that from the set of constraints C and the set of assumptions A we can infer that e has type t. /is a function. t{sla) is texual substitutuion a [ETTOP] C.A - e:Top [EVAR] C.A.x:t - x:t [ABS] f- e and C.A.x:s *- e:t ree in e C,A ~/:s-*t [APPL] C.A - 8**t and C.A - Θ:S C,A -fl ~ )Λ (simple DANCE needs not RECD. record components accessed individually: r.L ) [ERECD] C,A *- e ' Q- and . . . and C,A ■- e-.Ω n C,A .- (Lιa>βι L n =e n ) : RECORD L-rt, L n :t- END RECORD [SEL] C.A - x : RECORD L, :t* L n :t n END RECORD C,A ■ - x.L) : t f i ε l.n [VART] C,A - x : VARIANT L:(L, ... -); L, :s 1 L*,:s n END VARIANT C.A - Θ : (Li ... Ln) [GEN] C.a__s,A - e.1 PACKAGE P[a] IS REQUIRED a:s END REQUIRED e END PACKAGE C.A - P : Va__s.t [SPEC] C,A - P : VaΩs.t and C - s*£s C,A - P.e[sι] : t{Sιla> IDEFN] C.A - e: t{slb} TYPE a[bj IS t; C.A «- e : a[s] [ETRANS] C.A - e : t and C - t£u C,A - e : u [HIDE] C,A - a : t PACKAGE P IS TYPE a IS PRIVATE; s END PACKAGE C,A - P : 3a£t .s [USE] C.A - P : Va£s.3tQ .u C.A - c : s IMPORT u FROM p : P[c=>a] END IMPORT PACKAGE BODY p:P IS TYPE b IS 1 1 ; C,A - p.u : P{cla} 20 Correctness Proofs My attendance at the Institute on Formal Development of Programs and Proofs sponsored by the University of Texas at Austin in 1987 changed my attitude toward correctness proofs. I am now completely convinced of the necessity of correctness proofs. Testing can at best show the presence of program errors. Only proofs can show the absence of program errors. The problems of debugging are greatly magnified by parallelism. Since no single thread of control exists, two executions of a parallel program may not yield the same results! Experiments showed that a particular program would execute correctly over 99% of the time, but would occasionally seize up in deadlock. However, when run in debug mode to generate trace files to find the bug, the program never deadlocked because generation of trace files changed the timing enough to avoid the problem. Programs for parallel computers simply cannot be developed by C hackers! Therefore the necessity for program proofs. I could plagiarize David Gries' excellent book, but will refer the reader instead fThe Science of Programming. David Gries, Springer- Verlag, 1981]. In Gries' book, weakest precondition predicate transformers refer to a particular place in the text of the program. In DANCE, the assertions are applied to the states of program variables denoted by interval temporal logic almost always referring to the first or last node of a state lattice. Such states are "between" execution of commands. Assertions in DANCE are enclosed in curly braces {}. Assertions form pre- and post-conditions for commands. This allows proof of program correctness. The form of DANCE assertions was adapted from fConcurrent Programming; Principle, and Pra ricp* Gregory R. Andrews, Benjamin/Cummings, 1991]. Andrews' notation is based on guarded commands like Gries, but from a parallel computing emphasis. For example an array, b, with length k can be asserted to be in order as: {Sorted(b) : forall i : l..k-l : b[i] <= b[i +1] } An assertion applies to the state of visible variables at the interval(s) when the following command is ready to execute. Andrews views assertions as part of the text of a program. DANCE shifts the view to the values of variables at a particular instant of time (i.e. state). If one thinks of a parallel program represented by interval temporal logic as a lattice made from threads of state transitions, seeing opportunities for parallel execution becomes easier. Each thread has (for the most part) its own private variables that can be manipulated and proven to not affect other threads. Perhaps the word "thread" may be confused with popular "multi- threaded" operating systems such as Mach. Such threads are merely sequential processes. Arbitrary interleaving of sequential process states is a paradigm that clouds parallel conception and requires inspection of the states of all processes, rather than just the variables whose values they share. Two similar concepts must be kept clear: temporal predicates and assertional predicates. Temporal predicates serve as commands that create a model, in interval temporal logic, making the predicates true over the interval. Assertional predicates have no concept of time, only state. Assertional predicates aren't actually executed (but checking assertions during debugging would be nice). Rather they declare allowable values for state variables and are used for specifying what to do and proving it was done. 20.1 Weakest Precondition Predicate Transformers Any formal logical system consists of rules defined in terms of: a set of symbols, a set of formulas constructed from these symbols, a set of distinguished formulas called axioms, and a set of inference rules. Formulas are well- formed sequences of symbols (they are grammatically correct). The axioms are special formula that are assumed to be true. Inference rules specify how to derive additional true formulas from axioms and other true formulas. Inference rules have the form: Hι , H 2 H n Where the H; are hypotheses, and C is a conclusion. Both the hypotheses and conclusion are formulas. The meaning of an inference rule is that if all the hypotheses are true, then we can infer that the conclusion is also true. A proof in a formal logic system is a sequence of lines, each of which is an axiom or can be derived from previous lines by application of an inference rule. A theorem is any line in a proof. Thus, theorems are either axioms or are obtained by applying an inference rule to other theorems. By itself, a formal logical system is a mathematical abstraction-a collection of symbols and relations between them. A logical system becomes interesting when the formulas represent statements about some domain of discourse and the formulas are theorems that are true. This requires that we provide an interpretation to the formulas. An interpretation of a logical system maps each formula to true (T) or false (F). A logic is sound with respect to an interpretation if all its axioms and inference rules are sound. An axiom is sound if it maps to true; an inference rule is sound if its conclusion maps to true, assuming all the hypotheses are true. Thus, if a logical system is sound, all theorems in the logic are true statements about the domain of discourse. In this case, the interpretation is called a model for the logic. Earlier we talked about "models" of interval temporal logic. Those models were interpretations that assigned values to state variables to make the temporal predicates true. Generating a model for a temporal predicate is executing a DANCE program. (Hey this is pretty goofy stuff and the mathematical terminology overlaps causing confusion. Sorry, but that's the way it is.) This section defines assertions which are really outside the program, but allow proof that the programs commands (imperative directives, temporal predicates) do what we claim it does. A logical system is complete with respect to an interpretation if every formula that is mapped to true is a theorem; i.e. every true formula can be proven with the logical system. Unfortunately, parallel programs cannot have both sound and complete axiomatizations as a logical system. This is because program behavior must include arithmetic, and a well-known result in number theory, Gόdel's incompleteness theorem, states that no formal logical system that axiomatizes arithmetic can be complete. However, a logic that extends another logic can be relatively complete, meaning that it introduces no additional source of incompleteness beyond that already in the logic it extends. Fortunately, relative completeness is good enough since the arithmetic properties that we will employ are certainly true, even if not all of them can be proven formally. 20.2 Propositions Propositional logic is an instance of a formal logical system that formalizes what is often called "common sense" reasoning. The formulas of the logic are called propositions. The axioms are special propositions assumed to be true. The inference rules allow new, true propositions to be formed from existing ones. In propositional logic the symbols are: Propositional constants: T and F (for "true" and "false") Propositional variables: identifiers (p, q, r, ...) Propositional operators: NOT, AND, OR, «=>, and = The formulas of propositional logic can be single constant or variable symbols, or constructed by using propositional operators to combine two other formulas. There are five propositional operators: negation (NOT), conjunction (AND), disjunction (OR), implication («->), and equivalence (=). Their interpretations are given below. C fl NOT noAND α p QR α Q_ qp = q F F T F F F T T F T T F T F F F T F F T T T T T T T Given a state s, we interpret a propositional formula P as follows. First, replace each propositional variable in P by its value in s. Then use the table to simplify the result. When there are multiple operators in a formula, negation has the highest precedence, followed by conjunction, disjunction, implication and finally equivalence. As with arithmetic expressions, parentheses may be used to force a different order of evaluation or to make the formula clearer. For example if p is false and q is true in a state, then the interpretation of the formula p OR (p =*- q) p OR (p => q) - F OR (F -» T) = F OR T = T A formula P is satisfied in a state if it is true in that state. Formula P is saύsfiable if there is some state in which it is satisfied. Formula P is valid if it is satisfiable in every state. A valid proposition is also called a tautology. In a sound propositional logic, an axiom is a tautology since it is assumed to be valid. There are many tautologies in propositional logic, just as there are many identities in trigonometry. The ones that we will use most are listed in the table below. These tautologies are called propositional equivalence laws since they allow a proposition to be replaced by an equivalent one. Applications of the propositional equivalence laws are transitive—in any given state, if P=0_and Q=..R, then P=-R. Law of Negation: P = NOT(NOT(P)) Law of Excluded Middle: P OR NOT(P) = T Law of Contradiction: P AND NOT(P) - F Law of Implication: P => Q_= NOT(P) OR Law of Equality: (P=Q) = (P =*Q) OR (Q.=*P) Laws of OR-simplification: P OR P = P PORT = T P OR F = P P OR (P AND Q) - P Laws of AND-simplification: P AND P = P P AND T = P P AND F = F P AND (P OR Q) = P Commutative Laws: (PAND Q) = (Q,AND P) (P OR Q) = (Q.OR P) (P = 0) = (Q_- P) Associative Laws: P AND (Q.AND R) = (P AND Q) AND R P OR (Q.OR R) = (P OR Q) OR R Distributive Laws: P OR (Q.AND R) = (P OR Q) AND (P OR R) P AND (Q.OR R) = (P AND Q) OR (P AND R) DeMorgan's Laws: NOT(P AND Q) = NOT(P) OR NOT(Q) NOT(P OR Q) = NOT(P) AND NOT(Q) AND-elimination: (P AND Q) -» P OR-introduction: P => (P OR Q) Both AND-elimination and OR-introduction are examples of weakening a proposition. For example if a state satisfies (makes true) P AND Q_ then it satisfies P alone. Thus, P is a weaker proposition than P AND 0_ since (in general) mare states satisfy just P than satisfy both P and Q, Similarly, in the OR-introduction tautology, P OR Q.is a weaker proposition than P. Since F implies anything, F is the strongest proposition (no state satisfies it), and T is the weakest proposition (every state satisfies it). 20.3 Predicates Propositional logic provides the basis for assertional reasoning. However, by itself it is too restrictive since the only propositional constants are T and F. DANCE has additional data types besides just boolean. Thus, we need a way to manipulate any kind of boolean-valued expression. Predicate calculus extends propositional logic to support this generalization. The extensions to propositional logic are: Any expression that evaluates to T or F can be used in place of a propositional variable; or Existential (EXISTS or 3) and universal (FORALL or V) quantifiers are provided to characterize sets of values. This is the same reserved word (FORALL) as the parallel command, but within a predicate it is. different! Since 3 and V are not ASCII characters, DANCE programs use instead. 3 and V are used in mathematical definitions where applicable. The symbols of predicate calculus formulas are those of propositional logic plus relational operators and quantifiers. The formulas, called predicates, are propositional formulas in which relational and quantified expressions may replace propositional variables. To interpret a predicate in some state, we first interpret each relational and quantified expression, and then interpret the resulting propositional formula. When dealing with sets of values, we will often want to assert that some or all values of a set satisfy a property. We could do this informally, but quantified expressions provide a more precise and compact method. The existential quantifier, EXISTS, allows one to assert that some element of a set satisfies a property. It appears in expressions of the form: (EXISTS b- , ..., b n : R : P ) The bi following the quantifier are new variables called bound variables; R is a formula that specifies the set of values (the range) of the bound variables; P is a predicate. The interpretation of existential quantification is true if for some combination of values of the bound variables (within range R), predicate P is true. The universal quantifier, FORALL, allows one to assert that all elements of a set satisfy a property. Universal quantification takes the form: (FORALL bi , ..., b n : R : P) Again, the bj are bound variables, R specifies the range of values of bound variables, and P is a predicate. The interpretation of universal quantification is true if P is true for __ϋ combinations of the values of the bound variables. In a quantified expression, the scope of a bound variable is the expression itself. Thus, a quantified expression is like a nested block. Many quantified expressions will also reference program variables. Thus, confusion could result if the name of a bound variable conflicts with that of another variable, and should be avoided. Laws for manipulation of quantified expressions: DeMorgan's Laws for Quantifiers: (EXISTS B : R : P) = NOT(FORALL B : R : NOT(P)) (FORALL B : R : P) = NOT(EXlSTS B : R : NOT(P)) Conjunction Law: (FORALL B : R : P AND Q) = (FORALL B : R : P) AND (FORALL B : R : Q) Disjunction Law: (EXISTS B : R : P OR Q) = (EXISTS B : R : P) OR (EXISTS B : R : ) Empty Range Laws: (FORALL B : F : P) = T (EXISTS B : F : P) = F (NUM B : F : P) = 0 (SUM B : F : P) = 0 (PROD B : F : P) - 1 The counting quantifier, NUM, adding quantifier, SUM or ~~ , and multiplying quantifier, PROD or II* can be helpful in describing properties of groups of elements. Textual substitution is the final concept from predicate calculus employed: Textual Substitution: If no variable in expression e has the same name as any bound variable in predicate P, then Px e is defined to be the result of replacing every free occurrence of x with e. Textual substitution is used for the assignment command. 21 Brian' s Programming Logic A programming logic is a formal system that facilitates making precise, mathematical, statements about program behavior. Brian's Programming Logic, BPL, derives from Andrews' PL, suitably modified for the DANCE language and its model of computation. As with any formal logical system, BPL contains symbols, formulas, axioms, and inference rules. The symbols of BPL are predicates, curly braces, and DANCE programming language commands. The formulas of BPL are triples of the form: Above, P and Q.are predicates; C is a simple or compound command. P and Q . are assertions about the behavior of C. In DANCE, C is a temporal predicate which is made true by construction of an interval. An interval (or subinterval) is a lattice states. The assertion P applies to the first state in the interval; Q. applies to the last state. In P and Q, the free variables ( those not bound by a quantifier) are either state variables or logical variables. State variables are introduced explicitly by declarations and are manipulated by execution of DANCE commands. Logical variables are special variables that serve as placeholders for arbitrary values; they appear only in predicates (between braces), not in DANCE commands. Interpretation of a Triple: Let each logical variable have some value of the correct type. Then the interpretation of a triple { P } C { Q } is true if, whenever execution of C is begun in a state satisfying P, both the execution of S terminates and the resulting state satisfies Q, Please note: Andrews uses partial correctness in his triple interpretation. He then requires a separate proof of termination to derive rorai correctness. 1 prefer a total correctness interpretation which includes the termination requirement. In a triple, predicates P and Q.are called assertions since they assert that the program state must satisfy the predicate in order for the interpretation of the triple to be true. Assertions characterize acceptable program states; they may be considered specifications, as contracts between the implementer (programmer) and the user (customer). Predicate P is called the precondition of C, denoted pre(C). Predicate P characterizes the states of program variables; P must be satisfied before execution of C begins. Predicate Q. is called the postcondition of C, denoted post(C). Q. characterizes the states of program variables after execution of C. Two special assertions are provided: T and F. Predicate T (true) characterizes all program states; predicate F (false) characterizes no program state. In order for the interpretation of a triple to be a model for programming logic BPL, the axioms and inference rules must be sound. This ensures that all theorems provable in BPL are sound. For example, the following triple should be a theorem: {x=0} x := x+1 {x=1 ] However the following triple should not be a theorem since assigning a value to x cannot magically set y to 1. {x=0} x := x+1 {y=1 } In addition to being sound, BPL is relatively complete so that all triples that are both true and represent useful computation are in fact provable as theorems. 21.1 Axioms This section presents axioms of BPL along with informal justifications of their soundness and relative completeness. Expression evaluation is assumed to be side-effect free; i.e. only functions may be invoked within expressions because they are side-effect free. The skip command changes no variables. If predicate P is true before execution of skip, it remains true when skip terminates. Skip Axiom: { P ] skip { P } An assignment command assigns a value (say e) to a variable (say x) and thus in general changes program state. At first glance, it might seem that the axiom for assignment should start with the precondition, pre(x:-=e), and that the postcondition should be pre(x:=e) plus a predicate to indicate that now x has value e. However, going in the other direction results in a much simpler axiom for assignment. Assume that the post condition of an assignment is to satisfy a predicate P. Assignment (with side-effect free expression evaluation) changes only the designated variable x, so all other program variables have the same value before and after the assignment. Second, x has a new value, e, and thus any relation pertaining to x that is to be true after the assignment has to have been true before with x replaced by e. Textual substitution makes exactly this transformation. Assignment Axiom: { Px e } x := e { P } To illustrate the use of this axiom, consider the following triple: { T } x := 5 { x=5 } This is a theorem since: ( x=5 )x 5 - ( 5-5 ) - T This indicates that starting in any state and assigning a value to a variable gives the variable that value. As a second example, consider the triple: { y__1 } χ :=_ 4 { (y=l ) AND (x--4) ] This is also a theorem since: ((y-=1 ) AND (x=4))x 4 = ((y--l ) AND (4=4)) = ((y«1 ) AND T) = (y=l ) This illustrates that relations about variables that are not assigned to are not affected by assignment command execution. The semantics of array and record elements can be tricky, but understood that arrays and records are like functions from indices (for arrays) or field selection (for records) to the values of the appropriate element. Assignment to an element returns the whole array or record with only that element changed. 21.2 Inference Rules The program state changes as a result of executing assignment commands. The axioms for these statements use textual substitution to introduce new values into predicates. The inference rules of BPL allow the theorems resulting from these axioms to be manipulated and combined. There is one inference rule for each of the statements that affect the flow of control in a DANCE program: sequential composition (;), parallel composition (& and FORALL), alternation (IF FI), and iteration (DOOD). Each inference rule has two parts separated by a horizontal line. If the statement(s) above the line are true, the statement below the line can be inferred to be true. The first rule is for sequential composition. If a triple, { P } SI { Q } is true and another triple { Q } S2 { R } is true where { } is the same assertion in both cases, then the sequential composition of Sl and S2 is true, { P } S1 ; S2 { R }. Sequential Composition Rule: { P } SI { Q ] and { Q } S2 { R } { P } Sl ; S2 { R } Suppose we have an alternative command of the form: Where each of the B's are boolean valued expressions and the S's are DANCE commands. By the DANCE semantics at least one of the B's (guards) must be true. Then if whenever both P and Bi are true { P and Bi } and the statement Si is executed and Q.is true of the state that results { Q. } then we can infer that the whole alternative command is true. Alternative Rule: FORALL i : i IN (1..n) : { P and B| } Sj { } { P 1 IF Y { Q } Suppose we have an iterative command of the form: Where the B's are boolean-valued expressions and the S's are commands. The predicate I is the loop invariant; it is true before and after each evaluation of the loop. For one of the commands to be executed (Si) its guard (Bi) must evaluate to true. Therefore, if each triple of the form { I and Bi } Si { I { is true (i.e. execution of commands Si when Bi=true keeps the loop invariant true), then when the loop terminates (all the guards are false) we can infer that { I } DOIT { I and not (B- or ... or B π ) } is true. Iterative Rule: forall i : i IN (1..n) : { I and Bj } S, { I } { I } DOIT { I and not (Bi or ... or B n ) } The iterative rule, as stated, has a problem: partial correctness. For an iterative command to terminate, eventually all the guards must be false. To prove termination an integer bound function must be included as discussed in the later section on weakest precondition predicate transformers. { I and Bj and bound=bd } S \ { I and bound<bd}} { I } DOIT { I and not (Bi or ... or B n ) and not (bound > 0) } The rule of consequence allows us to strengthen a precondition, weaken a postcondition, or both. This rule is not really necessary since it follows from the underlying predicate calculus, it is handy in proofs, and will later be used to understand weakest-precondition predicate transformers. Suppose predicate PI implies predicate P, predicate 0_ implies predicate Q.1, and the triple { P } S { Q } is true. Then we can infer that { PI } S { Q1 } is true. Rule of Consequence: PI => P, { P } S { Q ], Q => Q1 { PI } S { Q1 } Parallel composition (&) and the parallel command (FORALL) are much more difficult to make rules for. The problem is "interference." Interference occurs when more than one intervals reference the same variables. Sometimes interference is desired. Accessing shared variables with combinable operations allows access to data structures in parallel. Andrews insists on "interference freedom." Consequently his programs are littered with "await" statements that block execution until some boolean expression is true. By dumping any operation that may read or write a variable used by a different process into an appropriately guarded await statement, Andrews achieves interference freedom. Furthermore, Andrews makes an unwarranted assumption that assignment commands are executed atomically. Assumptions of atomicity (a command or group of commands are executed indivisibly) pervade Andrews' book by placing angle brackets around whatever is to be atomic. To achieve atomicity, some form of critical section must be enforced, which, of course, destroys efficiency. For DANCE we need an inference rule something like: for all intervals that may be executed concurrently all the variables that each interval accesses (either read or write) are either 1) Read-only constant values, 2) Variables no other interval accesses, or 3) Shared variables properly accessed with combinable operations. Read-only constants and private variables are obviously O.K., but what are "properly accessed" shared variables? Remember that the combinable operations have hardware-enforced atomicity; they are both indivisible and can be performed simultaneously. The result of combinable operations (even if they occur at the same clock cycle) is the same as if they occurred in some sequential order. The shared variables are properly accessed if- and-only-if the correctness of the accesses is order-independent. Now we are ready to form the parallel rule for DANCE. Parallel Composition Rule: { P1 } SI { Ql }, { P2 } S2 { Q2 ), and AccessO (S1 ,S2) { PI and P2 } Sl & S2 { Ql and Q2 } where AccessOK(Sl,S2) is defined as: forall variables Vi occurring in Sl or S2 neither Sl nor S2 changes the value of Vi, only one of Sl or S2 accesses (read or write) Vi, or Sl and S2 access Vi solely via combinable operations and the truth of Ql and 0_2 is independent of the result of the combinable operations; or more formally: Access0 (S1 ,S2) - Wk Vi e Sl U S2 : (Vi £ lhs(Sl ) U lhs(S2) ) or ( Vi e SI and Vi e S2) or (Vi e S2 and Vi $ S1 ) or ( (Vi e co(S1 ) and Vi e co(S2) ) -» (co(S1 ) co(S2) and co(S2) - co(Sl ) ) Ihs(S) « { v l v := e e S ] co(S) ■ { v I fetch_add(v,e,v2) e S or swap(v,e,v2) e S or ... } Since the parallel command (FORALL) is merely a parameterized generalization of parallel composition, the parallel composition rule may be simply extended. Parallel Command Rule: foral! i : i in range : { P(i) } S(i) { Q(i) ] and AccessOK(S(i)) and P -> P(i) and (i) => Q { P } FORALL i IN range DO S { Q } Where P(i), S(i), and OJi) are commands or assertions where the variable i has assumed one of the values in the range of the parallel command. These inference rules cover only the major language constructs. Such things as types, whether constructed or abstract, have no inference rules as such. These rules do however allow inferences to be made about the behavior of the DANCE control structures. The next section considers full¬ blown proofs using BPL. 21.3 Program Correctness Proofs Complete proofs are onerous both to concoct and to follow. Only one example of a complete proof will be presented, and that of a very small program: Dijkstra's linear search. An array a[l..n] is given for some positive integer n. Also given is that the value x is held by some element of a. The problem is to find the first occurrence of x in a. -there is some element holding the value searched for { P: n>0 and (exists j : 1 <*=j<=n : aU]= ) } ~n is positive and some element of "a" holds value x i := 1 ; DO (a[i] /= x) => i := i+1 OD { LS: a[i] = x and (forall j : 1 <=j<i : a[j] /= x) } -a[i] holds x and no element of a with smaller index holds x The following steps constitute the proof: -> Assignment Axiom, a postualte of BPL (1) { P and 1 =1 } i := 1 { P and i=1 } - * AND-simplification of ( 1) (2) { ( P and 1 =1 )=P } •=> Rule of Consequence, ( 1) and (2) (3) { P } i := 1 { P and i=1 } = * > Assignment Axiom, (3) (4) { P and (forall j : 1 <=j<i+1 : a[j] /- x) } i := i +1 { Inv: P and (forall j : 1 <=j<i : afj] /» x) } Note: the assignment axiom just plugs in i+1 for i in the post condition. Also, the postcondition has been labeled Inv for the loop invariant. => substitution for label Inv into (Inv and (a[i] /= x) ) (5) (Inv and (a[i] /= x) ) -> ( P and (forall j:1 <=j<i: a[j] /= x) and (a[i] /= x)) => extending the range of j in (5) (6) { ( Inv and (a[i] /- x) ) - ( P and (forall j : 1 <=j<i+1 : a[j] /= x) ) } => Rule of Consequence, (4) and (6) (7) { Inv and (a[i] /- x) } i :- i + 1 { Inv } => Iterative Rule, (7) (8) { Inv } DO (a[i] /= x) => i := i+1 OD { Inv and (a[i] - x) } *- * • Empty Range Law and (4) (9) { (P and i=l ) -> Inv } = * Sequential Composition Rule, (8) and (9) (10) { P } • :- i ; DO (a[i] /= x) => i := i+1 OD { Inv and (a[i] = x) } = * • Law of Equality and substitution for Inv and LS (11) { (Inv and a[i]=x) => LS } = * • Rule of Consequence, ( 10) and (11) (12) { P } i := 1 ; DO (a[i] /= x) => i := i+1 OD { LS } As this example illustrates, even for trivial programs a full-blown proof is voluminous and hard to follow. The proof outlines in the next section present a short-hand method to prove programs correct. 21.4 Proof Outlines A proof outline, sometimes called an annotated program, consists of program commands interspersed with assertions. A complete proof outline contains at least one assertion before and after each command. A complete proof outline encodes applications of the axioms and inference rules for each command in the program. Simply the assertions before and after a command form a triple that correspond to application of the axiom or inference rule defining the behavior of the command. Assertions in proof outlines can be viewed as precise comments in a programming language. They characterize exactly what is true of the state at various points in the program. Just as it is neither necessary nor useful to place a comment between every command in a program, it is usually neither necessary nor useful to place an assertion before every command in a proof outline. Thus we will place assertions at critical points to help provide convincing evidence that a proof outline represents a proof. Some day I hope to create a "proof verifier" that can automatically determine whether a program meets its specification. Such a proof verifier will likely need a complete proof outline. 21.5 Program Derivation The linear search example showed how to use the programming logic BPL to construct a proof of correctness. Usually a programmer is given a specification of initial conditions (precondition) and the final goal (postcondition). Additionally, the program is expected to terminate. While at the University of Texas at Austin "Year of Programming" a few years ago, I encountered a different way about thinking about programs and their construction: program derivation. What was so weird was working backwards from a goal instead of running a mental interpreter. Program derivation depends on viewing commands as predicate transformers: functions that map predicates to predicates. 21.6 Weakest Preconditions Suppose the goal of a program S is to terminate in a state satisfying predicate Q, The weakest precondition of cominand (or composition of commands) S and a predicate Q, denoted wp(S, Q), is a predicate characterizing the largest set of states such that, if execution of S is begun in any state satisfying wp(S, Q), then the execution is guaranteed to terminate in a state satisfying Q, The predicate transformer wp is called the weakest since it produces a predicate characterizing the largest set of states that lead to a totally correct program. The function "wp" is an odd function. It takes as its arguments a predicate and a command. It returns another predicate. In most cases wp(S, Q) is not even computed. It is sufficient to show that the precondition implies wp(S,Q). Rather than a bunch of inference rules, weakest precondition predicate transformers use a single inference rule for program proofs. In practice, wp is easier to work with. Suppose we have a triple { P } S { Q } we wish to prove is correct. Weakest Precondition Inference Rule: wρ(S, OJ = R and P *= R { P } S { Q } If P => wp(S, Q) then { P ] S { Q ] is a theorem of BPL and hence is true. The following laws allow manipulation of expressions containing wp. Law of Conjunction: wp(S, Q) and wp(S, R) = wp(S, Qand R) Law of Disjunction: wp(S, Q) or wp(S, R) -» wp(S, Qor R) Note that the law of disjunction is only an implication while the law of conjunction has equality. 21.7 Command Semantics using Weakest Preconditions The skip command always terminates and changes no program variable. wp(skip, Q) = Q An assignment command terminates if the expression and target variable are well defined. Henceforth, wee assume this to be true. If execution of x:-=e is to terminate in a state satisfying Q, it must start in a state in which every variable except x has the same value, and x is replaced by e. As for the assignment axiom, this is exactly the transformation provided by textual substitution. wp(x := e,Q) =Q e Sequential composition of commands leads to composition of wp. wp( Sl ; S2, Q) = wp(Sl, wp(S2, Q)) Parallel composition of commands leads to conjunction provided they are AccessOK. AccessOK(Sl, S2) -> wp(Sl & S2, Q) = wp(Sl, Q) and wp(S2, Q) An alternative command terminates if all expressions in the guards are well defined, one (or more) guard is true, and the selected command Si terminates. For alternative command of the form: The weakest precondition predicate transformer for alternative commands is: wp(IFFY, Q) = (B1 =>wp(S1 , Q)) and ... and (Bn=>wp(Sn, 0 ) The iterative command has a messy weakest precondition since it may not terminate. In fact wp(DOIT, Q) is hard to determine and not very useful. Instead, an implication will be given based on a loop invariant and bound function. DOIT: DO Bl => SI [] Let BB be the disjunction of the guards: BB = Bl or B2 or ... or Bn. Let I be a predicate, and "bound" be an integer-valued expression whose value is non-negative. Then 1 _ * wp(DOIT, I and not(BB)) provided (I and Bi) = * • wp(Si, I) for l<=i<=n, (I and BB) =» bound > 0, and (I and Bi and bound=B) => wp(Si, bound<B) for l<=i<=n. The main implication is the essence of a loop invariant-it is true before execution of the loop and still true after enough executions so all the guard are false. The invariant and a true guard must imply the weakest precondition of the command with the true guard. As long as any guard is true the bound function is positive. Finally, execution of a command with a true guard reduces the bound function. The constantly decreasing, yet non-negative bound function ensures termination. Derivation of loop invariants and bound functions is an art. David Gries' Science of Programming has the best description of how to determine loop invariants. I'll not bother here. The weakest precondition for the parallel command will finish our suite of predicate transformers. FA: FORALL i IN range DO S wp(FA, Q) = wp(S[il], Q) and wp(S[i2], Q) and ... and wp(S[ik], Q) where il, i2, ... , ik are the values i takes in the range and S[ix] is the statement S with i replaced by the value ix. 22 An Extended Example: Sorting To better explain the features of the DANCE language, a sorting package is presented. The syntax is "full-blown" DANCE and so is slightly different from the syntax in previous sections. The line numbers at the left are not part of the program, but allow discussion of program text. 1 package Sort[rt,ar,<] 2 is 3 — this package contains a generic, parallel sorting procedure. 4 -- an array, its type, and a total-order operator are provided by 5 - the calling package. 7 required 8 -these type parameters must be supplied to use the package 9 type rt; -this is the type of elements to be sorted 10 type ar is array [<>] of rt; -this type is the array to be sorted 1 1 function <' ( a : rt; b : rt) return bool; 12 {(c<d) and (d<e) -> (c<e)} 13 -a total order operator must be supplied for the element type 14 end required 16 procedure sort( orig : IN SPREAD ar; final : OUT SPREAD ar; low, high : IN integer); 17 {pre: forall m,n, : i <= m,n <= j : orig[m] /= orig[n] } 18 -the elements are distinct 19 {post: forall i,j : low <= i < j <= high : final[i] < finalfj] 20 AND final = permutation(orig) } 21 -the post condition of sort says that the elements are in order and 22 --the result has the same elements as the original array 23 end package -Sort 25 --the text above this comment is the specification of an abstract data 26 -type called Sort and is called a "package" 27 -the following text denotes an object, "ParallelSort" of type "Sort" 28 -and is called a "body" 29 -only bodies can be executed 31 package body ParallelSort:Sort 32 is 33 procedure sort( orig : IN SPREAD ar; final : OUT SPREAD ar; low, high : IN integer) 34 is 35 declare 36 inter : SPREAD ar[low..high]; 37 -array "spread" over many banks to reduce memory contention 38 pivot : integer; 40 begin --of Sort 41 {pre: forall m,n, : i <= m,n <= j : a[m] /= a[n] } 42 -the elements are distinct 43 if 44 (low = high) -> finalflow] := orig[low] 45 {forall i,j : low <= i < j <= high : final[i] < final[j]} 46 [] -this symbol can be read as "or if"; it is two square brackets [ ] 47 (low > high) => skip 48 [] 49 (low < high) => 50 begin 51 partition(low, high, orig, inter, pivot) 52 {forall m,n : (low <= m < pivot < n <= high) : inter[mj < inter[n] } 53 ; -explicit sequential composition 54 sort( interflow.. pivot-1 ], final[low..pivot-1 ], low, pivot-1 ) 55 {forall i,j : low <= i < j <= pivot-1 : final[i] < finalfj]} 56 & --parallel composition 57 sort( inter[pivot+1..high], final[pivot+l ..high], pivot+1 , high) 58 {forall i,j : pivot+1 <= i < j <= high : finalfi] < final[j]} 59 & 60 final[pivot] :-_ inter[pivot] 61 end 62 fi 63 {post: forall i,j : low <= i < j <= high : final[i] < final[j] 64 AND final = permutation(orig) } 65 end -of sort procedure 67 procedure partition( i,j : IN integer; a[i..j] : IN SPREAD ar; 68 hold[i..j] : OUT SPREAD ar; k : OUT integer) 69 is 70 {pre: forall m,n, : i <= m,n <= j : a[m] /= a[n]} 71 -the elements are distinct 72 {post: forall m,n : (i <= m < k < n <= j) : hold[m] < hold[k] < hold[n] ] 73 -the lower part's values are less than the values in the upper part 74 declare 75 low_index := i, high-index := j : SHARED integer; 76 h := (i + j) DIV 2 : integer; 77 -choose an entry from a[i..j] to use as a pivot. 78 -pick from the middle: h = (i+j) DIV 2 79 begin 80 {i <= h <= j} 81 forall t in i..j do 82 begin 83 if 84 ( a[t] < a[h] ) => 85 declare 86 s : integer; -declare local s 87 begin 88 fetch_add(low_index, 1 , s); 89 ~fetch_add indivisibly reads the value in the first 90 -variable, adds the second parameter returning the 91 -result in the third, it is combinable 92 hold[s] :- a[t] 93 {forall m : i <= m <= s : hold[m] < a[h]] 94 end 95 [] 96 ( a[h] < aft] ) => 97 declare 98 s : integer; 99 begin 100 fetch_add(high_index, -1 , s); 101 hold[s] := aft] 102 {forall m : s <= m <= j : afh] < holdfm]} 103 end 104 [] 105 ( aft] = afh] ) -> skip 106 fi 107 end -of forall 108 ; 109 k := lowjndex & hold[low_index] := afh] 1 1 0 {forall m,n : (i <= m < k < n <= j) : holdfm] < holdfk] < holdfn] } 1 1 1 end - of partition 1 13 end body -of package ParallelSort:Sort Line 1 begins the package declaration called "Sort." The package declaration, which continues through line 23, defines an abstract data type (ADT). ADTs are more powerful than constructed data types in that they contain operations (functions and/or procedures) that manipulate internal, hidden data structures. DANCE packages are even more powerful than most ADTs in that they allow parametric or inclusion polymorphism. Line 1 includes three parameters: "rt", "ar", and "<". These parameters are necessarily types— sets of values. The bounded universal type quantification affected by these parameters allows the package to be used with many other (currently unspecified) types so long as they uphold the bounds required of them. Line 2 follows Ada syntax; "is" commonly connects the header of a declaration with its body. Lines 3, 4, & 5 are comments. Comments begin with two dashes and continue to the end of line. Lines 7 through 14 are a requirement clause. It demands that "rt" be any type whatsoever; "ar" be a one-dimensional array of unspecified length whose elements are of type "rt"; and "<" be a two-input function on elements of type "rt" and returning a boolean result. The assertion in line 12 demands that the function "<" be transitive, otherwise sorting will not produce an ordered list of elements. Line 6 declares that the package contains a single item-the procedure "sort", with four parameters: "orig" of type "ar" to be supplied by the caller (IN), "final" which holds the result (OUT), and "low" and "high" which indicate the range of "orig" to be sorted. The reserved word "SPREAD" causes the dynamic memory allocation of "orig" and "final" to put parts of it in many different memory banks (processors) so that "orig" and "final" may be concurrently accessed without contention. When concurrently accessible data structures are placed in a single memory bank, memory conflicts can significantly degrade performance. Line 17 demands as the precondition for invoking "sort" that the elements of the array are distinct-no duplicates. Modifying the code to handle duplicates requires an additional array to hold multiple matches of the chosen pivot. Lines 19 and 2β, the postcondition, assure that when "sort" is done, the elements will be in sorted order. Other programs that invoke "sort" can use the postcondition in their own proof outlines. Such hierarchical proofs are essential for large programs. Line 23 concludes the package declaration for "Sort". Line 31 declares that the following package body is an object named "ParallelSort" of type "Sort". Because "ParallelSort" is of type "Sort" the programmer using it needs know nothing about its internal details, just that it operates as specified by "Sort". In fact there could be many different objects of type "Sort". Sequential "BubbleSort" or "Quicksort" could be written, also of type "Sort", that would perform the same operation, but in a different way. A program written to use type "Sort" could use any of the different "ParallelSort", "BubbleSort", or "Quicksort" with equal correctness, but different performance. Line 33 begins the declaration of the procedure "sort" and must exactly match the declaration in the package "Sort". Lines 35 through 38 introduce local variables "inter" and "pivot" by existential quantification. They are only visible within the following begin-end block which corresponds to a domain when the program is executed. Line 40 begins the body of "sort", which is just one big alternative command (if-fi). Line 41 restates the precondition. Line 43 begins the alternative command which stretches to line 62. Line 44 contains one of the two alternatives. The guard, "(low=high)" is evaluated and if true, the body may be executed. (If more than one guard is true the choice of body is nondeterministic.) In this case, an array with a single element is already sorted. Line 45 is the assertion established by executing the first alternative. Note that it is the same as the overall postcondition for the whole procedure. Line 46, the "box" separates alternatives. Line 47, which should never be executed, is necessary to ensure that at least one guard is true. Line 49 guards the third alternative in which there is more than one thing to sort. Line 51 invokes the "partition" procedure. Line 52 asserts that the data have been divided (partitioned) such that one part holds elements that are all less than the pivot and elements in the other part are greater than the pivot. This corresponds to the postcondition of "partition". Line 53, a simple semicolon, represents sequential composition of temporal logic formulae. Lines 54 and 57 recursively invoke "sort" on each part divided by "partition". Line 56, an ampersand, represents parallel composition; the two sorts may execute concurrently. Lines 55 and 58 assert that each part is in order which implies lines 63 and 64, the postcondition, that the whole array (of this invocation) is in order. Lines 59 and 60 copy the value of the pivot concurrently with sorting of lines 54 and 57. Line 63 marks the end of the procedure "sort". Lines 67 and 68 declare the procedure "partition" and its parameters. Note that the arrays passed must be "SPREAD". Lines 70 and 72 are the precondition and postcondition, respectively, for "partition". The postcondition asserts that the pivot, "k", divides "hold" into greater and lesser parts. Lines 74, 75, and 76 introduce local variables "low_index", "high_index", and "h". Initially "low_index", "high_index", and "h" have values "i", "j", and the average of "i" and "j" respectively. The reserved word "SHARED" demands that the declared variables reside on a non-cacheable page; all accesses must use the Layered network-even if it happens to reside in local memory. Line 79 begins the main body of procedure "partition". Any value for "h" between "i" and "j" inclusive will satisfy the assertion in line 80; the average was chosen to minimize pernicious data arrangements. Line 81 uses the parallel command to create j-i+1 subintervals each with a different value for "t" from "i" to "j" inclusive to be executed concurrently. Since the parallel command is really bounded, universal, temporal quantification, "t" is really a constant and must not be assigned in any subinterval. Line 82 begins the block for which values of "t" are valid. Line 83 begins an alternative command. Line 84 compares the value of the element of "a" unique to this subinterval with the common pivot using the total order operator that must be supplied to use the package. Lines 85 and 86 create a new local variable "s" that is used in line 88 to determine the index to which this element "aft]" should be assigned. In this case "aft]" is smaller than the pivot "a[h]" so "low_index" is used to assign a slot in the lower half of "hold" in line 92. Since "low_index" was declared "SHARED" any number of fetch-adds may occur at the same clock cycle with different values for "s" returned as if they had been executed in some sequential order. We don't really care which element of "hold" gets the value of "a[t] " so long as it's in the lower part (less than the pivot) and no other subinterval tries to assign the same element. Line 93 asserts that all the elements between "i" and "s" inclusive are smaller than the pivot. Line 95 separates alternatives of the alternative command starting at line 83. Lines 96 through 03 are similar to lines 84 through 94 except that the value of "a[t]" is assigned to the high end of "hold". Line 105 handles the pivot and does nothing. Line 106 ends the alternative command; line 107 ends the parallel command; line 108 performs sequential composition. Line 109 copies the index of the pivot to "k" and the value of the pivot into the hole left in "hold". The assertion on line 1 10 states that the elements on the low end are smaller than the elements on the high end. The inequality is tricky; "k" will be left one greater than the index of the last element considered that was smaller than the pivot— that's the hole where the pivot's value goes. Line 110 is the postcondition that procedure "partition" seeks to establish. Line 111 ends "partition"; Line 113 ends the whole package body of "ParallelSort". To invoke "ParallelSort", the package must be imported and specialized. Use of "ParallelSort" to sort employee records alphabetically by last name is shown below. In a declaration part: import sort from ParallelSort:Sort [rt => employee- record, ar => employee_array, "<' => (employee_record.last_name < employee_record.last_name)] end import In the body of the calling procedure: ParallelSort.sort(unsorted_team,sorted_team,1 , number_of_swedes) Of course "sort" could be used several different ways in the same procedure provided that it is imported with appropriate type parameters each time. This package is also used for an example of parallel execution in OPERATION OF INVENTION. 23 Example Programs The following DANCE programs were selected to demonstrate the efficiency and elegance of DANCE compared with the code in The Design and Analysis of Parallel Algorithms. Selim G. Akl, Prentice-Hall, 1989. 23.1 Selection Akl dives into "selection" first. His algorithm picks the kth largest item from an unordered sequence. He then uses selection to pick the median of the sequence for partitioning of the sequence as part of sorting. He claims optimality (in order terms) for his algorithm, but it is so twisted and hard to follow, his claims are difficult to verify. However, for this algorithm to have any chance, the number of elements in the sequence must be much larger (orders of magnitude) than the number of processors. 23.2 Merging Akl needs merging to sort. DANCE doesn't. Akl devotes a long chapter to merging on various models and algorithms to merge two sorted sequences into a single sequence. Merging is an excellent example of a simple, efficient sequential program becoming a monstrous parallel program. Akl's algorithms are so complex and muddy I have trouble accepting his analysis that his merging algorithm is cost optimal (Akl p. 78). 23.3 Sorting Akl then uses his inelegant selection and merging to concoct an ugly, slow sorting algorithm. Sorting with combinable operations is a breeze. The trick is to use fetch-add operations to determine the array index during partition. An arbitrary element is chosen as the pivot. Each element is then compared to the pivot and if it is smaller, put at the beginning of the array, otherwise put at the end of the array. Isoefficiency analysis: DANCE program used as an example earlier is similar to quicksort which recursively partitions the list into larger and smaller halves, and then sorts the halves. The recursion stops when a partition has but a single element. If a median value is used, then each step the algorithm requires logN waves of partitioning. However finding the median requires (at least) examining every element which is really wasted effort. If an arbitrary element is chosen as the pivot then O(logN) waves are needed, but each wave is much faster. Picking an arbitrary pivot element takes U ca ι c time, the unit time of a sequential computer to perform a single operation-roughly one instruction. Comparing an element with the pivot, computing its new index, and copying it takes 3*U ca ι c time. Therefore the amount of time for each wave is 3*N*U C ai r + number of pivot picks*U ca ic- °n the first wave a single pivot will be selected. On each wave the number of partitions roughly doubles, so throughout the whole sorting procedure about N will be chosen. Since there will be (on average) logN waves the total amount of time for quicksort is: Tcalc = (3*N*logN + N)*U calc The analysis of the DANCE sort (on Multitude with combinable operations) is similar except that partitioning (and sorting) can be done in parallel. Choosing the pivot must be done before partitioning, so we incur some Amdahl's law penalty, but since many pivots can be chosen simultaneously as the algorithm progresses, and large partitions will automatically have more processors assigned to them the penalty is negligible, so the hit we take in performance is a single U ca ] C per wave (hence the logN term below). So the total amount of time to compute in parallel is: T p = (3*N*logN/P + logN) * U ca ] c The overhead T 0 is: To + Tcalc = P p T 0 = (3*N*logN+ P*logN) * U ca . - (3*N*logN + N)*U cak T 0 = (P*logN-N) * Ucalc To maintain the same efficiency, the ratio of T c to T ca ι c must be held constant: T 0 /T ca i c = = (P*logN-N) / (3*N*logN + N) So the number of processors must increase as: P = (K*(3*N*logN + N) + N)/ logN P = O(N) + 0(N/logN) + 0(N/logN) = O(N) So the parallel quicksort is (order) optimal. Increasing the number of processors proportionally with the number of elements maintains the same efficiency. Furthermore I expect real running times with P<=N to have extremely high real efficiencies since the vast majority of operations are completely parallelizible. 23.4 Matrix Manipulation Matrix operations are obligatory for many numerical analysis problems and many other problems such as connected components. function Transpose(B:matrix, s:integer) return matrix -the type "matrix" must be declared elsewhere is declare C : SPREAD matrix; begin forall i,j in [l ..s, 1..s] do C[i,j] := B[j,i] return C end -of Transpose function Mu!tiply(A,B:matrix; s:integer) return matrix is declare C : SPREAD matrix :=0; begin forall i,j in [1..s,1 ..s] do declare k : integer :=1 ; -local k for each parallel interval begin {inv: (sum m : k <= m <= s : A[i,k]*B[k,j]) + Cfi ] = val} {bound: s-k >= 0} do (k <= s) => begin Cfi ] :- A[i,k] * Bfk ] + Cfij] ; k :- k+1 end od {Cfij] = val = sum m : 1 <= m <= s : A[i,k]*B[k,j]} end -of forall {C = A*B} return C end -of Multiply For both of these matrix operations the parallel versions do no extra work so the efficiency is 1 with unit-linear isoefficiency. Well not quite since the burden of scheduling requires a fetch-add to get an index, and a DIV- MOD to extract the i,j indices of the element of matrix C to be calculated. Still, for large matrices that overhead is negligible. Akl presents some ugly algorithms for matrix transposition and multiplication for mesh, shuffle, and cube architectures vividly demonstrating how hard it is for programmers to map their algorithms onto static network architectures (Akl, ch. 7). Furthermore his CRCW multiplication algorithm (Akl p. 187) has a serious flaw in that his definition for write conflicts has "the sum of the numbers to be written is stored in that location." This risks not having all writes exactly simultaneous, thereby overwriting partial sums already computed. 23.5 Linear Equation Solution The granddaddy of all linear equation solution algorithms is Gauss-Jordan elimination. Given a matrix A and a vector b determine a vector x such that Ax=b. Akl's CREW SIMD algorithm (Akl, p. 231) does not test for division by zero yet makes no explicit requirement that the matrix be non-singular or have a non-zero diagonal. function gauss_jordan(a : matrix; b : vector; size:integer) return vector is declare Ap : SPREAD matrix[size,size] := A; x : SPREAD vectorfsize] := b; i,k : SHARED integer; j : integer=-1 ; begin {pre: forall m : 1 <= m <= size : Afm.m] /= 0} --non-zero diagonal do 0 <= size) => begin forall i,k in 1..sizej..size+1 do if (i = j) => skip f] not(i = j) => if (k > size) => xfi] := xfi] - (Ap[i,j]/Ap[jJ])*x[j] f] not(k > size) => Apf i.k] := Apfi.k] - (Ap[i.j]/Ap iJ])*Ap[j,k] fi fi j := j+1 end od ι forall i in L.size do xfi] := xfi]/Ap[i,i] r return x end -of Gauss-Jordan This adaptation of Akl's Gauss-Jordan algorithm seems to be correct, but I have not concocted a loop invariant to prove it. The following matrix_invert procedure was developed from scratch and has a loop invariant. Since I = A-lA (where I is the identity matrix) we can multiply (on the right side) by a sequence of matrices Bi, transforming A into the identity matrix and I into A-l. So if initially C=I and D-=A then the invariant will be C=A- ID. When D is transformed into I then C=A- 11= A- 1. So how are the Bi matrices determined? Well, we aren't going to exactly form the matrices Bi, but rather perform the same operations on each side-first normalizing the diagonal element and then subtracting a multiple of that row from each other row. procedure matrix_invert(A : IN OUT SPREAD matrix; size : integer; singular : OUT bool) is declare C : SPREAD matrix[size,size] = 0; D : SPREAD matrixfsize.size] = A; i : SHARED integer; j : integers 1 ; begin singular := F; forall i in 1 ..size do Cfi.i] := 1 ~ \} o {inv: C = A- ID} [bound: size-j >= 0{ (j <= size) and not(singular) => begin -normalize diagonal element if (Dfjj] /- 0) ==> forall i in 1 ..size do Cfj.i] , Dfj.i] := C[j,i]/D[j,j] , D[j,i]/D[j,j] f] (Dϋ,j] = 0) -=> -pivot matrix by adding a lower row with a non-zero element declare k : integer:=j+l ; begin do (k <= size) and (Dfk.j] = 0) => k := k+1 od > if (k > size) => singular :-= T f] not(k > size) -> begin -pivot found in row k, add to row j forall i in L.size do Cfj.i] , Dfj.i] := Cfj,i]+C[k,i] , Dfj.i] + Dfj.i] -now normalize forall i in L.size do Cfj.i] , Dfj.i] := C[j,i]/D[j,j] , D[j,i]/D[j,j] end fi end --of pivoting to gen non-zero diagonal element fi end -of normalizing diagonal element -now subtract that row (scaled by Dfi j]) from all the others forall i in 1 ..size do if (i = j) => skip-the skip command does nothing [] (i /- J) -> declare k : SHARED integer; begin -DfiJ] is the multiple of row j to subtract forall k in 1 ..size do Cfi.k] , Dfi.k] :- C[i,k] - C[i,k]/D[i,j] , Dfi.k] - D[i,k]/D[i,j] end fi j := j+1 od {C = A- 1 AND D=l AND (j>size) OR singular} —copy inverse back to A if not singular if (singular) => skip f] not(singular) => Afl .. size,! ..size] := C[1..size,1 ..size] fi end --of matrix inversion There, now 1 have a nice matrix inversion algorithm that will allow computation of any number of "x" vectors by simple matrix-by-vector multiplication. Furthermore I am sure that anomalies like zero pivots and singular matrices are covered, notwithstanding the difficulty of using inexact floating-point numbers. I like rational numbers for this reason. Isoefficiency Analysis: This is hard since the number of pivots or even whether the matrix is singular is data dependent. So let's assume that pivots are never necessary (the diagonal never becomes zero). Then n iterations have two parallel steps (normalizing and subtracting) that can use up to n processors for a running time of 0(n*( 1+n)) = 0(n2). Therefore the overall cost is 0(n3), Akl claims the sequential time is O(nx) where 2 < x < 2.5, but the actual work required, W, is hard to determine. I'm confident that this algorithm will scale linearly even though I cannot perform an exact isoefficiency analysis. 23.6 Fast Forrier Transform The fast Forrier transform (FFT) is a common signal processing operation used to extract a frequency spectrum from a sampled signal. The FFT can also be used to transform a filtered frequency domain representaion to the time domain (Akl, p. 239). function rev(x,size : integer) return integer is --reverse the bits of a number x that is s bits long declare j : integer = size; k : integer = 0; begin do (j > 0) => begin if (x DIV 2**(j-l ) > 0) => k := k + 2**(size-j) f] not(x DIV 2**(j-1 ) > 0) => skip fi x := x MOD 2**(j-1 ) & j := j-1 end od return k end -of rev (it reverses bits) procedure fft( A : IN ivector; B : OUT ivector; size : IN integer) IS declare C,D : ivector; -an ivector is a vector of complex numbers --supplied by a package with complex data —structures and operations as yet uncoded h,p,q,k : integer; z,w : complex; n : integer:*-- 2**size -n is the number of elements in the sample begin h :== size-1 -h is the loop variable counting the number of —butterfly stages & w := e**(2*pi*i/n) — w is this goofy constant, complex number -used in the FFT formula -the "i" is the square root of -1 as a defined constant —the complex_arithmetic package C := A -assign whole vector to C > do (h>=0) => begin p := 2**h -difference of indexes that are butterflied & q := 2**(size-h) & z := w**p D[0..n-1 ] := C[0..n-1 ] -make copy of current vector forall k in 0..n-1 do if ((k MOD p) = (k MOD 2*p)) => Cfk], Cfk+p] := (Dfk] + D[k+p]*z**(rev(k,size) MOD q), (Dfk] - D[k+p]*z**(rev(k,size) MOD q) [] not((k MOD p) - (k MOD 2*p)) =_> skip fi h := h-1 end od forall k in 0..n-1 do Bfk] := Cfrev(k)] end -of fft This program is unproven. Rather it was adapted directly from Akl at my peril. 23.7 Connected Components Determine for each node (vertex) in a graph, its "component number." A component number is the lowest numbered vertex the node is connected to. In this way all nodes that are connected have the same component number, and just as importantly all nodes not reachable have different component numbers. FUNCTION components(A : array[0..size-1 ,0..size-1 ] OF bool; size : integer) RETURN arrayf0..size-1 ] OF integer IS -A holds the adjancy matrix of the graph -that is Afx.y] is true -iff node x is directly connected to node y -the vector returned contains the component numbers: the -lowest numbered node that can be reached in the graph DELCARE B : array[0..size-1 ,0..size-1 ] OF bool; t : array[0..size-1 ] of integer; i : integer; BEGIN -Afx.y] is true if node x is directly connected to node y {forall x,y : 0 <= x,y < size : A[x,y] => Connect(x.y) } connectivity(A,B,size) --B contains a matrix -B[x,y] is true if there is a path from node x to node y {forall x,y : 0 <= x,y < size : Bfx.y] => Path(x,y) } ; FORALL i IN 0..size-l DO DECLARE j : integer - 0; BEGIN -find the lowest numbered node we're connected to {inv: forall k : 0 <= k < j : NOT(B[i,j]) } DO NOT(B[i.j] OR (H)) -> J .- J+1 OD {inv AND (BfiJ] OR (i=j) ) } t[i] := j {t[i]=j AND (Bfij] OR i=j) AND inv} END {post: forall v : 0 <= v < size : (forall k : 0 <= k < tfv] : NOT(Path(x,y)) ) OR Tfv] = v } -this is yet another adaptation of Dijkstra's linear search RETURN t END -of components PROCEDURE connectivity B : IN SPREAD array[0..size-1 ,0..size-1 ] OF bool; C : OUT SPREAD array[0..size-1 ,0..size-1 ] OF bool; size : IN integer) IS -given an adjacency matrix B, compute reachability matrix C -that means Cfx.y] is true iff the is some set of nodes nl , n2,...nk -so that Bfx.nl ], B[n1 ,n2], ... Bfnk.y] are all true DECLARE D : SPREAD array[0..size-1 ,0..size-1 ] of bool; i : integer = 1 ; limit : integer:=integer(log2(size)); — how many iterations? -the second "integer" is an invocation of type conversion BEGIN {forall x,y : 0 <=- x,y < size : Bfx.y] => Connect(x.y) } {Connect(x.y) => Path(x.y) } C := B -copy input {forall x,y : 0 <= x,y < size : Cfx.y] => Path(x.y) } {forall x,y : 0 <= x,y < size : Cfx.y] => MaxPathLength(x.y) = 1 } {(V => MaxPathLength = u) => (V*V => MaxPathLength = 2*u)} -see Akl page 254 for justification DO {inv: C => (MaxPathLength - 2**(i-1 )) } {bound: limit - i >=- 0 } (i < limit) => BEGIN C := Multiply(C.C) —binary matrix multiplication » i := i+1 END OD {C => (MaxPathLength = size) } {forall x,y : 0 <= x,y < size : C[x,y] => Path(x.y) } RETURN C END -of connectivity Isoefficiency Analysis: Connected components calls "connectivity" and then uses linear search in parallel. Connectivity calls "multiply" logN times. Multiply takes 0(N3) operations. So the work involved is: W = 0(N3)*logN + O(N) * N = 0(N3) Matrix multiplication can use P=N2 processors but the linear search for the smallest one can only use N. So if you had enough processors, connected components could be computed in O(logN) time. But if you want to maintain perfect efficiency=l (no extra work except for scheduling) you can only use N processors. On a time-shared Multitude with many applications vying for all the juice they can get the unused processing power during linear search will be hungrily consumed. 23.8 Parallel Prefix Parallel prefix takes a vector and returns another vector containing partial "sums" of the given vector. The summing may be simple addition, or a more complex commutitve operation. Parallel prefix arises in a surprisingly large number of algorithms. procedure parallel_prefix(var spread A : the_array; N : integer); declare M : spread array[1 ..log2(N)] of the_array; j,s : shared integer; begin {A1-A} forall s in [1..N] do Mfs.O] := Afs] ; {(s in [1 ..N] => Mfs,0] = A[s]} j := 1 do {inv: forall n in [0..J-1 ] : Mfi.n] = sum k in [i-2**n+1 ..i]} {bd: log2(N)-j} (j <= log2(N)) -> begin forall s in [1 ..N] do if (s-2**j > 0 ) -> Mfs.j] := M [sJ-1 ] + M [s-2**j+ 1 J-1 ] f] not(s-2**j > 0 ) -> M[s,j] := M[s,j-1 ] fi » j := j+1 end od {inv} {j>log2(N)} forall s in f1 ..N] do Afs] := M[s,log2(N)] end; --of parallel-prefix {A[s] = sum k in [1..s] : Al fk]} 24 BPL Soundness Proof The proof system BPL is sound for total correctness. A proof system is "sound" if it derives true facts from true facts. The proof system BPL concerns DANCE programs annotated with proof outlines. The soundness of BPL allows conclusion that a program is indeed correct. This is not to say that every correct program has a proof in BPL-that is a different property, "completeness." Rather, BPL is likely "incomplete" like the proof systems developed early this century by Russel, Whitehead, and Gόdel. For DANCE programs we seek to verify given proofs of correctness, not search for proofs of arbitrary programs. Due to the form of the proof system BPL, it is sufficient to prove that all axioms are true in the sense of total correctness and that all the inference rules of BPL are sound for total correctness. Then the result follows by induction on the length of proofs. We consider all axioms and proof rules in turn. Where applicable, soundness of weakest-precondition (wp) predicate transformer definition is proven as well. Including wp into BPL expands the ways to prove a program is correct. 24.1 Skip model: WjffSKIP]] a true. axiom: { P } skip { P } wp: wp(skip, Q) = Q grammar: <command> ::= "SKIP" proof: SKIP makes no state change whatsoever. Whatever predicates are true before execution are true afterward so both the axiom and wp for SKIP are sound. 24.2 Assignment model: tø[[X:-e]] - W βn _(i)[[X]] = W start( i) f[e]]. axiom: { Px e } x := e { P } wp: wp(x := e, Q) = Qx e grammar: <command> ::= <variable_name> ":=" <expression> proof of axiom: If the predicate P with all occurrances of X replaced by e is true at the beginning of the interval I, then P will be true at the end of the interval because the value held by X at end(i) is determined by the value of e at start(I). This is exactly the same as replacing all occurances of X with e in P, so the axiom for assignment is sound, proof of wp: Applying Weakest Precondition Inference Rule: wp(x := e, Q) = Qx e and P => Qx e { P } x := e { OJ Since by the Axiom of Assignment, { Qx e } x := e { Q }, and P __. Qx e then by the Rule of Consequence, { P } x := e { Q} so wp for assignment is sound. 24.3 Sequential Composition model: Wj[[wι ; W 2 ]] ■ true iff there exist two subintervals of i, jci and kci, such that Wj[[wι]] = true, W f[w 2 ]] = true, and end(j) = start(k). rule: { P } SI { R } and { R } S2 { Q } { P } Sl ; S2 { Q } wp: wp(S1 ; S2, Q) - wp(S1 , wp(S2, O ) grammar: <command> ::= <command> ";" <command> proof of rule: Let , W 2 =S2. Given that j satisfies {P}S1 {OJ and k satisfies { }S2 {R} and Qsatisfies end(j)=start(k), then i satisfies i ; w 2 by definition of the model. Since by construction P satisfies start(j)=start(i), and R satisfies end(k)=end(i) then interval i satisfies { P } SI ; S2 { R } so the Sequential Composition Rule is sound. proof of wp: Let R = wp(S2, Q), then by definition of wp {R}S2{Q} Let P = wp(Sl,R), then by definition of wp {P}S1 {R} and so, by the Sequential Composition Rule { P } SI ; S2 { Q } Since, by construction, P = wp(S1,wp(S2)) wp for sequential composition is sound. 24.4 Concurrent Composition model: Wjffw- & W2]] - true iff EXIST j,k : jci and kci : (W j ffw- ]] = true) AND (W k ff 2]] - true) AND (startCj) = start(k) - start(i)) AND (end(j) - end(k) = end(i)) Rule: { PI } SI { Ql } and { P2 } S2 { Q2 } and AccessOK(S1 ,S2) { PI and P2 } SI & S2 { Ql and Q2 } AccessOK(S1 ,S2) is defined as: AccessOK(S1 ,S2) - V Vi : Vi e SI U S2 : (Vi §_ Ihs(SI ) U l s(S2) ) or (Vi ε SI and Vi $ S2) or (Vi e S2 and Vi g SI ) or ( (Vi e co(S1 ) and Vi e co(S2) ) _* (co(S1 ) c co(S2) and co(S2) - co(S1 ) ) Ihs(S) - { v l v := e e S } co(S) - { v I fetch_add(v,e,v2) ε S or swap(v,e,v2) ε S or ... } wp: AccessOK(S1 , S2) =-> [wp(S1 & S2, Q) ■ wp(S1 , OJ and wp(S2, OJ] grammar: <command> ::= <command> "&" <command> proof: Both the rule and wp for concurrent composition invoke the strange relation AccessOK. AccessOK allows noninterference to be proved definition. For now, consider only intervals lacking combinable operations. AccessOK(A,B) = lnterferenceFree(A,B) The relation AccessOK is stronger than Owicki-Gries' noninterference (not falsify a proof outline, [A&O p. 216]). There exist lattices that uphold noninterference but not AccessOK. Such intervals are disallowed. The premise of the rule states: { PI } SI { Ql } and { P2 } S2 { Q2 } and AccessOK(S1 ,S2) so AccessOK(S1 ,S2) = * lnterferenceFree(S1 ,S2) which means that Sl cannot affect the truth of { 0_2 }, and S2 cannot affect the truth of { OJL }, therefore, by the model for Wj[[Sl & S2]], intervals j and k are constructed such that Wj[[S1 ]] = true and W k [ [S2]] = true and Wst_rt(i)[[P1 AND P2]] - true and W e nd(i)[fQ- AND Q2]] = true therefore we can finally conclude { PI and P2 } SI & S2 { Ql and Q2 } so the rule is sound, proof of wp: The weakest precondition predicate transformer definition uses an implication: AccessOK(S1 , S2) = * [wp(S1 & S2, OJ = wp(S1 , Q) and wp(S2, Q)] So only when the temporal commands are AccessOK will Sl & S2 have a defined meaning. Now since we are interested in programs that have meaning, we can assume AccessOK(Sl,S2) is true. Therefore wp(Sl,X) is independent of wp(S2,X) whatever the predicate X. By the model (startO) = start(k) = start(i)) AND (end(j) = end(k) = end(i)) which means that both wp(Sl, Q) and wp(S2, Q) must be true at the exact same state, namely start(i). The fact that the intervals are lattices of states is crucial. The way that interval i is composed from separate (independent, interference-free) intervals j and k allows proof that their concurrent execution does both things at the same time as if they has been executed alone. 24.5 Existential Quantification model: W [ [DECLARE X=c begin w end]] - true iff Wstart(«)[[X]] - c, and røjffw]] - true , rule: (3 dl ,d2 : domain : _δ[[d1 ]] c _&[[d2]]) AND (3 s : string : (dl && "." s) = d2) AND ( W : variable : V ε _Z>[[d1 ]] =» Vε _B[[d2]] ) AND (X ε 2>[[d2]]) AND (X-c) AND (P _» PI X-) AND (Ql -» Q) { P } DECLARE X=c begin { PI } w { Ql } end { Q } wp: wp(DECLARE X=c begin { PI } w { Ql } end.Q) = wp(w,Q1 ) provided (Ql =* Q) and all the uses of X are contained within the begin-end. Means of defining variables, their scope (visibility), an initial value, and type, grammar: <command> ::= "DECLARE" [<variable_name> ":" <type> ( "=" <expression>)]+ "BEGIN" <command> "END" proof of both rule and wp: Existential quantification is how variables are brought into existance. The new variable (and its value) is hidden for use exclusively by w. So despite what values w may temporarily assign to X, those actions cannot affect any other concurrent computation. It is the essence of AccessOK. I assume both the rule and wp are sound by definition. 24.6 Bounded Universal Quantification model: WjffFORALL X IN b DO w]] - true iff there exist subintervals of i, jl ...jn, jlci ... jnci such that W j i [[DECLARE X=x- begin w end]] - true, and Wj 2 [ [DECLARE X=x 2 begin w end]] = true, and ... and W j nf [DECLARE X=x n begin w end]] - true where x- , xz x n are all the values for X that make b true. rule: (forall i : i in range : { P(i) } S(i) { Q(i) } ) and (forall i,j : i in range :AccessOK(S(i),S(j)) ) and (forall i : i in range : P => P(i) ) and (forall i : i in range : Q(i)) => Q { P } FORALL i IN range DO S { Q } wp: AccessOK(S) => [ wp(FORALL i IN range DO S, OJ ■ wp(S[i1 ], OJ and wp(S[i2], OJ and ... and wp(S[ik], OJ ] where ii , i2 ik are the values i takes in the range and Sfix] is the statement S with i replaced by the value ix, Si x . grammar: <command> ::= "FORALL" <variable_name> ":" <type> "IN" [ <range> I <boolean_expression> ] "DO" <command> proof of rule: The objective of this proof is given a host of assumptions that { P } is true for the first state of the interval and { 0_ } is true for the last state in the same interval. The model creates a host of little domains, each with its own parameter. The rule includes the requirement that each little domain be AccessOK, whereas the model does not. Execution attempts to construct a lattice that make the program true. If the model's subintervals jl, j2, ... can be created that make each individual subinterval true, then the combined interval satisfies the whole parallel command. Each of the subintervals (j l,j2, ...) must be interference free in order to be constructed. So the requirement for interference freedom is implied in the model's definition. And now to the proof... (forall i : i in range : { P(i) } S(i) { Q(i) } ) and (forall i,j : i in range :AccessOK(S(i),S(j)) ) are premises that allow use of the Parallel Composition Rule inductively: { P(il ) AND P(i2) AND ... } S(i1 ) & S(i2) & ... { Q(i1 ) AND Q(i2) AND ... } The last two premesis (forall i : i in range : P => P(i) ) and (forall i : i in range : Q(i)) => Q allow use of the Rule of Consequence to conclude: { P } S(i1 ) & S(i2) & ... { Q } Which is exactly the meaning of { P } FORALL i IN range DO S { Q } so the rule is sound. proof of wp: AccessOK(S) _ * . [ wp(F0RALL i IN range DO S, Q) - wp(S[il ], OJ and wp(S[i2], OJ and ... and wp(S[ik], Q) ] The weakest-precondition predicate transformer (wp) for the parallel command (FORALL) is the conjunction of wp's for each value of i in the range. Of course the executions must be mutally AccessOK. Therefore the executions of S with different i must be interference free. The model creates subintervals jl, j2, ..., jk that makes S true for every value of i in the range il, i2, ..., ik. Therefore, at a minumum, the weakest precondition of concurrent execution must imply all of the individual executions: wp(F0RALL i IN range DO S, OJ => wp(S[i1 ], OJ wp(FORALL i IN range DO S, OJ -> wp(S[i2], Q) and ... and wp(FORALL i IN range DO S, OJ = * wp(S[ik], Q) The weakest function wp( FORALL i IN range DO S, Q) that implies each the individual wp(S[i], Q) is the conjunction of those individual wp's, so wp(FORALL i IN range DO S, Q) = p(S[i1 ], Q) and wp(Sfi2], Q) and ... and wp(Sfik], OJ and wp for the parallel command is sound. 24.7 Alternative Command model: WifflF Bl => Sl D B2 => S2 0- • -D Bn => Sn FI]] - true iff (W st art(i)[[Bl]] or W s tar (i) [[B2]] or ... or Wstar t (i) f-Bn]]) and (Ws t art(i)[[Bl] Wi[[S1 ]]) and (W staϊt (i) [[B2]]-*Wi[[S2]3) and ... and (W_ι__ t <i)[ . Bn_]-»WflSn]]) rule: forall i : i IN (1..n) : { P and Bj } Sj { Q } { P } IFFY { Q } wp: wp(IFFY, Q) = (Bl =>wp(S1 , Q)) and ... and (Bn=>wp(Sn, Q)) grammar: <command> ::= "IF" <boolean_expression> "=>" <command> f "[]" <boolean_expression> "=->" <command> ]* " FI" 24.8 Iterative Command model: WiffDO Bl => SI [] B2 => S2 []. . .[] Bn => Sn OD]] « true iff there exist subintervals of i, j1 ...jn, jl ci ... jnci such that ( U jk = i) and e [l..n ] (start(i)=-start(j1 ) and end(j1 )=start(j2) and ...and end(jn)=end(i) ) and forall k : 0<k<=n : ( and ... and (Ws t ar t ( j k)[[Bn]]=>W jk [[Sn]]) ). rule: forall i : i IN (l ..n) : { I and Bj and bound=bd } Sj { I and bound<bd}} and Bp>(bound > 0) { I } DOIT { I and not (B- or ... or B n ) and not (bound > 0) } wp: Let BB be the disjunction of the guards: BB = Bl or B2 or ... or Bn. Let I be a predicate, and "bound" be an integer-valued expression whose value is non-negative. Then I => wp(DOIT, I and not(BB)) provided (I and Bi) => wp(Si, I) for 1 <= =n, (I and BB) =-> bound > 0, and (I and Bi and bound=B) => wp(Si, bound<B) for 1 <«i<=n. 24.9 Next model: W s [[o(e)]] - true iff W t [[e]] = true and glb(s,t)=s and lub(s,t)=t and not (3 u εi : glb(s,u)=s and lub(u,t)=t). 24. 1 0 Non-deterministic Permutation: model: W i [fndperm(e 1 e k )]] - (e* s βks) where j : i <=j<=k : 3 : _π* = β j s and V n : 1 <=-n<=k, n--j : not(e m = e n ). 24.1 1 Fetch-Add model: W jk [[FETCH_ADD(X,e-,Yι) & FETCH_ADD(X,e2,Y2) & - & FETCH_ADD(X,e k ,Y k )]] - true iff there exists some ndperm(e- ,..., e ) = (e- s ,..., eι~j) such that Wikf[0(X=X+e 1 +e 2 + ... +e ]] = true and W ik [[θ(Yι s =X)]] = true and and ... and ... +e -is)]] - true. wp of one fetch-add: wp(FETCH_ADD(X,e,Y),OJ - (QYχ)Xχ +e wp of concurrent fetch-adds: wp(FETCH_ADD(X,e 1 ,Y 1 ) & FETCH_ADD(X,e 2 ,Y2) & - & FETCH_ADD(X,e k ,Y k )) » forall ndperm(eι e ) = (e 1 s eis) such that ( •••((QYlsχ) Y 2sχ + e1s)- --) Y ksχ+e1s+e2s+ ... +ek-1s 24. 1 2 Fetch-And odel: W ik [[FETCH_AND(X,eι,Y 1 ) & FETCH_AND(X r e 2 ,Y2) & - & FETCH_AND(X,e k ,Y k )]] _■ true iff there exists some ndperm(eι e) = (eι s ej<s) such that W i [fO(X=X AND ei AND e 2 AND ... AND €*]] = true and ANDe- s )]] = true and Wi [[0(Y3s-=XANDe ls ANDβ2s)]] = true and ... and AND eis AND e 2s AND ... AND e k -ι s )]]. wp of one fetch-and: wp(FETCH_AND(X,e,Y),Q) - (QYχ)X XA NDe wp of concurrent fetch-ands: wp(FETCH_AND(X,eι,Yι) & FETCH_AND(X,e 2( Y2) & - & FETCH_AND(X,e,Y k )) - forall ndperm(e- e) = (e ls ,..., βks) such that (•••((Q Y 1sχ)Y2sχANDe1s)---)XχANDe1sANDe2sAND ... ANDeks 24.13 Fetch-Or model: W/ ik f[FETCH_OR(X,eι,Yι) & FETCH_OR(X,e 2 ,Y2) & - & FETCH_OR(X,e k ,Y k )]] -true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., e ks ) such that OR e OR β2 OR ... OR e]] - true and - true and OR eι s )]] - true and eis OR β2s)]] = true and ... and Wik[[0(Yks-=X OR eι s OR e 2s OR ... OR e k -ι s )]] = true. wp of one fetch-or: wp(FETCH_OR(X,e,Y),Q) - (QYχ)XχoRe wp of concurrent fetch-ors: wp(FETCH_OR(X,eι ,Yι) & FETCH_OR(X,e 2 ,Y2) & - & FETCH_OR(X,e k ,Yk)) ■ forall ndperm(eι ,..., e k ) = (e ls eι~-) such that (...((QYlsχ)Y2sχθRe1s)---)XχθRe1sORe2sOR ... OReks 24.14 Fetch-Xor model: W, k [[FETCH_XOR(X,eι,Y 1 ) & FETCH_XOR(X,e 2 ,Y2) & ... & FETCH_XOR(X,e k ,Y k )]] -true iff there exists some ndperm(eι ,..., e) = (βι s e ks ) such that W ik [[θ(X=X XOR ei XOR e 2 XOR ... XOR e k ]] = true and XOR βι s )]] = true and XOR eis XO -2 S )]] ■ true and ... and W ik [[o(Y ks =X XOR eis XOR e 2s XOR ... XOR e k -ι s )]] - true, wp of one fetch-xor: wp(FETCH_XOR(X,e,Y),OJ - (QYχ)XχχoRe wp of concurrent fetch-xors: wp(FETCH_XOR(X,eι,Yι) & FETCH_XOR(X,e 2 ,Y2) & ... & FETCH_XOR(X,e k ,Y k )) ■ forall ndperm(βι ,..., e k ) = (eι s ,-, eks) such that (•• ■ ((Q Y 1sχ)Y2sχχθRe1s)-")XχXO e1sXORe2sXOR ... XOR eks 24.15 Swap model: W ik [[SWAP(X,eι,Yι) & SWAP(X,e 2 ,Y2) & - & SWAP X.ek.YiJ]] - true iff there exists some ndperm(eι ,..., e k ) = (βι s ,..., eta) such that Wiid X βks]] = true and = true and rø_k[-θ(Y 2s -eis)]] = true and ... and = true, wp of one swap: wp(SWAP(X,e,Y),Q)-(QYχ)Xe wp of concurrent swaps: wp(SWAP(X,eι,Yι) & SWAP(X,e2,Y 2 ) & ... & SWAP(X,ek,Yk)) ■ forall ndperm(eι ,..., β ) = (βι s βks) such that ( ...((QYlsχ) Y2se1s)...)Xeks 25 Multitude Architecture Discussion of architecture and hardware is split between this section and OPERATION OF INVENTION. Generally static structures are covered here and dynamic operations in OPERATION. Nominally, a collection of processors that share memory via a Layered network constitute a Multitude architecture computer as shown in Figure 23. Each processor (also called node) has its own microprocessor (CPU), cache, memory management units (MMU), memory, input/output controller (IOC) and Layered network interface. The Layered network interface comprises a request chip and a response chip. The processor accesses remote memory banks through the request chip, which initiates a network request. The request is routed to the response chip of the designated processor which accesses local memory and sends a response back through the network following the same path back to the original request chip. A processing node is shown in Figure 24. Cache coherency protocols hobble many architectures. Multitude requires no cache coherency hardware, because correct DANCE programs will never have the same value in different caches at the same time! Therefore Multitude's caches can be as fast as the fastest sequential processor with no coherency hardware at all. Every other system (we know of) that provides shared memory has some cache coherency protocol with its attendant higher cost and reduced performance. 25.1 Memory Accessing The variables whose states comprise nodes (or vertices) in a lattice satisfying a DANCE program all reside in main memory: the combined, dynamic, random-access memory (DRAM) in all the processing nodes connected via a Layered network. The 64-bit physical address is divided between node number and memory address within the node as shown in Figure 25. The physical address specifies a byte; full, 64-bit word operations must begin at addresses evenly divisible by 8~the number of bytes in a word. (Many of the following numbers are merely "best guess." For example, 64-bit microprocessors are on the market today and seem to be the trend; 64 is not a magic or crucial parameter.) 25.2 Address Space Partition The physical memory is partitioned during boot according to how it will be used: OS-only, local, spread, and global. OS-only memory contains instructions and tables that must not be accessed by user programs. Local memory is reserved for the microprocessor in the same node as the memory bank. Global memory can be accessed by any processor. Spread memory must be accessed by shuffling address bits so consecutive (word) addresses reside in different memory banks to alleviate contention. OS-only memory is also local, global, or spread depending how the OS uses it. Figure 26 shows a possible partition of memory across several banks. 25.3 Addressing Modes Multitude instructions allow five addressing modes: immediate, direct, indirect, indirect with displacement, and indexed. Immediate addressing uses a constant within the instruction for its parameter value. Direct addressing uses the value in the specified memory location. Indirect addressing dereferences the address twice; the value is at the address contained in the memory location whose address is contained in the instruction. Indirect addressing with displacement (or just displacement) adds a constant to the address found in memory to determine where the value is. Indexed addressing uses two addresses, one to an integer (the index) and another to an address of an address (the base) to get the eventual address of the value. In the following Cfaddr] means the contents of memory location addr. In all cases, addr must be an address (sometimes of a memory location containing another address), disp must be an integer constant, and indx must be the address of an integer. 25.4 Tagged Memory Words Every memory word (8-bytes) has a 3-bit tag indicating its type. Tags cannot be directly modified by OS routines, and cannot be modified at all by user programs. Any attempt to use a word in conflict with its tag will be intercepted by the memory controller and will force a high-priority interrupt for the offending processor. Tags are but one way that erroneous program behavior is detected and crucial OS tables are protected. Tag types are: address, integer, float, instruction, segment capability, page descriptor, value, or nil. Error correction coding (ECC) requires 8 additional bits to achieve single error correction/double error detection (SEC/DED). Figure 27 shows the word stored in RAM by the memory controller. 25.5 Segmented Addressing All virtual addresses (those seen and manipulated by programs) are composed of a segment number, and an offset within the segment. The segment number indexes a User Segment Table (UST) which holds, among other things, the address of the origin of the segment and the length of the segment. The effective virtual address (EVA) is the origin in the UST concatenated with the offset within the segment as shown in Figure 28. At the same time the offset can be compared with the length to check that the reference is in bounds. 25.6 Paging Effective virtual addresses are translated into physical addresses using pages. A page is a chunk of memory that may be stored on disk when not in active use. Paging allows the virtual address space to appear much larger than the physical address space, by keeping most of the pages on disk. A page fault occurs when a value is accessed that currently resides on disk. If there is an empty page in memory, the IOC controlling the disk will copy the page from disk to memory and update the page table with the new, temporary, physical address of the restored page. Translation of EVA to physical address is similar to segmented address translation except that the page table is indexed with the most significant bits of EVA instead of a separate segment field. Paged address translation is depicted in Figure 29. 25.7 Activation block An activation block holds variables (and other necessary information) for procedures, functions, packages, and other language constructs. Activation records (blocks) will be kept entirely on a heap. If a processor doesn't have a pointer for a block it cannot be accessed. This contrasts sharply with the stack indexes for activation records (Computer Architecture, Vol. 1, Richard Kain, 1989, Prentice-Hall, Chapter 5). If a processor needs an ancestor's block, a pointer to that activation record may be contained in an interval map table, or by traversing pointers linking children to their parent filling in the display at the same time. Displays allow determination of the address for any accessible variable with a single fetch (to obtain a pointer to the right activation record) and simple integer arithmetic (to add the proper offset within that activation record). Activation records of procedures that may be executed by any processor are held in "spread" (interleaved, commonly addressable) memory. Conversely some activation records are best put on the local stack of a specific processor, say, for example, the processor connected to a fiber-optic channel would handle external communication tasks. Procedures can not be so easily invoked as merely pushing parameters on the top of the stack before calling the procedure. A new activation record (context) must be created to hold all of the parameters the procedure will need. Each activation record corresponds with the symbol table associated with each (meaningful, non-empty) lexical scope. Symbol tables are constructed during compilation. Program variable names will eventually devolve into an activation block pointer and an offset into the activation block. Effective address computation at run-time can be simple indexing. In sequential, block-structured languages a stack adequately supports activation block allocation because the lifetimes of the activation blocks are nested (Kain, Vol. 1, pp 181-190). For DANCE programs such a relationship does not hold due to parallel execution. While it is true that parent activation record lifetimes encompass their children's, a parent may have many different children whose lifetimes are unrelated. Therefore activation blocks must be allocated from a heap. Heaps are necessary anyway to support program invoked memory allocation. Since DANCE supports exotic, polymorphic types, the size of the allocation block may not be known at compile-time. However the size of activation block must be determined when the procedure is invoked. The following procedure calling scenario steps are adapted from Kain, Vol. 1 p. 182. A) Compute the size of activation record required. B) Allocate that sized block from the heap. C) Copy the current activation record address to the display in the new block. The immediate parent is at the "top" of the display. D) Copy the display of the caller into the new block, (optional) E) Copy the actual parameters or their pointers into the block. F) Call the procedure, updating the activation block pointer to the new block. G) Perform the procedure. H) Upon completion, copy returned values into their respective actual parameter locations. I) Restore the caller's activation record pointer with the value at the top of the display. J) Release the unneeded activation block to the heap. Obviously, allocation and deallocation from a heap is more involved than simple addition used to modify the top-of-stack in sequential machines. Therefore heap allocation and deallocation must be both speedy and be performed in parallel without blocking. The simplest strategy would be fixed-size heap frames with a queue (or FIFO) policy (Kain, Vol. 1 pp. 168-171). A vector whose length is the total number of frames holds pointers to unallocated frames. Three integers then control heap allocation: NumFree, NextAlloc, and NextReturn. NumFree is decremented on allocation and incremented on deallocation with a fetch-add. Fetch-add could also be used on NextAlloc to get the index into the vector holding the address of an unused block. Deallocation uses NextReturn similarly. The disadvantage is that the size of the page frames is always the same. If the size needed for the allocation block exceeds the frame size, then some form of indirection must be used instead of simple indexing. Additionally program-directed allocations are often small (in bytes), thus wasting most of the frame. So fixed-size frames won't work. Kain discusses three variable-size allocation strategies: first fit, best fit, and buddy system (Kain Vol. 1, pp. 171-176). All three strategies involve subdividing large blocks (when less space is needed) and combining adjacent small blocks (when no available block is big enough). The buddy system is by far the most elegant since the blocks used are always powers-of-two sized. The buddy system also (seems) resistant to heap fragmentation where enough heap space is available, but in lots of noncontiguous little pieces. Furthermore both free space lists and free space tables do not lend themselves to simultaneous allocation and deallocation. My proposed solution is the buddy system. Several free space lists are used with all of the elements in a single table being the same size. When a block of the desired size is unavailable, a next larger sized block is broken in two (or more), one piece is used and the other is placed in the next smaller list. Suppose the smallest sized block is 8 bytes. Then different lists would handle 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8k, 16k, 32k, 64k, 128k, 256k, 512k, 1M, 2M, 4M and 16M bytes. These twenty-one lists could handle all the block sizes a program would need. A different pool of page frames is needed for each physical memory partition: OS-only, global, and spread. The local partition in each node will be managed with conventional stack and heap. Figure 30 shows a buddy-heap data structure. 25.8 Processor Flags Multitude processors contains no internal registers, per se. All of the instructions reference memory locations. However, several flags are available for branching or arithmetic calculations. The flags are: carry (C), overflow (V), zero (Z), negative (N), extension (X), and test result (R). These flags are set automatically during instruction execution. They may be used for conditional branching. They may not be written or read any other way. 25.9 Three Address Code This section defines an intermediate code representation that can both: Serve as a target language for compilation, Be easily translated into specific ISAs by a compiler back-end. The form of expression will be quadruples or three-address code. Each of the "addresses" will include the object's type, context, and offset within context. The instructions for the Multitude architecture are based (loosely) on the intermediate language outlined in the "dragon" book (Compilers: Principles. Techniques, and Tools. Aho, Sethi, & Ullman, Addison-Wesley 1986, Chapter 8). The compiler back-end will be able to generate actual machine code for any commercial-off-the-shelf (COTS) microprocessor from the three address code. Five addressing modes will be used: direct, indirect, indexed, displacement, and immediate. For direct addressing the value to be used is held in the memory location specified by the field. For indirect addressing the memory location specified by the field holds the address of the parameter to be used. For immediate addressing, the parameter to be used is held within the instruction. Indexed addressing adds memory location containing an integer to a memory location containing an address to determine the address of the operand. Not all the instructions need three addresses. In all cases, the last address receives the value when one is returned. All addresses are segmented. Instructions denote the type of operands (e.g. integer or float) and their size in bytes. A size of "w" indicates whole word, "b" indicates byte, and a digit represents the number of bytes (at most eight). Some instructions need no type or size. If the size is unspecified, whole word will be assumed. Instructions will be represented in assembly-language by: [label] instruction:type:size op1 ;op2;result -comment No spaces are allowed within the instruction name and a space (or tab) is required between the instruction name and its operands. Labels must be enclosed in square brackets and must be separated from the instruction name by a space (or tab). User programs may not use any assembly-language code whatsoever. Requiring and enforcing all code be generated by "trusted" compilers helps assure security and correctness. This is part of the "nothing illegal" philosophy. Much of this section is devoted to architectural features that implement an "everything checked" philosophy. Ensuring that programs do what they're supposed to do, and nothing else, assures correctness and security. The branch instruction is an exception to the above rule. Instead of type and size as part of the instruction name, the branch condition is appended. Branch conditions are: Examples of instructions: [ label] add:i:w a;b;c -add integers in a and b, put result in c mod:i:w c;#17;d -d := c MOD 17 eq:i:wd;#0 -d = 0? br:eq [done] -test equal flag add:i:w a;#1 ;a —increment a br:t [label] -always branch to label [done] -loop exit, continue Each instruction is typed where appropriate with i, f, b, r, or c for integer, floating-point, boolean, rational, or character respectively. The rational type operators are not included because although I like rational numbers for their exactness, lacking greatest, common divisor (GCD) hardware and processors that can handle arbitrarily large integers, rational instructions will not be provided. Since rational numbers are so clean arithmetically, inclusion of special GCD hardware will be advantageous. Memory allocation and deallocation (alloc & free) will be performed using the buddy system. Such operations are included as instructions (rather than OS calls) since they are so important. Aho, et. al. pp. 467-468 have three pointer assignments, two of which are covered by indirect addressing. In this case the value of location x becomes the address of location y (x := &y). The second case (x := *y) is accomplished by using the "mov" instruction with the N (indirect) addressing mode on the first operand. The third case (*x := y) uses mov with the indirect addressing mode on the last operand. The ptr instruction is not really necessary since the address of y could be accomplished by the mov instruction with the address of y as an immediate operand. 25.10 Scheduling Multitude wouldn't be a parallel architecture without the capacity to spawn, suspend and terminate intervals. Two different scheduling instructions are required because of the two types of parallel composition in DANCE. Scheduling a single process is performed by "sched". For example, in "sort" procedure two recursive calls to "sort" are invoked in parallel with simple assignment. The parent interval will schedule each new interval sequentially, but they will be picked up by different processors and executed concurrently. Barrier synchronization at the end of each new interval causes the last one to continue with the parent interval. The power and efficiency of the Multitude architecture comes not from the ability to schedule large-grained processes, but rather from scheduling (and executing) bunches of "light-weight" processes. A light-weight process runs in its parent's environment (activation block) obviating the need for each light-weight process to allocate its own activation block thus permitting parallel execution with a minimum of fuss. The instruction "schedm" schedules many processes at once that share the same environment and code. The combinable, fetch-add operation will be used by each light-weight process to determine a unique parameter so that although running the same code, the processes do different things. Both sched and schedm require two parameters that contain addresses. One for the proper activation block and the other for the first instruction to be executed. The instruction "susp" suspends execution of the current processes and schedules the processes for later execution starting at the following instruction. The instruction "term" terminates the current process. 25.11 Combinable Operations Combinable operations (f dd, for, fand, fxor, swapi, swapf, & swapb) are included as instructions since they are so critical to parallel processing without locks or critical sections. Inclusion of combinable operations in the ISA of any microprocessor will make it my choice for Multitude. Alas, I'm aware of none that do. Therefore the node processor will probably need some kludge that transforms a write followed by a read to some specific physical address range into a combinable operation. 25.12 Segment Capability Every segment has an associated capability. A capability is used to form an effective virtual address by concatenating the address of the segment's origin with the offset provided by the instruction. The segment's length is used for range checking. The page size holding the segment has two purposes: additional memory allocation and releasing memory when no longer needed. A memory allocation request may be satisfied by the OS by using the extra space in a page without needing to get fresh memory from the heap. A capability also has access rights that limit how a segment can be used. A segment must be global, spread, shared, or private; a segment may be read-only; a segment may be protected; a segment may be OS-only. Global: resides in memory accessible by any processor. Spread: same as global, except that the bits used for memory bank number come from the least significant end of the virtual address to spread the segment over many or all the working memory banks. Shared: non-cacheable; every access must use the Layered network; memory locations serving as targets for combinable operations must be on shared pages. Private: resides in local memory for this node. Read-Only: no writing; all instructions reside in read-only segments. Protected: can force write-back of cache values; data structures that are accessed using combinable operations to assure non-interference reside in protected segments. The wback instruction forces writing back to main memory every value in that segment that may have been loaded into cache. This allows temporary caching of values from a data structure visible by any processor. Executing wback before barrier synchronization assures that main memory reflects the true state of the interval representing execution of the program as a whole. In addition, protected segments are expected to reduce LRU-induced write-back upon cache-miss; protected segments will leave many empty cache entries after wback. OS-only: OS-only segments will never appear in a User Segment Table. Requirements: Bit field indicating authorizations necessary to access the segment. A set of corresponding authorization bits (access rights) bound to each user, are compared with the requirements. All requirements must be met (corresponding authorization bits set) to access the segment. A limited number of discretionary access controls can be directly implemented in hardware. Security level: Users may read their level or lower; write their level or higher; and do combinable operations on their level only. Figure 31 shows the bits in a capability. 25.13 Page Descriptor A page descriptor is used to translate effective virtual addresses into physical addresses; check access rights, security level, type of operation; and record page use for virtual memory page swapping. Figure 32 shows the bits in a page descriptor. 25.14 Temporary Variables At any time, the processor may use implicitly temporary variables that are stored solely in cache. Of course temporary variables must be written before read. The cache immediately makes a place for the temporary variable, if it is not present already. All temporary variables are discarded upon wback, or suspending or terminating the current interval. Procedure and functions that use temporary variables must wback before they return. 25.15 Linking A compiled DANCE program will comprise many, separate code modules. The modules have placeholders for addresses not known when they were compiled. The process of connecting the modules is known as "linking." Linking may either be static (all links made when program is loaded, before execution) or dynamic (some links made during execution). Multitude can do either; static typing is sometimes required to ensure security. The beginning of every linkable subprogram contains a link record. The link record contains an entry for every symbol exported (defined in this module, used elsewhere) or imported (defined elsewhere, used in this module). Every export entry contains the exported symbol, its type, and its address. Every import entry also contains a symbol and its type, but its address is nil. Filling in all the nil addresses in every import entry in every code module is the process of static linking. Dynamic linking fills in as many links as are known when the program is loaded, but leaves some links nil. These dynamic links are made in the activation record for this subprogram. Space is allocated for the dynamic links when the subprogram's activation record is allocated. Instructions use the link record for links established before execution. The link record is part of the program segment and will be read-only after loading. Instructions use the activation record for dynamic links. In either case, displacement addressing will be used to reference imported symbol- displacement from the origin of program text or the origin of the activation record. Linking must check to see that the types linked together are compatible. Usually this is just integer, float, or instruction, but type checking of the polymorphic abstract data types provided by DANCE will entail comparing type expressions similar to that used in the compiler. Figure 33 depicts static linking. OPERATION OF INVENTION This section expounds the "operation" of this invention, the model of computation described earlier. That entails disclosing how someone would actually "use" this invention. This invention is a process for transforming information into a desired form. Generally such transformation is called "computation." This invention performs computation so that many parts of the computation may proceed concurrently. Concurrent computation is achieved by constructing a particular kind of mathematical object that satisfies (make true) a temporal logic formula. DANCE programs are temporal logic formulas. The constructed mathematical objects are lattices of states. States are values of program variables at an instant of time. The first state holds the information as received; the last state holds the transformed information in the desired form. The operation of this invention is the construction of state lattices that satisfy DANCE programs. Correct DANCE programs describe how to construct a satisfying state lattice. 1 Use of the invention To use this invention one must write a correct DANCE program, compile the program into executable form, and link-load-execute it on a Multitude- architecture computer. 1.1 Writing a DANCE program Composing a DANCE program and its proof outline together is more art than craft. Gries' Science of Programming well explains derivation of sequential programs. Writing parallel programs in DANCE is similar, but seeks to maximize the opportunities for concurrent computation. In general, new data structures accessed via combinable operations will be required to achieve non-interference without blocking. 1.2 Compiling Compilation of DANCE programs differs little from compilation of ordinary languages. We believe the best way to do so is to use an attribute grammar to define language syntax and semantics. Currently those who compile large programs perform "parallel" compilation by using several networked workstations to compile individual "compilation units." Rich opportunities for concurrent computation within compilation units can be realized with a DANCE compiler written in DANCE. Lexical analysis - Parsing - Tree building - Tree decoration - * Code generation 1.3 Execution on a Multitude-architecture MPP Prior to execution, the program must be loaded, linked, and scheduled. Proper linking of packages with type parameters (e.g. ParallelSort) is tricky; every invocation uses the same object code with different, appropriate links. This facilitates use of pre-compiled libraries and minimizes the need for recompilation when the program is modified. 2 How to write a DANCE program Once the new paradigm embodied in DANCE is understood (a major change from the inherently sequential way people think), DANCE programs will be as easy to write as Pascal programs. We expect that high school graduates with a firm grounding in algebra and two years of trade school will write most DANCE programs used—unlike the graduate degrees and agonizing labor necessary to program currently available MPPs. 2.1 Determine what you want Software engineering methods' "waterfall" model stress requirements evaluation prior to design and coding. Determine "what" the program should do, rather than "how" it should do it. 2.2 Formally specify requirements Natural language specifications must be transformed into assertions written in the first-order predicate calculus used for proof outlines. For most applications this means composing the precondition the program can assume and the postcondition the program must assure. Real-time and transaction- based systems will write pre- and postconditions for every event that can occur with the caveat that several (or many) events may be serviced simultaneously. Formal requirements can form a contract between developers and users. Proof outlines show that a program meets its formal specification-not that the system does what's intended. Assuring that the specification accurately captures the desires of users will remain a human-intensive activity forever. Fortunately, the process of transforming natural language specifications into formal assertions identifies ambiguities and lapses in the natural language specification early in the development process. Thus critics of correctness proofs because they only assure a program meets its specification not what is wanted, are utterly wrong. Formality assists conformance to expectations. 2.3 Choose concurrently accessible data structures Unfortunately, dusty-deck FORTRAN programs cannot be automatically converted into DANCE and obtain efficiencies possible. New, concurrently- accessible data structures that use combinable operations must be created. Fetch-add is useful for accessing arrays to assure that at most one interval is reading or writing any array element. Similarly, swap is useful for manipulating dynamic data structures like linked-lists and trees. 2.4 Array Queue Figure 12 depicts a simultaneously accessible queue. For simplicity we (initially) assume that the queue size is sufficiently large enough so that there is always enough room for another put, and there is always something in the queue for a get. Put code: fetch_add(tail,1 ,put_index); Q[put_index MOD QJength] :_- put_ value Get code: fetch_add(head,1 ,get_index); get_value := Q[get_index MOD QJength] Now let's assume that the queue may be empty or full, in which case the interval will be suspended, and retried later. Put code: {inv: 0 <= min_in <= (tail-head) MOD QJength <= max_in < QJength } { OLDTAIL - tail } put: success := F; do (not success) -> declare size : nat; begin fetch_add(max_in, 1 ,size); if (size < QJength ) -> begin fetch_add(tail,1 , put_index); Q[put_index MOD QJength] := put_value; fetch_add(min_in,1 .dummy); success := T end [] NOT(size < QJength ) -> begin fetch_add(max_in,-1 .dummy); suspend end fi end od { inv AND exists i : OLDTAIL <= i < tail : Q[i]=put_ alue AND AccessOK} Get code: { inv } { OLDHEAD = head AND OLDQ = } get success := F; do (not success) -> declare size : nat; begin fetch_add(min_in,-1 .size); if (size > 0) -> begin fetch_add(head,1 ,get_index); get_value := Q[get_index MOD Q_length]; fetch_add(max_in,-l .dummy); success := T end [] NOT(size > 0) -> begin fetch_add(min_in,1.dummy); suspend end fi end od { inv AND exists i : OLDHEAD <= i < head : OLDQ[i]=get_vaiue AND AccessOK} 2.5 Linked List Queue Figure 13 depicts insertion of a record into a linked list. A swap is used to exchange a pointer in the linked list. When only a single insertion occurs, the swap exchanges a pointer to the new record with the pointer to the "next" record. Assigning the returned value to the "next" entry in the new record keeps the list linked. The code to insert a new record, "e" after record "c" will look something like this: e := NEW record_type; —fill in my_record swap(c.next,e,old_ptr); e.next := old_ptr; But if another interval tries to insert a record, say "f", at the same place at the same time, the swaps will combine nondeterministically. The value swapped into "e.next" will always be one of the swap parameters, "e" or "f"; the value swapped out of "e.next" will be returned as "old_ptr" to only one interval; and the value returned as "old_ptr" to the remaining interval is the swap parameter of the other interval. In Figure 13 the value swapped into "cnext" was "e", "e.next" became "f", and "f.next" got the value originally in "e.next" which was "d". Since swap is nondeterministic, record "f" could have been inserted before record "e", but in any case all of the records will be inserted between "c" and "d" no matter how many records are inserted simultaneously, but the inserted records could be in any order. Figure 14 depicts a queue made from a doubly-linked list. Such a queue is limited only by the memory available and consumes no more memory than needed. Put code: my_record := NEW record_type; -fill in my_record swap(tail,my_record, old__tail); my_record.last :-= old_tail; old_tail.next := my_record Get code: my_record := head; swap(head,my_record.next,dummy); This works fine for put; simultaneous swaps to "tail" allow multiple records to be added to the queue concurrently, but get only works for one interval at a time. 2.6 Dynamic Double-Ended Queue Making the queue work with multiple, simultaneous "get"s gets messy. Put code: my_record := NEW record-type; —fill in my_record { list = SET L : : L=head OR (EXISTS L2 : L2 IN list : L=L2.next) } — list is the set of all pointers such that either the pointer is head or -another pointer in the list has it as the next pointer {inv: min_in + lookers <= NUM r : r IN list: r.b} -min_in is at most the number of pointers in the list whose records -have the b flag true (default value) swap(tail,my_record,old_tail); my_record.last := old_tail; if (old-tail = nil) -> swap(head,my_record,dummy1 ) [] (old-tail /= nil) -> old_tail.next := my_record fi; my_record.b := T; fetch_add(min_in,1.dummy); {inv AND AccessOK} Get code: get success := F; {inv} do (not success) -> declare size : nat; begin {OLDMIN = min_in AND OLDLOOK - lookers} fetch_add(min Jn,-1 ,size); {min-in = OLDMIN-1 } if (size > 0) -> begin {lookers = OLDLOOK+1 } my_record := head; {OLDB - my_record.b} fetch_and(my_record.b,F,success); {inv2: success OR NOT OLDB} do (NOT success) -> begin my_record := my_record.next; {OLDB = my.record.b} fetch_and(my_record.b,F,success) {inv2 } end od {success AND exclusive(my_record)} swap(head,my_record.next,dummy) end [] NOT(size > 0) -> begin {min_in - OLDMIN-1 } fetch_add(min_in,1 .dummy); {(min-in = OLDMIN) AND (lookers = OLDLOOK) } -well the above assertion isn't quite accurate, but the -net change in min_in and lookers is zero suspend end fi end od {inv AND exclusive(my_record) AND success AND AccessOK} Operation of Parallel Sort Returning to the earlier example of ParallelSort, execution will be explained using references using Chicago font ( 1234567890) to line numbers of the package in section: An Extended Example: Sorting. Suppose the employee names to be sorted are: Johnson, Swenson, Anderson, Larson, Carlson, Jackson, Henderson, and Peterson. Execution will (try) to make the following temporal logic formula true by constructing a lattice of states: sort(unsorted-employees,sorted_employees, 1 ,number_of_employees) where "unsorted_employees" have the names above and the "number_of_employees" is eight. Initially the graph is only a start node as shown in Figure 15. Exchanging actual parameters for formal parameters (just a name change, no copying or computation) yields the state at the beginning of procedure "sort", line 33, depicted in Figure 16. To satisfy temporal predicate "sort" we must satisfy its body. The body of "sort" is just an alternative command. To satisfy an alternative command, we must satisfy a choice with an open guard. Since "low" is less than "high" we must satisfy the choice from line 50 to line 61. To do this we must construct an interval that first satisfies "partition", line 51 , and then satisfies two temporal predicates "sort", lines 54 and 57, and an assignment, line 60, concurrently. Figure 17 depicts such an interval. The dashed lines represent some lattice of states that satisfy the temporal predicate on the line indicated. The interval satisfying "partition", line 51 , must satisfy its definition starting on line 67. As usual, the names of the formal and actual parameters differ without performance impact. The body of "partition", lines 79 through 111 , consists of a parallel command, line 81 , followed by a pair of concurrent assignments, line 189, as shown in Figure 18. The body of the parallel command is an alternative command with three choices, lines 82 to 107. The last choice, line 105, is selected when considering the pivot, and does nothing. The other two choices perform a fetch-add, line 88 or 100, and then an assignment, line 92 or 101 , using the value returned by the fetch-add. Fetch-add is one of the combinable operations that allow concurrent intervals to use the same state variables simultaneously in a non-deterministic, yet controlled manner. State transitions involving combinable operations that occur simultaneously are crossed with a striped, horizontal bar. Figure 19 shows a few, possible concurrent intervals that satisfy the alternative command, lines 83 to 186. Many different intervals can satisfy a temporal predicate, and yield exactly the same result! The previous example illustrates that many different state lattices (intervals) can satisfy "sort"-even a sequence. The best intervals are short (few arcs in longest path from start to end of the lattice) and bushy (much concurrent computation). Figures 20, 21, and 22 exemplify a particular state lattice, that could be an interval that sorts the roster for the Swedish Bikini Team. The first state is just the values of variables in the initial invocation of "sort" as in Figure 15. The states are labeled with line numbers, but should not be thought of as the location of a program counter or place in the text of the program. Debuggers are universally used for sequential program development. A debugger allows the programmer to step through the code and examine any variables desired. This works because there is a single locus of control which can be stopped to examine state variables. The semantics of a traditional, prior-art, language for Figure 15 would be: "Push current state variables onto the stack; jump to the beginning of the 'sort' subroutine; and sequentially execute the instructions found there." In contrast, DANCE semantics for Figure 15 are: "make a lattice of states that satisfies sort(unsorted-employees,sorted_employees,1 ,number_of_employees) where the values of variables in the initial state are as given." Therefore the states of the lattice really don't correspond to any particular line of the program at all, but to tell which temporal predicate is satisfied by the subinterval containing the numbered state. Figure 20 deliberately resembles Figure 17 which depicts an interval (lattice of states) to satisfy "sort". Each of the dashed lines in Figure 17 represents a subinterval that satisfies the numbered line of the program. The subinterval from 67 to 1 1 1 in Figure 20 replaces 51 in Figure 17. Similarly subintervals 54, 57 and 60 in Figure 17 are replaced by subintervals beginning at states labeled 54, 57 and 60 which continue off the page into Figure 21. States 88 in Figure 20 hold initial values for each subinterval labeled 82-107 in Figure 19. The values for "s" in states 88 or 100 are the first actual state change-the result of a fetch-add. The state in 1 9 is the last state of the subinterval satisfying "partition". States 54 and 57 begin subintervals that make each (smaller) "sort". Those subintervals have the same shape as Figure 17. The process of inserting subintervals (lattices) continues until the predicate "sort" can be satisfied by a single assignment, 44. Each subinterval satisfying "sort" ends with a state labeled 65. Finally the array "sorted_emρloyees" gets its proper value in the last state of the interval in Figure 22. Figure 34 represents another lattice satisfying "sort" for different data without state variable values in each node of the graph. Sorting a million records will still have the same shape for the interval which satisfies "sort", just wider and deeper. Sort Assembly Language: [sort] 4 Hardware Operation The section Multitude Computer Architecture in the DESCRIPTION OF INVENTION summarized what the hardware necessary to execute is. This section describes what the hardware does. Well sort of . . . Patent disclosure requirements demand the best embodiment known to the inventor be described well enough so that a "routineer in the art," a person familiar with the topic of the patent and standard practices and technologies of the discipline but otherwise unimaginative, may use my invention. This patent (application) describes the best embodiment known. That said, the three-address code, slick memory management, capabilities, descriptors, hidden registers, and tags will almost certainly be emulated by early prototypes. Loral Defense Systems-Eagan is building a Layered network prototype system with custom GaAs switch chips that will be operational in early 1996. They are using off-the-shelf VME-card processors for the nodes. Combining is not done in the network, but rather in the custom hardware (two big chips) that handles network operations for each node. But still, it will likely be the first platform to execute a DANCE program in parallel, with all the performance-degrading emulation by conventional hardware. 4.1 Hidden Registers Several hidden registers are used by the microprocessor, but are not directly accessible: instruction counter, user, nesting level, interval, access rights, and security level. The hidden registers are shown in Figure 35. The instruction counter (ic) contains the address of the next instruction to be executed by this processor. The "user" register identifies what program the processor is doing. It is implicitly part of every address the processor generates. It tells which User Segment Table defines segments for this program. When Multitude architecture computers are time-shared as a public utility, many different user's programs will be running concurrently. Processors will be switching between users frequently. The user register ensures that user programs cannot access memory locations or other resources without an explicit entry in the User Segment Table. This is a valuable security mechanism. The "level" register defines the nesting level of the currently executing code. The default value for the segment number of every address is contained in the level register. Variables declared by a parent subprogram and visible at this level have segment numbers relative to the level register. The "interval" register designates an interval Mapping Table (IMT). An IMT maps segment numbers up to the level register into the true segment numbers. In this manner many intervals may be executing concurrently, at the same level, but still use independent segments in the same User Segment Table. The "access rights" and "security level" registers hold a user's permissions and security classification. These are compared with segment capabilities and page descriptors to enforce security policies. 4.2 Memory Management Unit The Memory Management Unit (MMU) performs address translation, reference monitoring, cache operation, virtual-memory page swapping, and node bus transactions. The TLB described below usually resides in the microprocessor, but may be included on the MMU chip. In fact, the MMU may not even have its own chip, but be also included in the microprocessor. The address translation performed by the MMU are depicted in Figure 36. 4.3 Translation Lookaside Buffer The Translation Lookaside Buffer (TLB) implements segmented addressing. It translates addresses generated by the processor (user.segment.offset) into effective virtual addresses (EVA). The TLB holds temporary copies of capabilities held in a User Segment Table (UST). The TLB is a cache for address translations. The UST defines segmented-to-EVA translation. The user register indicates which UST defines this user's capabilities. The interval register indicates segment substitution for low-numbered segments. It has as many entries as the level register. Segment numbers greater than level are unmodified. This allows multiple intervals to be executing at the same level but with different local variables. The TLB holds copies of recently used capabilities for rapid access as shown in Figure 38. EVA is translated into a physical address using a page table. Since a segment resides on a single page, inclusion of a page descriptor in the TLB allows direct translation from segmented address to physical address. Figure 39 shows the TLB augmented with page descriptors. The page size determines how many bits of page origin are used to form the physical address; larger pages use fewer bits. 4.4 Cache The cache holds values in fast static RAM (SRAM). The values in the cache are the same as the values obtained from main memory in absence of the cache, except for temporary variables which have no main memory addresses at all. Shared variables are never cached. Fortunately the vast majority of state variables can be cached. The wback instruction requires that the cache entries for a particular segment be known. A cache line holding four, 64-bit words needs 335 bits not including parity is depicted in Figure 40. Wide cache lines strain limitations of chip bonding pads and package pins. Multiplexing can alleviate pin-out restrictions at the cost of two-cycle cache access. Standard, two-way, set-associative, LRU- replacement cache management is fine, except protected segments must be written back upon wback. Figure 41 shows the set-associative cache. One method would be to read every cache line and compare its user and segment fields with the segment being written back, and if the cache line is dirty (has been written) writes the values back into main memory. The disadvantage of this approach is the time to cycle through every cache line. Augmenting the TLB with three parameters can substantially reduce the number of cache lines considered for wback. Retaining the largest and smallest offset numbers used to write variables in the segment limits range of offsets that must be tested. Counting the number of writes tells the maximum number of cache lines that may need write-back. Write counts of zero, one or two are especially helpful. 4.5 Memory Controller The Memory Controller reads and writes the DRAM chips according to node bus transactions. It performs the fetch-op for combinable operations. It checks the tags for proper use. It clears (sets to nil) memory when allocated. 4.6 Input/Output Controller The input/output controller (IOC) swaps pages to disk, reads or writes disk files, and handles any other peripherals under command of the OS. 4.7 Request and Response Chips The request and response chips interface the node to the Layered network. 5 Construct Implementations The following construct implementations are the form of code the compiler would generate for the particular command. English descriptions of code appear where appropriate. 5.1 Alternative Command if gl => sl D g2 => s2 D • • • D gn => sn fi [if] -evaluate boolean expression for gl branchrgl sl -gl is condition code of μp flags: EQ,LT,CC, -evaluate boolean expression for g2 branch:g2 s2 —evaluate boolean expression for gn branch:gn sn goto error— an error has occurred [sl] —code implementing sl goto fi [s2] -code implementing s2 goto fi [sn] -code implementing sn [fi] —code executed after alternative command 5.2 Iterative Command do g => s od [do] -evaluate g branch:g od -g is condition code for branch -code implementing s branch:T do -always branch [od] -code executed after iterative command 5.3 Parallel Composition d & c2 & . . . & en [sync] shared int -sync must be on shared, noncacheable page [par] mov #n;sync;1 -initialize sunc to n alloc sp,p;#c1 size;d block -allocate spread, protected -fill in state variables in activation block: cl block schedd block;cl -schedule starting at cl using cl block alloc sp,p;#c2size;c2block -fill in state variables in c2block schedc2block;c2 alloc sp,p;#cnsize;cnblock -fill in state variables in cnblock schedcnblock;cn term —end this interval [d ] -code implementing cl wback cl block -clear cache fadd:i:w sync;-1 ;tmp1 -segment for sync is level-1 eq:i:wl ;tmp1 —are we last to sync? branch EQjrestl term -no, quit [restl ] rtn rest -yes, continue parent [c2] -code implementing c2 wback d block fadd :i:w sync;-1 ;tmp2 eq:i:w1 ;tmp2 branch EQrest2 term [rest2] rtn rest [cn] -code implementing cn wback cl block fadd :i:w sync;-1 ;tmpn eq:i:w1 ;tmpn branch EQrestn term [restn] rtn rest [rest] -copy state variables (if needed) -last sync does rest free cl block free c2block free cnblock -code following the parallel composition 5.4 Parallel Command FORALL i IN j..k DO body [i] shared int ~i in activation record, filled w/ address by alloc [bar] shared int --barrier syncronization variable [forall] alloc:w sh;#2;i -get address for i and bar mov j; [i] --initialize i sub:i:w j;k;tmp add±w 1 ;tmp;[bar] -number of subintervals schedm myblock;body;[bar] term [after] -code following parallel command [body] fadd :i:w [i];1 ;tmpi -code for body using tmpi for i fadd w [bar];-1 ;tmpr eq±wl ;tmpr branch EQgoaft term [goaft] goto after Math Symbols e member of Predicate Calculus: Law of Negation: P - N0T(N0T(P)) Law of Excluded Middle: P OR NOT(P) = T Law of Contradiction: P AND NOT(P) - F Law of Implication: P_->Q=NOT(P)ORQ Law of Equality: (P=Q) = (p=>Q) OR (Q=>P) Laws of OR-simplification: P OR P = P PORT=T P OR F = P P OR (P AND Q) = P Laws of AND-simplification: P AND P = P PANDT = P P AND F - F P AND (P OR Q) = P Commutative Laws: (PANDQ) = (QANDP) (P OR Q) = (QOR P) (P-Q)-(Q-P) Associative Laws: P AND (QAND R) = (PANDQ) AND R P OR (QOR R) = (P OR Q) OR R Distributive Laws: P OR (QAND R) = (P OR Q) AND (P OR R) P AND (QOR R) - (P AND Q) OR (P AND R) DeMorgan's Laws: NOT(PANDQ) = NOT(P) OR NOT(Q) N0T(POR Q) = NOT(P) AND NOT(Q) AND-elimination: (P AND Q) => P OR-introduction: P => (P OR Q) DeMorgan's Laws for Quantifiers: (EXISTS B : R : P) = NOT(FORALL B : R : NOT(P)) (FORALL B : R : P) = NOT(EXISTS B : R : NOT(P)) Conjunction Law: (FORALL B : R : P AND OJ = (FORALL B : R : P) AND (FORALL B : R : Q) Disjunction Law: (EXISTS B : R : P OR Q) = (EXISTS B : R : P) OR (EXISTS B : R : Q) Counting Law: FORALL I : QNOT R : (NUM Bl : R OR I : Q= (NUM B2 : R : Q) +1) Summing Law: FORALL I : QNOT R : (SUM Bl : R OR I : Q= (SUM B2 : R : Q) + I) Product Law: FORALL I : QNOT R : (PROD Bl : R OR I : Q= (PROD B2 : R : Q) * I) Empty Range Laws: (FORALL B : F : P) = T (EXISTS B : F : P) = F (NUM B : F : P) = 0 (SUM B : F : P) = 0 (PROD B : F : P) = 1 Textual Substitution: If no variable in expression e has the same name as any bound variable in predicate P, then P χ e is defined to be the result of replacing every free occurrence of x with e. Simple DANCE Grammar This is a simplified grammar for DANCE. There are several (small) differences from the full-blown grammar. In particular, parameters in function and procedure declarations are separated by semicolons; parameters in function and procedure invocations are separated by commas. Productions within o Literals within "" One or More + Zero or more * Choice I Grouping [] Zero or One () <name> ::= <_dent-f_en> <_ιame> ::= <name "[" <expression> + "]" <name> ::= <name> "." <label> <name> ::= <name> "." <enumeration_identifieι> <expression> ::= <function_name> "(" <expression>+ ")" <expression> ::= <expression> <functi.on_symbol> <expression> <expression> ::= "(" <expression> ")" <expression> ::= <variable_name> <expression> ::= <literal> <command> ::= "SKIP" <command> ::= <procedure_name> "(" <expression>+ ") " <xommand> ::= <eommand> ";" <command> <command> ::= <command> "&" <command> <command> ::= "BEGIN" <command> "END" <command> ::= "DEC1ARE" [<variable_name> ":" <type> ( "=" <expression>)]+ "BEGIN" <command> "END" <command> ::= <variable_name> ":=" <expression> <command> ::= "FORALL" <variable_name> ":" <type> "IN" [ <range> I <boolean_expression> ] "DO" <command> <command> ::= "IF" <boolean-_xpression> "=>" <command> [ "D" <boolean_expression> "=>" <command> ] * "FI" <command> ::= "DO" <boolean_expression> "->" <command> [ "D" <boolean_expression> "=>" <command> ]* "OD" <command> ::= "RETURN" <expression> <declaration> ::= "DECLARE" <variable_name> + ":" <type> ";" <declaration> ::= "TYPE" <type_name> ( "[" <type_name>+ "]" ) "IS" <tyρe> ";" <declaration> ::= <package> I <package_body> <declaration> ::= "PROCEDURE" <procedure_name> "(" [<variable_name>+ ":" <type>]+ ")" ( "IS" <command> ) ";" <declaration> ::= "FUNCTION" <function_name> "(" [<variable_name>+ ":" <type>]+ ")" "RETURN" <type> ( "IS" <command> ) ";" <tyρe> ::= "INTEGER" I "BOOL" I "RAT" I "FLOAT" I "CHAR" <type> ::= "(" [<ddentifier>+ I <literal>+] ")" <type> ::= "ARRAY" "[" <range>+ "]" "OF" <type> <type> ::= <type> "=>" <type> <type> ::= "RECORD" <label> ":" <type> ["," <label> ":" <type>] * "END" "RECORD" <type> ::= "VARIANT" <discriminant> : <enumeration_type> ";" <label> ":" <type> ["," <label> ":" <type>]* "END" "VARIANT" <type> ::= <type_name> "[" <ιype_name> + "]" <type> ::= "PRIVATE" <package> ::= "PACKAGE" <package_name> ( "[" <name>+ "]" ) "IS" <requirement_clause> * <import_clause> * <declaration>+ ( "PRIVATE" <declaration>+) "END" "PACKAGE" <package_body> ::= "PACKAGE BODY" <body_name> ":" <package_name> "IS" <declaration>+ "END" "PACKAGE" <requirement_clause> ::= "REQUIRED" <package_name> ":" <declaration>+ "END" "REQUIRED" <import_clause> ::= "IMPORT" [ "ALL" I <name>+ ] "FROM" (<body_name> ":") <package_name> ( "(" [ formal-parameter "->" actual-parameter ]+ ")" ) "END" "IMPORT" DANCE Language Summary Expressions in DANCE are built inductively as follows: Variable names or constants: X, Y, Z, 5, 1509, 0 Functions: f(eι, ..., e ), where f is a function symbol defined by W, k>= 0, and ei, ..., e are expressions. Parentheses may be used for grouping expressions. Functions of two parameters may be represented in infix notation. Formulas (constructions of temporal predicates => DANCE programs) are built inductively as follows: Predicates: p(eι, ..., ey , where p is a predicate symbol defined by W, k>= 0, and ei, ..., e k are expressions. Logical connectives: wi & W 2 and wi ; W 2 where wi, and W 2 are predicates. Predicates may be grouped with BEGIN-END. Variables/Constants/ Literals: Wj[[X]] « W S [[X]] where X is a variable name, s is a state, i is an interval, Sj are the states comprising the interval, X e s, and s e Sj. Functions: rø_[[f(eι, .... e k )j] _ Wj[[f]] (rø_[[eι_], ..., WitM) Temporal Predicates: Wi[[p(eι, ..., e k )]] _ Wj[[p]] (Wι[[eι]], ..., Wi[[eιJ]) Skip : Wj[[SKIP]] - true. Sequential Composition: there exist two subintervals of i, jci and kci, such that W j [[wι]] = true, W k [[vv 2 ]] = true, and end(j) = start(k). Concurrent Composition: 3 j,k : jci and kci : (W j [[wι]] = true) and (W k [[w 2 ]l - true) and (start(j) = start(k) = start(i)) and (end(j) = end(k) = end(i)) Existential Quantification: Wi[[DECLARE X=c begin w end]] - true iff Wstaπ(i) [Ml = c, and W.[[w]] - true. Bounded universal quantification: Wi[[FORALL X IN b DO w]] - true iff 3 jl...jn : jlci ... jnci : Wji [[DECLARE X=x- begin w end]] - true, and Wj 2 [[DECLARE X=x 2 begin w end]] - true, and ... and W jn [[DECLARE X=x n begin w end]] = true where xi, X2, ..., x n are all the values for X that make b true. Assignment: - røend(i)_[X]] = W start(i )[[e]]. Alternative command: røitr F Bl -> Sl D B2 => S2 Q ... Q Bn => Sn FI]] _ true iff (W s tart(i) [[Bl]] or W sta π(i) [[B2]] or ... or Wstartω [[Bn]]) and (Wβ t aιt(i) [[Bl]] - * Wι[[S1 ]]) and (W sta r t (i)[[B2]] _ * Wj[[S2]]) and ... and (W st _r t (i) [[Bn]] =- Wi[[Sn]]) Iterative command: Wi[[DO Bl => Sl D B2 => S2 D ... D Bn => Sn OD]] - true iff 3 jl.-.jc : jlci ... jcci : ( U jk = i) and fc e[l..c] (start(i)=start(j1 ) and end(j1 )=start 2) and ...and end(jc)=end(i) ) and V k : 0<k<=n : ( (W s an (jk) [[Bl]] -» rø jk [[S1 ]]) and (w; start(jk) [[B2]] - Wj k HS2]]) and ... and (W st ar t(jk) [[Bn]] -* W J [[Sn]]) ). Next: W s [[0(e)]] - true iff W t [[e]] = true and glb(s,t)=s and lub(s,t)=t and not (3 uei : glb(s,u)=s and lub(u,t)=t). Non-deterministic Permutation: Wi[[ndperm(eι ,..., e k )]] - (eι s ,..., e^) where V j : l<-.j<=-k : V n : l<=n<=k, n * j : not(e m = e n ). Fetch-Add: W? ik [[FETCH_ - ) D(X,eι,Yι) & FETCH_ADD(X,e 2 ,Y 2 ) & ... & FErCH_ADD(X,e k ,Y k )]] » true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., e k s) such that ». +e_J] = true and røik. Yis=X)]] = true and Wikl Y2s=X+eis)]] = true and rø_k[[0( Y3 =X+eis+e 2 s)]] = true and ... and Wu-.-CH Yks=X+eis+e2 S + ... +ek-ι s ) ]] = true. ?W ik [[FETCH_AND(X,eι,Yι) & FETCH_AND(X,e2,Y2) & ... & FETCH_AND(X,e k ,Y k )]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., e^) such that AND ei AND e 2 AND ... AND ejj] = true and Wi k [[0( Yis=X)]] = true and Wi k [[next(Y 2s =X AND eι s )]] = true and Wi k [[0( Y3s=X AND eis AND β2s)]] = true and ... and Wi k t Yks=X AND eis AND e 2s AND ... AND βk-is)]]. W ik [[FETCH_OR(X,eι,Yι) & FETCH_OR(X,e 2 ,Y2) & ... & FETCH-OR(X,e k ,Yk)]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., e^) such that rø_k[-0(X « X 0R ei OR e 2 OR ... OR ej] = true and Wik[[0(Yis=X)]] = true and W ik [[0(Y2s= OR eι s )]] = true and Wik-[0( v 3s=X OR eis OR e 2s )]] = true and ... and Wjk[[0( Yks=X OR eis OR e 2s OR ... OR β k -i . )]] = true. W ik [[FETCH_XOR(X,eι,Yι) & FETCH_XOR(X,e 2 ,Y2) & *•• & FETCH_XOR(X,ek,Yk)]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,*.., e^) such that true and W_k[[0(Yis-X)]. = true and true and Wik[[0( Y3s= XOR eis XOR e 2s )]] = true and ... and Wik[fO( Yks=X XOR eis XOR e 2s XOR ... XOR β k -is)]] = true. Wik[[SWAP(X,eι,Yι) & SWAP(X,e 2 ,Y2) & ... & SWAP(X,e k ,Y k )]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., e ks ) such that Wik[[0(X-€ks3] - true and Wik[[0( Yis-X)3] = true and and ... and Wik[[0(Y_s-ek-iβ)]] β true. Domains: Define the domain operator 3>, by: 2)[[d]] ■ { (n,v) I n e identifier, v e Type(n) } a>[[dl]] c _δ[[d2]] iff 3 s : string l dl && s = d2 neighbors(dl,d2) - _δ[[dl]] ns[[d2]3 0. remote(dl,d2) . _&[[dl]] f| -&[[d2]] = 0. Skip Axiom: { P } skip [ P } Assignment Axiom: { P* e ) x := e { P } Sequential Composition Rule: {P}Sl{Q and{Q}S2{R} [PJS1 ;S2 {Rj Alternative Rule: i : ie[1..n]: { P and Bj { S t \ Q} { P } IFFY [ Q . } Iterative Rule: { I and Bj and bound=bd } Sj { I and bound<bd}} { I ] DOIT { I and not (B- or ... or B n ) and not (bound > 0) } Rule of Consequence: P1=>P,{P}S{Q},Q=>Q1 {PH S [OJ] Parallel Composition Rule: { PI } Sl { Ql } AND { P2 ] S2 { Q2 } AND AccessOK(Sl,S2) { PI and P2 } Sl & S2 { Ql and Q2 } AccessOK(S1,S2) - Wi : Vi e Sl U S2 : (Vietlhs(S1)Uths(S2) ) or (Vi e SI andVigS2) or (Vi eS2 andVig Sl) or ( (Vieco(S1)andVieco(S2) )-=> (co(S1 ) c co(S2) and co(S2) c co(Sl ) ) Ihs(S) -{ vlv:=eeS} co(S) ■ { v I fetch_add(v,e,v2) e S or swap(v,e,v2) e S or ... } Existential Quantification Rule: (3 dl,d2 : domain :2)[[dl]]c_Z)[[d2]]) and (3 s : string : (dl + "." + s) = d2) and ( V : variable : V e 2>[[dl]] => V e 2>[[d2]]) and (X e 2>[[d2]]) and (X-c) and (P = * ■ pix c ) and (Ql _•* Q) { P } DECLARE X=c begin { PI} w { Ql } end { Q } Parallel Command Rule: (V i:ie range : { P(i) } S(i) { Qfi) }) and (V i,j : i,j e range :AccessOK(S(i),S(j)) ) and ( i : i e range : P => P(i) ) and ( i:ierange:OJi))=>Q. [P] FORALL i IN range DOS { Q} Weakest Precondition Predicate Transformers: Law of Conjunction: wp(S, Q) and wp(S, R) - wp(S, Q.and R) Law of Disjunction: wp(S, Q) or wp(S, R) -» wp(S, Qor R) Weakest Precondition Inference Rule: wp(S, Q) = R and P = * * R {P}SJQJ Weakest preconditions of statements: wp(skip, Q) = Q wp(x := e, Q) = Q?e wp(Sl; S2, Q) = wp(Sl, wp(S2, OJ) AccessOK(Sl, S2) _» wp(Sl & S2, Q) = wp(Sl, Q) and wp(S2, Q) wpdFFY, Q) = (Bl=>wp(Sl, Q)) and ... and (Bn=>wp(Sn, Q)) WP for Iterative Command: Let BB be the disjunction of the guards: BB = Bl or B2 or ... or Bn. Let I be a predicate, and "bound" be an integer-valued expression whose value is non-negative. Then (I and BB) => (bound > 0) and ( V i : 1 <»i<=n : (I and Bi) => wp(Si, I) and (I and Bi and bound=X) => p(Si, bound<X) ) I => wp(DOIT, I and not(BB)) WP for Parallel Command: wp( FORALL i IN range DO S, Q) = wp(S[il], Q) and wp(S[i2], Q) and ... and wp(S[ik], Q) where il, i2, ... , ik are the values i takes in the range and S[ix] is the statement S with i replaced by the value ix, S* x . DANCE Language Reference Manual This DANCE Language Reference Manual parallels the Ada language reference manual (MIL-STD-1815A). Principally, naming conventions, packages, and subprogram syntax are based on Ada which in turn was based on Pascal. The grammar of DANCE is defined using LNF. LNF productions can be parsed by a modified CYK algorithm. LNF is defined in Appendix C which contains the complete grammar. Appendix D has LNF productions for DANCE in lexicographic order. B. l Lexical Elements This chapter defines the lexical elements of DANCE. Lexical elements are denoted with the same LNF notation as the language grammar. LNF notation is described in appendix C. Simply, LNF productions have either one or two non-terminals (enclosed in angle brackets o) or a single terminal on the right hand side. For lexical token definition only, the vertical bar, I, should be read as "or." B. l.l Character Set Any lower case letter is equivalent to the corresponding upper case letter, except within character strings and character literals. All language constructs may be represented with a basic character set, which is subdivided as follows: <upper_case_letter> ::= "A" I "B" I "C" I "D" I "E" I "F" I "G" I "H" I "I" I "J" I "K" I "L" I "M" I "N" I "O" I "P" I "OJ' I "R" I "S" I "T" I "U" I H M I "ΛA '■* j "Y" I "γ» { ti ti <digit> ::= "0" I "1 " I "2" I "3" I "4" I "5" I "6" I "7" I "8" I "9" <special_character> ::= HU M I > ■ £" I "αi" I "Λ, " I "(" I "V I "*" I " 4 ." I " " I lf_ll I ,1 II I II /It I 11.11 I 11.11 I II II I If ff I II- II I II II I Hill I "[" I "]" I "!" I "$" I "?" I "@" I " Λ " I "~" <lower_case_letter> ::= "a" I "b" I "c" I "d" I "e" I "f" I "g" I "h" I "i" I "j" I " " I "I" I "m" I "n" I "o" I "p" I "q" I "r" I "s" I "t" I "u" I "v" I "w" I "x" I "y" I "z" <space> ::= " " <other_speciaLc aracter> ::= "\" I I I "[" I "}" The other_special_character group is specially used to allow parallel lexical analysis. B.1.2 Lexical Units and Spacing Conventions A program is a sequence of lexical units; the partitioning of the sequence into lines and the spacing between lexical units does not affect the meaning of the program. The lexical units are identifiers (including reserved words), numeric literals, character literals, strings, delimiters, and comments. A delimiter is either a special-character or a compound-symbol: <compound_symbol> ::= "=>" I ".." I "**" I ":=" I "/=" I ">=" | "<=" | "<>" Adjacent lexical units may be separated by spaces or by passage to a new line. An identifier or numeric literal must be separated in this way from an adjacent identifier or numeric literal. Spaces must not occur within lexical units, except in strings, comments, and the space character literal. Each lexical unit must fit on a single line. Any of carriage return, line feed, vertical tabulate, or form feed, and only these causes passage to a new line. Horizontal tabulate is allowed within comments. Between lexical units, horizontal tabulate is equivalent to a space, backspace is disallowed, and delete, null, or any other ASCII characters are ignored. B.1.3 Identifiers Identifiers are used as names and also as reserved words. All characters, including underscores, are significant. Lower case letters are equivalent to the corresponding upper case letter. <identifier> ::= <letter> <identifier> ::= <letter> <id1 > <id! > ::= <letter_or_digit_or_u> <id1 > ::= <letter-or_digit_or_u> <id! > <underscore> ::= "_" <letter_or_digit_or-U> ::= <letter> <letter_or_digit_or_u> ::= <digit> <letter_or_digit_or_u> ::= <underscore> <letter> ::= <upper_case_letter> <letter> ::= <lo er_case_letter> Examples: COUNT X get_symbol a_l B.1.4 Numeric Literals There are two classes of numeric literals: integers and floats. No computer has ever represented real numbers--we merely approximate them with floating point numbers. Rational numbers are denoted by a vertical bar between integers. <int> ::= <digit> <int> ::= <digit> <int> <numeric_literal> ::= <int> <numeric_literal> ::= <int> <dot_integer> <dot_integer> ::= <dot> <int> <dot> ::= "." <numeric_literal> ::= <int> <exponent> <dot_Ln_exp> ::= <dot_integer> <exponent> <numeric_literal> ::= <int> <dot_i_n_exp> <exponent> ::= <e> <plus_int> <exponent> ::=- <e> <minus_int> <e> ::= "E" <plus> ::= "+" <minus> ::= "-" <plus_int> ::= <plus> <int> <minus_int> ::= <minus> <int> <numeric_literal> ::= <int> <bar_int> <bar_int> ::= <bar> <int> <bar> ::= "I" Examples: 3.0E1 3E+1 3 3.0 913 B.1.5 Character Literals A character literal is formed by enclosing one of the 95 ASCII graphic characters between string bracket characters: accent and apostrophe. In order that arbitrary strings of characters may be represented, any included string or assertion bracket character or backslash (\) must be prefaced by a backslash. This representation of character literals allows parallel lexical analysis. <character> ::= <upper_case_letter> <character> ::= <digit> <character> ::= <special_character> <character> ::= <space> <c aracter> ::= <lower_case_letter> <character> ::= <backslash> <other_special_character> <character_literal> ::= <accent> <ch_ap> <ch_ap> ::= <character> <apostrophe> <accent> ::= <apostrophe> ::= ""' Examples: H' Λ}' B.1.6 Character Strings A character string is a sequence of zero or more characters prefixed and terminated by string bracket characters-accent and apostrophe. Character strings are represented this way to allow lexical analysis in parallel. Having different characters delineate string literals allows determination of string boundaries without global information. <character_string> ::= <accent> <st_ap> <character_string> ::= <accent> <apostrophe> <st_ap> ::= <string> <apostrophe> <string> ::= <character> <string> ::= <character> <string> The length of a string is the length of the sequence represented. Backslashes use to delinate special character literals are not counted for string length. Catenation must be used to represent strings longer than one line, and strings containing control characters. The plus character, +, is used for string literal concatenation. Examples: String containing' + ACS1I.CR + ASCII.LF + Control Characters' String containing bracket characters V V\\\{\}' B.1.7 Comments A comment starts with two hyphens and is terminated by the end of line. It may only appear following a lexical unit, or at the beginning or end of a program unit. Comments have no effect on the meaning of a program- their sole purpose is the enlightenment of a human reader. Example: --This is a comment. B.1.8 Pragmas Pragmas (compiler directives) are not included in DANCE at this time. B.1.9 Reserved Words The identifiers listed below are reserved words that have special significance in DANCE. Declared identifiers may not be reserved words. Reserved words' lower, upper, or mixed case equivalents are considered the same. B.2 Declarations and Constructed Types DANCE defines several forms of named entities. A named entity can be: a number, an enumeration literal, an object (variable), a discriminant, a record component, a type, a subprogram (procedure or function), a package, or a parameter. B.2.1 Declarations A declaration associates an identifier with a declared entity. Each identifier must be explicitly declared before it is used. A declaration may declare one or more entities. <declaration> = <object_declaration> <declaration> :: :_- <number_declaration> <declaratk>n> :: = <type_declaration> <declaration> :: = <subprogram_declaration> <declaration> ::= <package_declaration> <declaration> ::= <rename_clause> <declaration> ::= <requirment_clause> <declaration> ::= <import_clause> <declaration> ::= <declaration> <declaration> The process by which a declaration achieves its effect is called the "elaboration" of the declaration. This process generally involves several successive actions (at compile time): 1) The identifier of a declared entity is "introduced" at the point of its first occurrence; it may hide other previously declared identifiers from then on. 2) The entity is elaborated. For all forms of declarations, except those of subprograms and packages, an identifier can only be used as a name of a declared entity once the elaboration of the entity is completed. A subprogram or package identifier can be used as a name of the corresponding entity as soon as the identifier is introduced, hence even within the declaration of the entity. Elaboration at compile time consists of placing an entry in the symbol table corresponding to the scope of the declaration. 3) The last action performed by the elaboration of an object declaration may be the initialization of the declared object(s), if the initialization is a constant value. B.2.2 Object and Number Declarations An object is an entity that contains a value of a specified type. Objects can be introduced by object declarations. Objects can also be components of other objects, or formal parameters of subprograms. <object_declaration> ::= <obj1 > <obj2> <object__declaration> ::= <obj1 > <subtype_indication> <obj1 > ::= <identifier_list> <colon> <colon> ::= ":" <obj1 > ::= <identifierjist> <colon_constant> <objl > ::= <identifier_list> <colon_shared> <obj1 > ::= <identifier_list> <colon_spread> <objl > ::= <identifier_list> <colon_remote> <constant> ::= "CONSTANT' <shared> ::= "SHARED" <spread> ::= "SPREAD" <remote> ::= "REMOTE" <colon_constant> ::= <colon> <constant> <colon_shared> ::-= <colon> <shared> <colon_spread> ::= <cotan> <spread> <coton_remote> ::= <colon> <remote> <obj2> ::= <subtype_indication> <assign_expression> <assign> ::= ":=" <assign_expression> ::= <assign> <express n> An object declaration may include an expression specifying the initial value for the declared objects. Otherwise the object has value "NIL" and will cause a run-time error if the value is used before a value is assigned. An object is constant if the reserved word "CONSTANT" appears in its declaration. Objects that are not constant are called "variables." A number declaration introduces one or more identifiers naming a number defined by a Jiterai_expression, which involves only numeric literals, names of numeric literals, parenthesized literal expressions and predefmed arithmetic operators. <number_declaration> ::= <identifier_list> <num_d1 > <num_dl > ::= <colon_const> <assign_expression> <ιdentifier_list> ::= <identifier> <identifier_list> ::= <identifier> <comma_id> <comma> ::= "," <comma_id> ::= <comma> <identifier_list> During elaboration, an object is bound to its type. Restriction of object behavior to its type (type checking) is essential to program correctness. Only when our programs do exactly what we say they do (nothing more or less) then quality programming can be achieved. C hackers beware! Your days of code slinging and bit fiddling are few. The new wave of software engineering will displace your midnight marathons by disciplined teams concocting understandable specifications, elegant designs, and clean, maintainable programs. B.2.3 Type Declarations A "type" denotes a set of values and a set of operations applicable to those values. "Scalar" types have no components; their values can be enumerated, integer or floating point. "Array" and "record" types are composite; their values consist of several component values. "Private" types have well defined possible values, but are not known to users of such types. Private types are equivalent to Cardelli & Wegner's existentially quantified types. Record types may have special components called "discriminants" whose values distinguish alternative forms of values. Discriminants are defined by a "discriminant part." A private type is known only by its name, its discriminants (if any) , and the set of operations applicable to its values. <type_declaration> ::= <type_d1 > <type_d2> <type_d1 > ::= <type_ident> <is> <type_d1> ::= <type_ident> <param_is> <is> ::= "IS" <type1 > ::= "TYPE" <type_ident> ::= <type1 > <identrfter> <param_is> ::- <lbracket_t_p> <rbracket_is> <lbracket_t_p> ::= <lbracket> <type_parameters> <lbracket> ::= "[" <rbracket> ::= "]" <rbracket_is> ::= <rbracket> <is> <type_d2> ::- <type_definition> <semicolon> <type_name> ::= <identifier> <type_parameters> ::= <type_name> <type_parameters> ::= <type_name> <comma_t_p> <comma_t_p> ::= <comma> <type_parameters> Examples: TYPE matrix IS ARRAY[1..100] OF INTEGER; TYPE operatorftypel ,type2] IS ARRAYftypel ] OF type2; TYPE array_of_integers IS operator[0..10,INTEGER]; B.2.3.1 Type Definition A type definition defines a type for type declarations and other type expressions. <type_definition> ::= <enumeration_type_definition> <type__definition> ::= <integer_type__definition> <type_definition> ::= <character_type_definition> <type_definition> ::= <boolean_type_definition> <type_definition> ::= <float_type_definition> <type_definition> ::= <array_type_definition> <type_definition> ::= <string_type_definition> <type_definition> ::= <record_type_definition> <type_definition> ::= <derived_type_definition> <type_definition> ::= <private_type_definition> <type_definition> ::= <subtype_indication> <type_definition> ::= <access_type_definition> <type_definition> ::= <function_type_definition> <type_definition> ::= <variant_type_definition> <type_definition> ::= <rational_type_definition> B.2.3.2 Subtype Indication A subtype indication is a type name with possibly a constraint limiting the range of values to a subset of the named type. <subtype_indication> ::= <type_name> <subtype_indication> ::= <type_name> <range_constraint> The elaboration of a type definition always produces a distinct type. The type identifier serves as a name of the type resulting from the elaboration of the type definition. Examples of type declarations: TYPE enumerated IS (one, two, three, four); —enumerated TYPE int IS integer RANGE 0..100; -integer TYPE ch IS char; -character TYPE b IS bool; -boolean TYPE fuzzy IS float RANGE 0.0..1.0; -floating point TYPE gooey IS ARRAYfenumera ted] OF fuzzy; --array TYPE s IS string; -string TYPE dj IS RECORD -record handle : string; salary : float; funny : bool; END RECORD; TYPE type_operator[typel ,type2] IS ARRAY[type1 ] OF type2; —type definitions with parameters are type operators TYPE gooey2 IS type_operator[enumerated,fuzzy]; -a derived type is created by applying a type operator TYPE myob IS PRIVATE; -private TYPE ptr IS ACCESS gooey2; -access TYPE two_input_integer_function IS [integer/integer] => integer; —function type TYPE either_or IS —variant VARIANT selector : enumerated; one : integer, two : float, three, four : fuzzy END VARIANT; TYPE rationa range IS Oil ..111 ; -rational B.2.4 Scalar Types Scalar types comprise discrete types and floating point types. Scalar types are ordered. Discrete types are enumeration types and integer types; they may be used for array indices, sequential looping, and parallel conjunction. A range constraint specifies a set of allowable values for a scalar type. The range L..R describes the values from L to R inclusive. L is the lower bound; R is the upper bound. A value satisfies the range constraint if its value is in the range. <range_constraint> ::= <range1 > <range> <rangel > ::- "RANGE" <range> ::= <literal> <dotdot_ literals* <dotdot_ literal> ::= <dotdot> <literal> <dotdot> ::- ".." Examples: * a'_ ' z' January..June where January and June are elements of an enumerated type. B.2.5 Enumeration Types An enumeration type definition defines an ordered set of distinct values that are denoted by enumeration literals. <enumeration_type_definition> ::= <lparens> <enum_rp> <enum_rp> ::= <enumerations> <rparens> <lparens> ::= "(" <rparens> ::= ")" <enumerations> ::= <enumeration_literal> <enumerations> ::= <enumeration_literal> <comma_e_t_d> <comma_e_t_d> ::= <comma> <enumerations> <enumeration_literal> ::= <identifier> <enumeration_literal> ::= <true> <enumeration_literal> ::= <false> <enumeration_literal> ::= <string_literal> <true> ::= ' ' <false> ::= "P Example: TYPE Month IS (January, February, March, April, June, July, August, September, October, November, December); B.2.6 Character Types A character type is an enumeration type that contains character literals. Character literals of character types may be used in character strings. <character_type_definition> ::= "CHAR" B.2.7 Boolean Types There is a predefined enumeration type "bool." It contains two literals: "T" for true and "F" for false ordered with the relation F < T. The evaluation of a "condition" must result in type bool. <boolean_type_definition> ::= "BOOL" B.2.8 Integer Types Ada requires all integers to be declared with range constraints. This makes me uncomfortable. I hope that Multitude will not place a bound on integer representation. Anyway: <integer_type_definition> ::= <range> <integer_type_definition> ::= <integer> <integer> ::= "INTEGER" Examples: TYPE century IS 1 ..100; TYPE days IS array[1..31 ] OF INTEGER; B.2.9 Floating Point Types Floating point types provide approximations to the real numbers. For this reason I choose not to call them "real." However, I have omitted (for now) accuracy constraints that numerical analysis users want. No fixed point arithmetic is directly supported. <float_type_definition> ::= <range> <float_type_definition> ::= <floatl > <float1> ::= "FLOAT 1 Examples: TYPE fuzzy IS 0.0..1.0; TYPE weights IS array[1..5] OF FLOAT; B.2.10 Rational Types 1 like rational numbers; they are exact. I want to put in fast GCD hardware to support Rat. A vertical bar separates the denominator from the numerator in rational literals. <rational_type_definition> ::= <range> <rational_type_definition> ::-= <rational1 > <ratk>naπ> ::= "RAT" Examples: TYPE exact IS 0I1 ..111 ; TYPE factors IS arrayp ..10] OF RAT; B.2.11 Array Types An array object is a composite object consisting of components of the same type. A component of an array is designated using one or more index values belonging to specified discrete types. An array object is characterized by the number of indices, the type and position of each index, the lower and upper bounds for each index, and the type of the components. <array_type_definition> ::= <a_r_d1 > <a_r_d2> <a_r_d! > ::-= <a_r_d3> <a_r_d4> <array1 > ::= "ARRAY" <a_r_d3> ::= <array1 > <lbracket> <a_r_d4> ::= <index> <rbracket> <of> ::= "OF" <a_r_d2> ::= <of> <subtype_indication> <index> ::= <discrete_range> <index> ::= <discrete_range> <comma_index> <comma_index> ::= <comma> <index> <discrete_range> ::= <subtype_indication> <discrete_range> ::= <range> <discrete_range> ::= <box> <box> ::= "<>" Unconstrained arrays have no range explicitly given. The range of the allowed indices is determined by the ranges of their types. Values for unassigned components are initially NIL. Constrained arrays have ranges in their declarations. Examples: ARRAY [1..100] OF employee_records ARRAY [January..December] OF monthly_sales ARRAY [1.. size, L.size] OF matrix_element ARRAY [<>] OF mystery_achievements B.2.12 String s The predefined type "STRING" denotes a one-dimensional arrays of the predefined type "CHARACTER," indexed by the predefined type "NAT." TYPE NAT IS INTEGER RANGE 0 .. integerjast-l ; TYPE STRING IS ARRAY[<>] OF CHAR; Concatenation is a predefined operator for the type STRING and for one- dimensional array types. Concatenation is represented as "+". The relational operators <, <=, >, and >= are defined for strings by lexicographic order. <string_type_definition> ::= "STRING" Example: ' This string has' + ' catenation.' -the first character of a string has index zero first_ch := some_string[0] B.2.13 Record Types A record object is a composite object consisting of named components, which may be of different types <record_type_definition> ::= <r_t_d1 > <end_record> <end_record> ::= <end> <record1 > <end> ::= "END" <record1 > ::= "RECORD" <r_t_d1 > ::= <recordl > <r_t_d2> <r_t_d2> ::= <r_t_d3> <r_t_d2> ::= <r_t_d3> <r_t_d4> <r_t_d4> ::= <comma> <r_t_d2> <r_t_d3> ::= <identifier_list> <colon_type> <colon_type> ::* <colon> <type> Example: TYPE date IS RECORD day : 1..31 , month : months, year : -1 0000..1 0000 END RECORD; B.2.14 Discriminants A discriminant is a value of an enumeration type (including INTEGER) that determines which possible variant part is used. B.2.15 Variants A variant is like a record in that it has named components, but a variant object may have only one actual component determined by its discriminant. <variant_type_definition> ::= <v_t_d1 > <end_variant> <end_variant> ::= <end> <variant> <variant> ::= "VARIANT" <v_t_d1 > ::- <variant> <v_t_d5> <v_t_d2> ::= <v_t_d3> <v_t_d2> ::= <v_t_d3> <v_t_d4> <v_t_d3> ::= <identifier_list> <colon_type> <v_t_d4> ::_• <comma> <v_t_d2> <v_t_d5> ::= <v_t_d6> <v_t_d2> <v_t_d6> ::= <discriminant_part> <semicolon> <discriminant-part> ::=* <identifier> <colon_e_t> <colon_e_t> ::= <cok>n> <enumeratton_type> <enumeration_type> ::= <enumeration_type_definition> <enumeratbn_type> ::= <type_name> The discriminant of a variant can only be changed by assigning the whole variant. Example: TYPE device IS (printer, disk, drum); TYPE state IS (open, closed); TYPE peripheral IS RECORD status : state, variant_part : VARIANT unit : device; printer : 1 ..page_size, disk, drum : RECORD cylinder : cylinder_index, track : track_number END RECORD END VARIANT END RECORD; periph.variant_part.disk.track —yields a track number from a peripheral periph.variant_part.unit -tells what device it is B.2.16 Access Types DANCE contains access types which are a lot like pointers. I couldn't do without them. As with Ada: <access_type_definition> ::= <access> <subtype_indicat n> <access> ::= "ACCESS" Example: TYPE peripheraLref IS ACCESS peripheral; B.2.17 Declarative Parts A declarative part contains declarations and related information that apply over a region of program text. Declarative parts may appear within blocks, subprograms, and packages. <declarative_part> ::= <declarative_item> <declarative_part> ::= <declarative_item> <declarative_part> <declarative_part> ::= <program_component> <declarative_part> ::= <program_component> <declarative_part> <declarative_item> ::= <declaration> <declarative_item> ::= <import_clause> <program_component> ::= <body> <program_component> ::= <package_declaration> <program_component> ::= <body_stub> <body> ::= <subprogram_body> <body> ::= <package_bodv> The body of a subprogram or package declared in a declarative part must be provided in the same declarative part. If the body of one of these program units is a separately compiled subunit, it must be represented by a body stub at the place where it would otherwise appear. <body_stub> ::= <pack_b1 > <is_separate> <is__separate> ::= <is> <separate> <separate> ::= "separate" B.2.18 Derived Type Definition A derived type definition provides a type expression including type parameters. Type parameters are enclosed in square brackets immediately following the type name in a type declaration. When a type declaration has parameters it does not create a new type, but rather a type operator. A derived type is created when the type operator is invoked with type parameters. <derived_type_definition> ::= <der_t_d1 > <der_t_d2> <der_t_d1 > ::= <identifier> <lbracket> <der_t_d2> ::= <identifier_list> <rbracket> Examples of derivied type: TYPE type_operator[type1.type 2] IS ARRAY[type1 ] OF type2; TYPE gooey2 IS type_operator[enumerated,fuzzy]; B.2.19 Private Type Definition Private type definitions only make sense within the visible part of the package stating only that the type exists. Functions and procedures provided by the package may use the private type. <private_type_definition> :;= <private> B.2.20 Function Type Definition In order to allow functions to be passed as parameters, a function type is necessary. <function_type_definition> ::- <lbracket_t_p> <rb_ar_type> <rb_ar_type> ::= <rb_ar> <type> <rb_ar> ::= <rbracket> <arrow> Examples: TYPE two_input_integer_function IS [integer/integer] => integer; integer_addition : two_input_integer_function; B.3 Names and Expressions B.3.1 Names Names denote declared entities such as objects, numbers, types, subprograms, and packages. Names can also denote operators, components of objects, or the result returned by a function. <name> ::= <function_call> <name> ::= <operator_symbol> <name> ::= <deminame> <deminame> ::= <identifier> <deminame> ::= <indexed_component> <deminame> ::= <selected_component> B.3.2 Indexed Components An indexed component denotes a component of an array. <indexed_component> ::= <deminame> <Lcl > <Lcl> ::= <lbracket> <Lc2> <i_c2> ::= <expression_list> <rbracket> <expression_list> ::= <expression> <expression_list> ::= <expression> <comma_exp> <comma_exp> ::= <comma> <expression_list> Examples: Monthly_sales[January] matrix[x, y] B.3.3 Slices A slice denotes a sub-array formed by a sequence of consecutive components of an array. <slice> ::= <slice1 > <slice2> <slicel > ::= <deminame> <lbracket> <slice2> ::= <discrete_range> <rbracket> Example: Monthly_sales[January..June] corner[l ..n/2,1 ..n/2] B.3.4 Selected Components Selected components are used to denote record components, contents of objects denoted by access values, and entities declared in the visible part of a package. <selected_component> ::= <name_dot> <enumeration_titeral> <selected_component> ::= <name_dot> <aU> <selected_component> ::= <name_dot> <enumeration_identifier> <all> ::= "ALL" <name_dot> ::= <deminame> <dot> The name of the object from which a component is selected must denote a record, variant, or package. When the name is an access type and the reserved word ALL is the selector, the entire object pointed to is selected (dereferenced). Yes, it would be much nicer to dereference with a single period, but that's not the way Ada uses access types. DANCE follows convention. Exapmles: record2. selection pointr.ALL complex *' B.3.5 Literals A literal denotes an explicit value of a given type. <literal> ::= <numeric_literal> <literal> ::= <enumeration_literal> <literal> ::= <character_string> <literal> ::= <nil> <nil> ::= "NIL" B.3.6 Aggregates An aggregate denotes a record or an array value constructed from component values. <aggregate> ::= <lparen> <agg1 > <agg1 > ::= <c_a_list> <rparen> <c_a_list> ::= <component_association> <c_a_list> ::= <component_association> <agg2> <agg2> ::= <comma> <c_a_list> <component_association> ::= <choice_arrow> <expression> <choice_arrow> ::= <choice> <arrow> <arrow> ::= "*=>" <choice> ::= <simple_expression> <choice> ::= <discrete_range> The expressions define the values to be associated with components. Component associations can be given by naming the chosen components. An aggregate must be complete; a value must be provided for each component of the composite value. Examples: (harry=>23, julie=>28, children=>0) todays_date := (day=>7, month=>November, year=>1 992) B.4 Expressions An expression is a formula that defines the computation of a value. Expressions in DANCE are (nearly) identical with expressions in Ada and Pascal. <expression> ::= <relation> <expression> ::= <relation> <exp1 > <exp1 > ::= <logical_operator> <relation> <relation> ::= <simple_expression> <relation> ::= <simple_expression> <rel1 > <re!1> ::= <relational_operator> <simple_expression> <simple_expression> ::= <u_o_term> <sιmple_expression> ::= <u_o_term> <s_e1> <u_o_term> ::= <term> <u_o_term> ::= <unary_operator> <term> <s_e1> ::= <adding_operator> <term> <term> ::= <factor> <term> ::= <factor> <terml > <term1 > ::= <multiplying_operator> <factor> <factor> ::= <primary> <factor> ::= <primary> <factor1 > <factor1 > ::*-■ <starstar> <primary> <starstar> ::= <exponentiation_operator> <primary> ::= <literal> <primary> ::= <aggregate> <primary> ::= <name> <primary> ::= <function_call> <primary> ::= <lparen> <primary1 > <primary1 > ::= <expression> <rparen> Each primary has a value and type. The only names allowed as primaries are names denoting objects. The type of an expression depends on the type of its constituents and on the operators applied. For the case where some programmer used operator overloading, the expression type must be determinable by context. Expression examples: j + 1 Ap[i,k] - (Ap[i,j]/Ap[j,j])*Ap[j,k] N0T(k > n) A[i,k] * B[k,j] + C[i,j] B.4.1 Operators and Expression Evaluation The following operators, divided into six classes, have a predefined meaning in DANCE. These operators may be overloaded for user defined types. They are given in order of increasing precedence. All operands of a factor, term, simple expression, relation, or expression are evaluated before application of the corresponding operator. B.4.2 Logical Operators The logical operators are applicable to Bool values and to one dimensional arrays of Bool values having the same number of components. <logical_operator> ::- <and> <logical_operator> ::- <or> <logical_operator> ::= <xor> <logical_operator> ::= <cand> <logical_operator> ::== <cor> <and> ::= "AND" <or> ::= "OR" <xor> ::= "XOR" <cand> ::= "CAND" <cor> ::- "COR" B.4.3 Relational Operators The relational operators have operandi of the same type and return values of the predefined type Bool. Equality (inequality) is defined for any two objects of the same type except when the type is "limited private." Operator Operation Operand type I Result type = - j (in)equality , any type ) Bool Equality for discrete types is equality of values. Equality for Float types requires some misty handwaving about nearly equal. If the numerical analysis people want some special equality test, they must supply it; DANCE compares for exact equality. For equality tests of arrays and records, corresponding components are compared. Order tests of arrays use lexicographic ordering. Ada permits arrays of different lengths to be compared. This works slickly for strings (arrays of characters) and provides "telephone directory" ordering of various lengths of strings. However, in DANCE the arrays or records compared must be the same type, including array length. The IN operator tests whether an discrete value lies within a discrete range or a given enumeration. <relational_operator> ::= <equal> <relational_operator> ::= <not_equal> <relational_operator> ::= <less_than> <relatk>nal_operator> ::= <at_most> <relational_operator> ::= <greater_than> <relational_operator> ::= <at_least> <relational_operator> ::= <in> <equal> ::- "-" <not_equal> ::= "/-" <less_than> ::= "<" <at_most> ::="<-" <greater_than> ::= ">" <at_least> ::= ">-" Example: ch IN O'. 9' ch lN Ta', * e\ 7, , V) B.4.4 Adding Operators The adding operators are as expected (+,-) except for the concatenation operator is also +. <adding_operator> ::= <plus> <adding_operator> ::= <minus> <pius> ::= "+" B.4.5 Unary Operators The unary operators are applied to a single operand and return a result of the same type. Unary plus (+) is unnecessary since it is an identity. Unary minus (-) causes negation of numeric operands. "NOT" is logical negation and may be used for one dimensional arrays of Bool. <unary_operator> ::= <plus> <unary_operator> ::= <minus> <unary_operatoπ> ::= <not> <not> ::= "NOT' B.4.6 Multiplying Operators Multiplication (*) and division (/) are straightforward for both Int and Float types. Modulus (MOD) and remainder (REM) are defined by the relations: A = (A/B)*B + (A REM B) where (A REM B) has the same sign as A and an absolute value less than B. A = B*N + (A MOD B) where (A MOD B) has the same sign as B, which must be positive, and N is some integer. Make sure you get the operator you really want. <multiplying_operator> ::= <multiply> <multiplying_operator> ::= <divide> <multiplying_operator> ::= <mod> <multiplying_operator> ::= <rem> <multiply> ::= "*" <divide> ::= "/" <mod> ::= "MOD" <rem> ::= "REM" B.4.7 Exponentiation Operator Exponentiation (**) of an operand by a positive integer is equivalent to repeated multiplication of the operand by itself. For fractional exponents: a**b = alog(b * log(a)) where log and alog are natural (base e) logarithm and anti-logarithm respectively. <exponentiation_operator> ::= "**" B.4.8 Type Conversions Type conversions may be made between scalar types and arrays of scalar types. <type_conversion> ::= <t_d > <t_c2> <t_cl> ::= <scalar_type> <lparen> <t_c2> ::= <expression> <rparen> <scalar_type> ::=- <character_type_definition> <scalar_type> ::- <integer> <scalar__type> ::= <float1> <scalar_type> ::= <rationa!1 > The only type names allowed are Int, Char, Rat, and Float. Conversions between character and integer types are based on ASCII. Integer types may be converted to Rat, Float, or Char. Char types may only be converted to integer. Float types may be converted to Int (by truncation) and Rat (by approximation). Rat types may be converted into Int and Float (by division). Type conversion examples: lnteger(5.3) Float(3) Char( 1 ) IntegerC a') Rat(3) B.4.9 Allocators The evaluation of an allocator creates an object and yields an access value that designates the object. <allocator> ::= <alloc1> <allocator> ::= ollocl > <constraint> <allocl > ::= <new> <type_name> <new> ::« "NEW" An implementation must guarantee that any object created by evaluation of an allocator remains allocated for as long as this object or one of its subcomponents is accessible directly of indirectly, that is, as long as it can be denoted by some name. An implementation may reclaim the storage occupied by an object created by an allocator, once this object has become inaccessible. 1 hope this is enough to prohibit security breaches due to use of stale access types. Allocator example: TYPE ptr IS ACCESS some_type; -in declaration part pointer : ptr; —in declaration part pointer := NEW ptr; -in subprogram body B.4.10 Static Expressions A static expression can be evaluated at compile time and has constant value. B.5 Commands Commands in DANCE are temporal predicates which are all made true by constructing a model. That model is the lattice of state variable values that comprise an interval. The semantics are also defined through weakest- precondition predicate transformers. B.5.1 Simple and Compound Commands A command may be simple, compound, have assertions, or may be the composition of other commands. Simple commands enclose no other command. A compound command can enclose other commands. An assertional command may be preceded or followed by an assertion that has no operational effect. Composition is used to combine commands to be executed serially or concurrently. Simple commands are single commands like assignment. Compound commands have complex structure. An assertional command has assertions before or after it. Assertions have no effect on program behavior. Sequential composition causes commands to be executed one after the other; parallel composition causes commands to be executed simultaneously. <command> ::= <simple_command> <command> ::= <∞mpound_cornmand> <command> ::= <assertionaLcommand> <command> ::= <sequentiaLcomposition> <command> ::= <parallel_composition> A simple command may be an assignment of an expression to a variable, a procedure invocation, a skip (do nothimg), specify the value to be returned by a function, or a special "combinable" command utilizing Layered network supported, indivisible operations. <simple_command> ::= <assignment_command> <simple_command> ::= <procedure_call> <simple_command> ::= <skip_command> <simple_command> ::= <return_command> <simple_command> ::= <combinable_command> <compound_command> ::= <altemative_command> <compound__command> ::= <iterative_command> <comppund_command> ::= <parallel_command> <compound_command> ::= <block> <assertk>naLcommand> ::= <command> <asserttonaLcommand> ::= <assertion> <command> <assertionaLcommand> ::= <command> <assert n> <assertional_command> ::= <assertion> <a_c2> <a_c2> ::= <command> <assertion> B.5.2 Skip and Assignment Commands The skip command does nothing. <skip_command> ::= "SKIP" model: Wj[[SKIP]] - true. axiom: { P } skip { P } wp: wp(skip, Q) = Q Assigning values to variables is undamental to the imperative style of programming. <assignment_command> ::= <name_list> <assign_expression_list> <assign_expression_list> ::= <assign> <expression_list> <name_list> ::= <name> <name_list> ::= <name> <c_n_l> a. ::= <comma> <name_list> The assignment command assigns the value of the expression to a state variable. Textual substitution is used for its semantics: model: Wi[[X:-e]] - W end(i )[[X]] - rø S tan(i) [[e]]. axiom: { P*e } x :- e { P } wp: wp(x := e, Q) = Q e Where P*. is the predicate where every free occurrence of x is replaced by the expression e. This is the same as the interval temporal logic definition where the value of x at the end of the subinterval is the same as the value of the expression evaluated at the beginning of the subinterval. Multiple assignments are equivalent to parallel composition of single assignments: xl ,x2,...,xn := el ,e2,...,en ■ xl := el & x2 := e2 & ... & xn := en Assignment examples: k :- k+1 h :- (i + j) DIV 2; x,y := y,x B.5.3 Array Assignment Arrays are functions from a discrete type to the base type. The whole array can be assigned, or member by member. Array assignment examples: C[i,j]:= B[j,i] C := B x[i] := x[i] p[i,i] a[i..j] := hold[i..j] B.5.4 Record Assignment Record assignment is straightforward. Either elements can be assigned singly by appending the record variable name with a period and the name of the record component, or the entire record can be assigned, provided the types are the same. Record assignment examples: disk_jockey.handle := x Slick Willie' disk_jockey.salary := 200000.00 disk_jockey.funny := F disk_jockey1 := disk_jockey2 B.5.5 Alternative Command The alternative command allows an alternative statement to be executed, provided its guard is true. If more than one guard is true, the altenative selected is nondeterministic. The semantics are given in terms of an inference rule, wp, and model. Suppose we have an alternative command named "IFFY": IFFY : if Bi => S- [] ... [] B n => S n fi with n guarded commands gives the inference rule: Alternative Rule: V i : ie[1..n] : { P and Bi } Si { Q ] { P } IFFY { Q } In terms of weakest preconditions: wp(IFFY, Q) = (Bl=>wp(Sl, Q)) and ... and (Bn=>wp(Sn, OJ) model: WifpF Bl => Sl D B2 => S2 Q ... □ Bn => Sn FI]] _ true iff (W s tart(i)[[Bl]] or rø start(i) [[B2]] or ... or W start(i) [[Bn]]) and (W.taπ(i) [[Bl]]=>Wi[[S1 ]]) and (W s * a r t (i)[[B2]]- * røi[[S2]]) and ... and (Wstaιt(i) [[Bn]]- * Wi[[Sn]]) <alternative_command> ::= <id_colon> <alt1 > <altemative_command> ::= <artl > <id_colon> ::= <identifier> <colon> <altl> ::= <if> <alt2> <if> ::= "IF" <alt2> ::= <alt3> <fι> <fi> ::= "FI" <alt3> ::= <be_a_c> <alt3> ::= <be_a_c> <alt4> <alt4> ::= <box2> <alt3> <box2> ::= "[]" <be_a_c> ::= <boolean_expression> <arrow_command> <arrow_command> ::= <arrow> <command> Alternative command example: IF (k > n) => x[i] := x[i] - (Ap[i,j]/Ap[j ])*x[j] [] NOT(k > n) -> Ap[i,k] := Ap[i,k] - (Apti.jJ/ApLj.jjrAplj.k] FI B.5.6 Iterative Command The iterative command provides looping. Although similar to the alternative command, the iterative command will continue to execute as long as at least one of the guards is true. model: Wi[[DO Bl => Sl D B2 => S2 fj *** D Bn => Sn OD]] _ true iff 3 jl...jc : jlci ... jcci : ( (j jk = i) and k e[l..c] (start(i)=start(j1 ) and end(j1 )=start(j2) and ...and end(jc)=end(i) ) and V k : 0<k<=n : ( (W,tart(jk) [[Bl]] - WjkffSI ]]) and (W βta _t< j k) [-B2]] _*- and ... and (W S taπ(jk) [[Bn]] => W jk [[Sn]]) ) Proving correctness of a iterative command requires construction of a loop invariant. Suppose we have an iterative command named "DOIT": DOIT : do Bi =>Sι [] ... D B n =>S„ od with n guarded commands gives the inference rule: Iterative Rule: { I and Bj and bound=bd } Sj { I and bound<bd}} { I } DOIT { I and not (Bi or ... or B n ) and not (bound > 0) } The iterative statement has a messy weakest precondition since it may not terminate. In fact wp(DO_T, Q) is hard to determine and not very useful. Rather merely an implication will be given based on a loop invariant and bound function. Let BB be the disjunction of the guards: BB = Bl or B2 or ... or Bn. Let I be a predicate, and "bound" be an integer-valued expression whose value is non-negative. Then (I and BB) => (bound > 0) and (V i : 1 <=i<=n : (I and Bi) => wp(Si, I) and (I and Bi and bound=X) => wp(Si, bound<X) ) I => wp(D0IT, I and not(BB)) Once predicate "I" and boolean function "bound" have been created and shown to have the desired properties, showing that the precondition, P, implies wp(DOIT, Q) where Q.is the postcondition is straightforward. Unfortunately, find proper loop invariants is difficult, but loops created without an invariant or bound function often execute erroneously. <iterative_command> ::= <id_cobn> <iter1 > <iterative_command> ::= <iter1 > <iter1 > ::= <do> <iter2> <do> ::= "DO" <iter2> ::= <iter3> <od> <od> ::= "OD" <iter3> ::= <be_a_c> <iter3> ::= <be_a_c> <iter4> <iter4> ::= <box2> <iter3> <be_a_c> ::= <boolean_expression> <aπOw_command> <arrow_command> ::= <arrow> <command> Iterative command example: do (k <= s) => begin a. :- A[i,k] * B[k,j] + C[i,j] k :- k+1 end od B.5.7 Blocks A block introduces a new scope within which objects declared cannot escape (except for those darn access types). A block provides existential quantification in temporal logic. By definition, variables declared local to a single interval cannot be accessed by other intervals. This property facilitates showing "AccessOK" for the variables accessed by intervals executing concurrently. Therefore, blocks are encouraged and global variables are discouraged. model: Wj [[DECLARE X-c begin w end]] _ true iff Wstart(i) [[X]] = c and Wj[[w]] = true, rule: (3 dl,d2 : domain :_Z>[[dlj] c3[[d2]]) and (3 s : string : (dl + "." + s) - d2) and ( V : variable : V €-Z>[_dl]] => V e _Z>[[d2]]) and (X E 2>[[d2]]) and (X=c) and (P => Pl* c ) and (Ql - * > Q) { P } DECLARE X=c begin ( PI} w { 0_1 } end { Q } wp: wp(DECLARE X=c begin { PI} w [ Ql } end,Q) - wp(w,Q1 ) provided (Ql => OJ and all the uses of X are contained within the begin-end. <block> ::= <block_body> <bbck> ::= <id_colon> <block_body> <block> ::= <dec_cLp> <block__body> <block ::= <both_id_dec> <btock_body> <both_ Ldec> ::= <k__cobn> <dec_cLp> <dec_d_p> ::= <declare> <declaration_part> <block_body> ::= <begin_command> <end> <begirι_command> ::= <begin> <command> <begin> ::= "BEGIN" <declare> ::= "DECLARE" <declaration_part> ::= <block_declaration> <declaration_part> ::= <declaration_part> <block_declaratιon> <block_declaration> ::= <object_declaration> <block_declaration> ::= <number_declaration> <block_declaration> ::= <rename_clause> Example of block: declare s : integer; begin fetch_add(high_index, -1 , s) » hold[s] :- a[t] end B.5.8 Sequential Composition The semicolon in Ada is merely a statement separator. In DANCE the semicolon is an operator on commands that causes them to be executed sequentially. <sequentiaLcomposition> ::= <command> <semi_command> <semi_command> ::= <semicok>n> <command> If we have two commands, Sl and S2 then we can use the following inference rule for sequential composition: Sequential Composition Rule: { P } S1 { Q} and { O S2 [ R } { P } Sl ; S2 { R } This is analogous to dividing an interval such that both temporal predicates are true, namely the state where Q holds. where S \ and S 2 are formulas means W_[[Si ; Szl] - true iff 3 j,k : jci and kci : (Wj[[Sι]] = true) and (W [[S 2 ]] - true) and (end(j) = start(k) ) This says "the meaning of Si ; S 2 is true in the interval i if-and-only-if there exists some subintervals of i, namely j and k, such that the meaning of Si is true during j and the meaning of S 2 is true during k." Note that the two subintervals of i share the state end(j) = start(k). Essentially Si is true on the first part of the interval and S 2 is true on the last part. In terms of weakest-precondition predicate transformers (wp): wp(Sl ; S2, R) = wp(Sl , wp(S2, R) ) Example of sequential composition: fetch_add(high_index, -1 , s) hold[s] := a[t] B.5.9 Concurrent Composition Concurrent (parallel) composition of commands uses the ampersand (&) to indicate that the two commands may be executed concurrently. <paralleLcomposition> ::= <command> <amp_command> <amp_command> ::= <ampersand> <command> <ampersand> ::= "&" Concurrent Composition Rule: { PI } Sl { Ql}, { P2 } S2 { Q2 }, and AccessOK(Sl,S2) { PI and P2 } Sl & S2 { OJ and Q2 where AccessOK(Sl,S2) is defined as: forall variables Vi occurring in Sl or S2 neither Sl nor S2 changes the value of Vi, only one of Sl or S2 accesses (read or write) Vi, or Sl and S2 access Vi solely via combinable operations and the truth of Ql and Q2 is independent of the order of the combinable operations. or more formally: AccessOK(S1 ,S2) ■ Wi : Vi e SI U S2 : (Vi e lhs(S1 ) U -hs(S2) ) or (Vi e SI andVieS2) or (Vi eS2 andVie SI) or ( (Vieco(S1)andVieco(S2) ) => (co(S1) cco(S2) and co(S2) c co(S1) ) Ihs(S) _|vlv:=eeS} co(S) - { v I fetch_add(v,e,v2) e S or swap(v,e,v2) e S or ... } In terms of intervals: 3 j,k : jci and kci : (W j [[wι]] = true) and (W k [[w 2 ]] = true) and (start(j) = start(k) = start(i)) and (end(j) = end(k) = end(i)) In terms of wp: AccessOK(Sl, S2) => [wp(Sl & S2, Q) = wp(Sl, OJ and wp(S2, OJ] Concurrent composition example: k := high_index & a[i..j] := hold[i..j] B.5.10 Parallel Command The parallel command is a generalization of concurrent (parallel) composition. For each value in the discrete range a command is executed with the value replacing the identifier in the command. <parallel_command> ::= <p_c_head> <p_c_body> <p_c_head> ::= <forall_id> <in_d_r> <forall_id> ::= <forall> <identifier> <forall> ::= "FORALL" <in_d_r> ::= <in> <discrete_range> <p_c_body> ::= <do> <command> <in> ::= "IN" model: Wi [[FORALL X IN b DO w]] - true iff 3 jl...jn : jlci ... jnci : j i [[DECLARE X=xι begin w end]] = true, and Wj 2 [[DECLARE X=x 2 begin w end]] = true, and ... and W jn [[DECLARE X=x n begin w end]] = true where xi, X2, ..., x n are all the values for X that make b true, rule: ( V i : i e range : { P(i) } S(i) { OJi) } ) and (V i,j : i,j e range :AccessOK(S(i),S(j)) ) and (V i : i e range : P => P(i) ) and ( i : i e range : Q_i)) => Q { P } FORALL i IN range DO S { OJ wp: AccessOK(S) =** [ wp(FORALL i IN range DO S, OJ = wp(S[il], OJ and wp(S[i2], Q) and ... and wp(S[ik], Q) ] where il, 12, ... , ik are the values i takes in the range and S[ix] is the statement S with i replaced by the value ix, S*χ. Of course programmers should use the FORALL command often. Parallel command example: FORALL i IN j..k DO strl (i) := str2(i) B.5.11 Return Command The return command specifies the value to be returned from a function. It causes no effect on execution or termination of functions or procedures. <return_command> ::= <return> <expression> <return> ::= "RETURN" Return command example: return C B.5.12 Combinable Commands Combinable commands allow many different intervals to modify the value of the same shared variable without mutual exclusion. Direct support for combinable operations in Layered networks provides the key to efficient parallel execution with the Multitude architecture. Many nifty data structures can be accessed using combinable commands that otherwise would require parallelism-destroying mutual exclusion. <combinable_command> ::= <c_fetch_add> <combinable_command> ::= <cJetch_or> <combinable_command> ::= <c etch_and> <combinable_command> ::= <c__fetch_xoι> <combinable_command> ::= <c_fetch_swap> <cJetch_add> ::= <fetch_add> <fetch_params> <fetch_add> ::= "FETCH_ADD" <fetch_params> ::= <f_p1 > <f_p2> <f_pl > ::= <f_p3> <f_p4> <f_p2> ::= <f_p5> <rparen> <f_p3> ::= <lparen> <name> <f_p4> ::= <comma> <expression> <f_p5> ::= <comma> <k_entifier> <c etch_swap> ::= <swap> <fetch_params> <swap> ::= "SWAP" <c_fetch_or> ::= <fetch_or> <fetch_params> <fetch_or> ::= "FETCH_OR" <c_fetch_and> ::= <fetch_and> <fetch_params> <fetch_and> ::- "FETCH_AND" <c_fetch_xor> ::= <fetch_xor> <fetch_params> <fetch_xor> ::= "FETCH__XOR" The first term (name) designates the variable to fetched and modified. The second term (expresseion) is the value to be added, anded ored, xored, or swapped. The third term (identifier) designates the variable into which the returned valued is assigned. Fetch-Add model: Wi k [[FKrCH_ADD(X,eι,Yι) & FETCH_ADD(X,e 2 ,Y 2 ) & *• * & FETCH_ADD(X,e k ,Yk)]] ■ true iff there exists some ndperm(eι ,..., ej = (eι s ,..., e^) such that = true and and ... and - +ek-is)]] = true , wp of one fetch-add: wp(FETCH_ADD(X,e,Y),Q) - (QWχ +e wp of concurrent fetch-adds: wp(FETCH_ADD(X,e 1 ,Y 1 ) & FETCH_ADD(X,e ,Y2) & - & FETCH_ADD(X,eι < ,Y|<)) ■ forall ndperm(eι βk) = (eι s ,..., βks) such that (...((QYisχ) γ 2sχ +e i s)...) x χ + e1 s+e2s+ ... +eks Fetch-And model: W ik [[FETCH_AND(X,e 1 ,Y]) & FETCH_AND(X,e 2 ,Y2) & * * • & FETCH_AND(X,e ,Y k )]] » true iff there exists some ndperm(eι -«•• ev) = (eι s ,..., - k s) such that AND ei AND e 2 AND ... AND _τJ] = true and AND eis)]] = true and AND eis AND e 2s )]] - true and ... and Wik[[0( Yks=X AND eis AND e 2s AND ... AND e k -i s )]]. wp of one fetch-and: wp(FETCH_AND(X,e,Y),OJ - (QYχ)*χ A NDe wp of concurrent fetch-ands: wp(FETCH_AND(X,e 1 ,Y 1 ) & FETCH_AND(X,e 2 ,Y2) & ... & FETCH_AND(X,ek,Yk)) * forall ndperm(eι ,..., e k ) = (e* s ,..., e^) such that (•••((Q Y '-χ) Y 2sχANDe1s)--.) X XANDe1sANDe2sAND ... ANDeks Fetch-Or model: Wik[[FETCH_OR(X,eι,Yι) & FETCH_OR(X,e 2 ,Y2) & **• & FETCH_OR(X,e k ,Yk)]] - true iff there exists some ndperm(eι ,..., e k ) = (eι s ,..., e^) such that Wi k [[0(X=X OR ei OR e 2 OR ... OR ejj] = true and Wikl-CXYis-X)]] " xτue and = true and Wi k [[0( Y 3 s=X OR eis OR e 2s )]] = true and ... and Wi [[0( Y k s=X OR eis OR e 2s OR ... OR βR-u)]] = true, wp of one fetch-or: wp(FETCH_OR(X,e,Y),Q) - (Oχ)XχoRe wp of concurrent fetch-ors: wp(FETCH_OR(X,eι,Yι) & FETCH_OR(X,e 2 ,Y2) & ... & FETCH_OR(X,ek,Yk)) - forall ndperm(eι e k ) = (eι s e ks ) such that (...((QYisχ) γ 2sχoRe1s)---) X XORelsORe2sOR ... OR eks Fetch-Xor model: W ik [[FETCH_XOR(X,eι,Y ] ) & FErCH_X0R(X,e 2 ,Y2) & ... & FETCH_XOR(X,e k ,Y k )]] _ true iff there exists some ndperm(eι ■•••• e k) = (eis ,—. e ks ) such that XOR e 2 XOR ... XOR ej] - true and true and eι s )]] = true and W ik [[0(Y3s=XXOReι s XORe 2s )]] = true and ... and Wi k f Y k s=X XOR ei s XOR e 2s XOR ... XOR e^i s )]] = true, wp of one fetch-xor: wp(FETCH_XOR(X,e,Y),OJ - (0χ) χχ 0 Re wp of concurrent fetch-xors: wp(FETCH_XOR(X,e 1 ,Y 1 ) & FETCH_XOR(X,e 2 ,Y2) & ••• & FETCH_XOR(X,ek,Yk)) ■ forall ndperm(e- e) = (eι s ,..., βks) such that (...((QY i sχ)Y 2 sχ oRe1s)--J X XXORelsXORe2sXOR ... XOR eks Swap model: Wik[[SWAP(X,eι,Yι) & SWAP(X,e 2 ,Y2) & ... & SWAP(X,e k ,Y k )]] - true iff there exists some ndperm(eι •— • e k ) = (eι s ,..., e^) such that W ik [[0(X=e ks ]] = true and iittOC Yis=X)]] = true and Wikt Y2s=eis)]] - true and Wa-f Y3s-€2s)]] = true and ... and WJJ-I Yks-ek-is)]] = true. wp of one swap: wp(SWAP(X,e,Y),Q)-(QYχ) χ e wp of concurrent swaps: wp(SWAP(X,e-,Yι) & SWAP(X,e 2 ,Y2) & - & SWAP(X,e k ,Y k ) ) m forall ndperm(e- ,..., e k ) = (e* s ,..., e ks ) such that (... ((QYι -x ) Y2. e1 s ) ...) eks B.6 Subprograms Subprograms are a kind of program unit of which programs can be composed. A subprogram is a program unit whose execution is invoked by a subprogram call. There are two forms of subprogram: procedures and functions. Procedures may contain side-effects; functions may not contain side-effects. Procedures may return a value. Functions and procedures that return a value may be used in expressions. Procedures without a return value may be invoked as a command. B.6.1 Subprogram Declarations A subprogram declaration declares a procedure or a function, as indicated by the initial reserved word. <subprogram_declaration> ::= <procedure_declaration> <subprogram_declaration> ::= <function_declaration> B.6.1 .1 Procedure Declaration The specification of a procedure specifies its identifier and its formal parameters (if any). <procedure_declaration> ::= <proc_id> <procedure_declaration> ::= <proc_id> <formal_part> <proc_id> ::= <procedure> <identifier> <procedure> ::= "PROCEDURE" B.6.1 .2 Function Declaration The specification of a function specifies its designator, its formal parameters (if any), and the type of the returned value. <function_declaration> ::= <func_des> <retum_type> <function_declaration> ::= <func_des_formal_part> <return_type> <func_des> ::= <function> <designator> <func_des_formal_part> ::= <func_des> <formal_part> <function> ::= "FUNCTION" <retum_type> ::-= <return> <type_name> B.6.1 .3 Subprogram Parameters A designator that is an operator symbol is used for the overloading of an operator. <designator> ::= <identifier> <designator> ::= <operator_symbol> <operator_symbol> ::= <string_literal> The formal_part contains the parameters demanded by procedures and functions. <formal_part> ::= <lparen> <f_p1 > <f_pl> ::= <par_spec_list> <rparen> <par_spec_list> ::= <parameter_specification> <par__spec_list> ::= <parameter_specification> <f_p2> <f_p2> ::= <semicolon> <par_spec_list> Each parameter-specification has a list of identifiers, a type and possibly a mode (if this is a parameter for a procedure). Function parameters have default parameter mode "IN". <parameter_specification> ::= <idJist_colon> <mode_type> <idJist_colon> ::= <identifier_list> <colon> <mode_type> ::= <mode> <type_name> <mode_type> ::= <type_name> B.6.2 Formal Parameter Modes The value of an object is said to be "read" when this value is evaluated; it is also said to be read when one of its subcomponents is read. The value of a variable is said to be "updated" when an assignment is performed to the variable, or when the variable is used as an actual parameter of a procedure call that updates that parameter; it is also said to be updated when one of its subcomponents is updated. A procedure may modify one of its OUT parameters at any time after invocation. Since parallel execution is permitted, more than one procedure can modify the same set of variables. In some cases this is precisely what is desired, especially when using combinable operations to guard a common data structure. <mode> ::= <in> <mode> ::= <out> <mode> ::= <in> <out> <out> ::= "OUT" A formal parameter of a subprogram has one of the three following modes: IN The formal parameter is treated as constant and permits only reading of the value of the associated actual parameter. OUT The formal parameter is a variable and permits updating of the associated actual parameter. IN OUT The formal parameter is a variable and permits both reading and updating of the associated actual parameter. If no mode is explicitly given, mode IN is assumed. All formal parameters of functions must be mode IN. B.6.3 Subprogram Bodies A subprogram body specifies the meaning (behavior) of the subprogram. The meanings of DANCE functions and temporal predicates are: Wi[[f(eι, ..., e k )]] - w,[[f]] (Wi[[eι]], ..., »/_[[€*]]) Wi[[p(eι, ..., e k )]] - Wj[[p]] (Wi[[eι]], ..., W . f.eiJ]) Subprogram bodies define Wj[[f]] and Wj[[p]]. The use of the same symbol, ei, ..., e , for the actual parameters of functions and procedures is misleading. For functions ei, ..., e k must be expressions; for procedures, ei . ..., e may be variables names, and must be if its mode is IN or IN OUT. The actual parameters ei, ..., ej- are substituted for the formal parameters used within the subprogram. Parameters for functions must be mode IN so the value of ei, ..., ej- is pertainent only at start(i). The caller of any procedure must assure noninterference of updates to OUT or IN OUT parameters with other, concurrent executions. <subprogram_body> ::= <subprogram_dec_is> <block> <subprogram_dec_is> ::= <subprogram_declaration> <is> B.6.4 Subprogram Calls A subprogram call is either a procedure call command, or a function or procedure call; it invokes the execution of the corresponding subprogram body-construction of a state lattice (interval) that satisfies the subprogram. <procedure_call> ::= <identifier> <procedure_call> ::= <identifier> <actual_parameter_part> <function_call> ::= <identifier> <function_call> ::= <identifier> <actual_parameter_part> <actuaLparameter_part> ::= <lparen> <a_p_p1 > <a_p_p1 > ::= <p_a_list> <rparen> <p_a_list> ::= <parameter_association> <p_a_list> ::-= <parameter_association> <a_p_p2> <a_p_p2> ::= <comma> <p_a_list> <parameter_association> ::= <actual_parameter> <parameter_association> ::= <f_p_arrow> <actual_parameter> <f_p_arrow> ::= <formal_parameter> <arrow> <formal_parameter> ::= <identifier> <actual_parameter> ::= <expression> <actuaLparameter> ::= <deminame> <actuaLparameter> ::= <type_conversion> Each parameter association associates an actual parameter with a corresponding formal parameter. A parameter association is said to be "named" if the formal parameter is named explicitly; it is otherwise said to be positional. For a positional association, the actual parameter corresponds to the formal parameter with the same position in the formal part. For each formal parameter of a subprogram, a subprogram call must specify exactly one corresponding actual parameter. B.6.5 Function Subprograms A function is a subprogram that returns a value, the result of the function call. A function may have no side-effects; that is, no assignment of state that is either visible outside the function or between consecutive function calls is permitted. This allows parallel execution of expressions consisting entirely of functions via the Church-Rosser property. B.7 Packages A package is a kind of program unit, of which programs are composed. Packages allow the specification of groups of logically related entities. In their simplest form packages specify pools of common object and type declarations. More generally, packages can be used to specify groups of related entities such as subprograms that can be called from outside the package, while their inner workings remain concealed and protected from outside users. Packages in DANCE support the abstract data typing of Cardelli & Wegner. Their bounded universal quantification is achieved via the "required" construct. B.7.1 Package Structure A package is generally provided in two parts: a package specification and a package body. <package_declaration> ::= <pack_sl> <pack_s2> <pack_s1 > ::= <pack_id> <is_dec> <pack_id> ::= <package> <package_name> <package> ::= "PACKAGE" <is_dec> ::= <is> <is_dec> ::= <is> <declarations> <declarations> ::= <declaration> <declarations> ::= <declarations> <declarations> <pack_s2> ::= <end_package> <pack_s2> ::= <private_dec> <end_package> <private_dec> ::= <private> <declarations> <private> ::= "PRIVATE" <end_package> ::= <end> <package> The name of the package body associates it with the corresponding package specification. B.7.2 Package Specifications and Declarations The first list of declarative items of a package specification is called the "visible" part of the package. The list of declarative items (if any) after the reserved word PRIVATE is called the "private" part of the package. Private parts provide existential quantification for abstract data types created with packages. An entity declared in the private part of a package is not visible outside the package. In contrast, expanded names denoting objects declared in the visible part can be used outside the package. Direct visibility (without the package name, dot, entity name) of such entities can be achieved by means of use clauses. B.7.3 Package Bodies Entities declared within a package body are only visible within the package body itself. A variable declared in the body of a package is only visible within this body, so its value can only be changed by procedures within the package body. <package_body> ::*= <pack_b1 > <pack_b2> <pack_b1 > ::= <package_body_id> <is_dec> <package_body_id> ::= <package_body1 > <id_packtype> <package_body1> ::= <package> <body1> <id_packtype> ::= <tdentifier> <colon_pt> <colon_pt> ::= <cofon> <package_name> <package_name> ::= <identifier> <package_name> ::= <identifier> <package_name> ::= <package_name> <dot_identifier> <dot_identifier> ::= <dot> <identifier> <body1> ::="BODY" <pack_b2> ::= <body> <end_package> B.7.4 Private Types The declaration of a type as a private type in the visible part of a package serves to separate the characteristics that can be used directly by outside program units from other characteristics whose direct use is confined to the package. <private_type_delcaration> ::= <p_t_d1 > <p_t_d2> <p_t_d1> ::= <type_ident> <d_p_is> <d_p_is> ::= <is> <d_p_is> ::= <discriminant_part> <is> <p_t_d2> ::= <private> <p_t_d2> ::= <limited> <private> <limited> ::= "LIMITED" <private> ::= "PRIVATE" <deferred_constant_declaration> ::= <identifier_list> <d_c_d1 > <d_c_d1> ::= <cobn_const> <type_name> <cobn_const> ::= <colon> <constant> B.7.5 Requirement Clauses Requirement clauses allow definition of packages that are parameterized when used. The package with a requirement clause becomes a bounded, universally-quantified type. <requirement_clause> ::= <required_rtypes> <end_required> <required_rtypes> ::= <required> <rtypes> <required> ::= "REQUIRED" <end_required> ::= <end> <required> <rtypes> ::= <rtype> <rtypes> ::= <rtype> <rtypes> <ιtype> ::= <type_declaration> <rtype> ::= <subprogram_declaration> <rtype> ::= <undefined_type> <undefined_type> ::== <type_name> <id_semi> <id_semi> ::= <identifier> <semicolon> B.8 Visibility Rules The rules defining the scope of declarations and the rules defining which identifiers are "visible" at various points in the text of the program are described in this chapter. The formulation of these rules uses the notion of a "declarative region." I have formally redefined such scoping rules using domains so that parallel correctness can be proven using Owicki-Gries. Such proofs formalize informal correctness justifications such as: "My interval, which is correct in isolation, is correct with an unknown collection of other intervals because no other interval can see or modify my state variables." The visibility rules, as stated, are difficult to understand. It uses such terms as "declarative region," "occur immediately," and "outer regions." However, the visibility rules are generally intuitive: • Variables declared within a subprogram or block are not visible outside. This corresponds to domains-variable/value bindings: •2>[[d]] - { (n,v) I n e identifier, v e Tyρe(n) } _5[[dl]] c 2>[[d2]] iff 3 s : string I dl + "." + s = d2 where + is string CONcatenation. That is that domain d2 can "see" all the values in every domain dl whose name is a shortened version of d2, but there may be variables declared within a subprogram or block in d2 but not dl. • a declaration makes that object visible throughout the program structure in which it appears, • an import clause makes the contents of a package visible. An intuitive understanding of scoping (visibility) is desirable since few programmers will refer to this section when they write code. model: Wj [[DECLARE X=c begin w end]] - true iff s t artU) WQ] - c, and Wj[[w]] = true. rule: (3 dl,d2 : domain : _Z>[[dl]] c__[[d2]]) AND (3 s : string : (dl && "." s) = d2) AND (FORALL V : variable : V IN 2>[[dl]]=* IN 2>[[d2]] ) AND (X IN _Z>[[d2]]) AND (X=c) AND (P => Pl x c ) AND (Ql =* OJ { P } DECLARE X=c begin { PI} w [ Ql } end { Q } wp: wp(DECLARE X=c begin [ PI} w [ Ql } end.Q) - wp(w,Q1 ) provided (Q1 => OJ and all the uses of X are contained within the begin-end. Means of defining variables, their scope (visibility), an initial value, and type. B.8.1 Declarative Region A "declarative region" is a portion of the program text. A single declarative region is formed by the text of each of the following: • A subprogram or package declaration together with the corresponding body (if any) and subunits (if any). * A record or variant type declaration. • A renaming declaration. • A block. • A parallel command (FORALL). In each of the above cases, the declarative region is said to be "associated" with the corresponding declaration or statement. A declaration is said to occur immediately within a declarative region if this region is the innermost region that encloses the declaration, not counting the declarative region (if any) associated with the declaration itself. In the following code fragment i and k occur immediately within the parallel command. FORALL i,k IN 1..size,j..size+1 DO IF (i = j) => skip [] NOT(i = j) => IF (k > size) => x[i] :- x[i] - (Ap[i,j]/Ap[j.j])*x[j] [] NOT(k > size) => Ap[i,k] := Ap[i,k] - (Ap[i,j]/Ap[j,j])*Ap[j,k] FI FI Those program constructs (e.g. function and procedure) have declarative regions associated with them, so the declaration occurs immediately in the enclosing declarative region (e.g. package). A declaration that occurs immediately within a declarative region is said to be "local" to the region. Declarations in outer (enclosing) regions are said to be "remote" to an inner (enclosed) declarative region. A local entity is one declared by a local declaration; a remote entity is one declared by a remote declaration. B.8.2 Scope of Declarations For each form of declaration, a certain portion of program text is called the scope of the declaration. The scope of a declaration that occurs immediately within a declarative region extends from the beginning of the declaration to the end of that declarative region; this part of the scope of a declaration is called the immediate scope. Furthermore, for any of the declarations listed below, the scope of the declaration extends beyond the immediate scope. In each of these cases, the given declaration occurs immediately within some enclosing declaration, so the scope of the given declaration extends to the end of the scope of the enclosing declaration. Any declaration that occurs immediately within the visible part of a package declaration. A component declaration. A discriminant specification. A parameter declaration. B.8.3 Visibility of Identifiers and Declarations The meaning of the occurance of an identifier at a given place in the text is defined by the visibility rules. The identifiers considered include any identifier other than a reserved word. The places considered are those where a lexical element can occur. A declaration is visible at a given place in the text when, according to the visibility rules, the declaration defines a possible meaning for this occurance. In some cases more than one declaration of an object is visible at a particular use of that object. This is poor programming practice that may confuse the programmer. However the ambiguity is resolved by the compiler as follows: first the innermost scope is checked, if the declaration is not found, successively ecnlosing scopes are checked until a declaration for the object is found. If no declaration is found an error is reported B.8.4 Import Clauses Import clauses specify use of packages. Unlike Ada, each item within a package must be explicitly named. This allows separate compilation of packages and checking when those packages are linked. <import_-Clause> ::= <i_c1 > <i_c2> <Lc1 > ::= <Lc3> <i_c4> <Lc3> ::= <import> <identifier_list> <import> ::= "IMPORT" <Lc4> ::= <from> <package_name> <Lc4> ::= <from> <i_c5> <Lc5> ::= <package_name> <actual_parameter_part> <from> ::= "FROM" <Lc2> ::= <end_import> <Lc2> ::= <r_clauses> <end_import> <r_clauses> ::= <rename_clause> <r_clauses> ::= <r_clauses> <r_clauses> <end_import> ::= <end> <import> Use of a rename clause is encouraged. Otherwise each object from the package must have the package name and a period preceeding the object name. <rename_clause> ::= <rename_name> <as_id> <rename_name> ::= <rename> <name> <rename> ::= "RENAME" <as_id> ::= <as> <identifier> <as> ::= "as" The following example imports the "Sort" package and supplies the necessary types and functions: IMPORT sort -the procedure name FROM ParallelSort: Sort -the package name ( rt -> employee_record, -parameters required ar => employee_array, x <= • => (string <= string) ) RENAME sort AS employee_sort END IMPORT B.9 Assertions Assertions in DANCE programs support formal specification of behavior and proof of program correctness. A programming logic is a formal system that supports the assertional (axiomatic) approach to developing and analyzing programs. (Much of this section was borrowed from Concurrent Programming. Gregory R. Andrews, Benjamin/Cunimings, 1991, chapter 1.) A programming logic includes predicates that characterize program states and relations that are the effect of program execution. This section summarizes relevant aspects of formal logical systems, propositional logic, and predicate calculus. <assertion> ::= <lbrace> <ass1 > <assl > ::= <ass2> <rbrace> <lbrace> ::= "{" <rbrace> ::= "}" <ass2> ::= <predicate> <ass2> ::= <id_colon> <predicate> B.9.1 Predicates Predicates are the building blocks for assertions. Assertions use only first- order predicates. <predicate> ::= <true> <predicate> ::= <false> <predicate> ::= <lparen> <pred1 > <pred1 > ::= <pred2> <rparen> <pred2> ::= <predicate> <log_op_pred> <log_op_pred> ::= <logical_operator> <predicate> <predicate> ::= <not> <predicate> <predicate> ::= <logic_variable> <predicate> ::= <existential_quantifier> <predicate> ::= <universal_quantifier> <predicate> ::= <counting_quantifier> <predicate> ::= <summing_quantifier> <predicate> ::= <product_quantifier> <predicate> ::= <relational_expression> <logic_variable> ::= <deminame> <logic_variable> ::= <function_call> When dealing with sets of values, we will often want to assert that some or all values of a set satisfy a property. We could do this informally, but quantified expressions provide a more precise and compact method. B.9.2 Existential Quantifier The existential quantifier, EXISTS, allows one to assert that some element of a set satisfies a property. It appears in expressions of the form: (EXISTS bi, ..., b n : R : P ) The bj following the quantifier are new variables called bound variables; R is a formula that specifies the set of values (the range) of the bound variables; P is a predicate. The interpretation of existential quantification is true if for some combination of values of the bound variables (within range R) , predicate P is true. <existential_quantifier> ::= <e_q1 > <predicate> <e_q1 > ::= <e_q2> <colon_p_r_colon> <e_q2> ::= <exists> <identifier_list> <exists> ::= "EXISTS" <colon_p_r_cok)n> ::= <colon> <p_r_colon> <p_r_colon> ::= <p_range> <colon> <p_range> ::= <range> <p_range> ::= <relation> B.9.3 Universal Quantifier The universal quantifier, FORALL, allows one to assert that all elements of a set satisfy a property. Universal quantification takes the form: FORALL bi, ..., b n : R : P Again, the bj are bound variables, R specifies the range of values of bound variables, and P is a predicate. The interpretation of universal quantification is true if P is true for all combinations of the values of the bound variables. <universal_quantifier> ::= <u_q1 > <predicate> <u_q1> ::= <u_q2> <colon_p_r_colon> <u_q2> ::= <forall> <identifier_list> The counting quantifier, NUM, returns the number of times the predicate is true for all combinations of bound variable values. <counting_quantifier> ::= <c_q1 > <predicate> <c_q1 > ::= <c_q2> <colon__p_r_cofon> <c_q2> ::= <num> <identifier_list> <num> ::= "NUM" The summing quantifier adds the values of the expression for all combinations of bound variable values. <summing_quantifier> ::= <s_q1 > <expression> <s_q1> ::= <s_q2> <colon_p_r_colon> <s_q2> ::= <sum> <identifier_list> <sum> ::= "SUM" The product quantifier multiplies the values of the expression for all combinations of bound variable values. <product_quantifier> ::= <p_q1 > <expression> <p_q1 > "= <p_q2> <colon_p_r_colon> <p_q2> ::= <prod> <identifier_list> <prod> ::= "PROD" In a quantified expression, the scope of a bound variable is the expression itself. Thus, a quantified expression is like a nested block. Many quantified expressions will also reference other variables. Thus, confusion could result if the name of a bound variable conflicts with that of another variable. Prof. Richter's mathematical logic class spent lots of energy exploring "free" and "bound" variables. Goofy things happen when a previously "free" variable is inadvertantly captured through stupid naming. For now, the "capture" problem will be avoided. CONCLUSIONS, RAMIFICATIONS, AND SCOPE OF INVENTION The model of computation of the DANCE programming language along with the special hardware features of the Multitude architecture allow efficient use of many computational elements. The scope of this invention is the process of state transformation that is DANCE/Multitude concurrent computation. Every execution of every DANCE program falls within the scope of this invention. The scope of the invention should be determined not by the embodiment espoused, but by the appended claims and their legal equivalents. Though constructing state-lattices to satisfy temporal formulae is how the concurrency is accomplished, many other things must be done right to achieve efficiency. Logical systems for data types, domains, assertions, and correctness proofs must be integrated with the temporal logic. DANCE programs will not run efficiently without the special features of the Multitude architecture. The operating system that allocates memory, sets and checks page and segment attributes, swaps pages for virtual memory, schedules and allocates work for processors, dynamically links and type- checks relocatable machine code, and handles input/output relys and supports on special features of the Multitude architcture, the DANCE language and its compiler. The ramifications of cheap, scalable computing power will dramatically change modern society. People from advertising directors to radiologists will use their workstations or personal computers for information entry and display, relying on a "big machine" out on the net to do in a few minutes what would take days on a workstation. Computational power utilities will spring up, offering competive and falling prices. Users will be able to set their priority level and pay accordingly. Programs will be proven correct and guaranteed not to have bugs. Multitude Corporation will license and certify several hardware vendors to make Multitude architecture computers, and will license and certify software vendors to sell DANCE compilers and programs written in DANCE.

Claims: CLAJDMS 1 A method of operating a plurality of computing means comprising the step of constructing a lattice to satisfy a temporal logic formula, the lattice having a plurality of nodes, each node representing a state that is a value of a program variable at a given moment, the nodes being interconnected by directed arcs, each arc representing a change in state affected by a computation associated with that arc 2 The method of claim 1 wherein the state values are elements of strong types, including either constructed types or types defined using existential and bounded universal quantification 3 The method of claim 1 wherein the visibility of variables in a particular state is restricted to domains 4 The method of claim 1 in which the states may occupy computing means located in disparate geographical locations utilizing remote procedure call semantics 5 The method of claim 1 and further including the step of providing correctness of the lattice construction automatically 6 The method of claim 1 and further including the step of providing correctness of the lattice construction manually 7 A computing system comprising (a) a plurality of computing means operatively coupled by an interconnection network means for communicating with one another, the network means including means for performing global memory access routing, request combining and response decombining, (b) each of the computing means having an instruction set including combinable operations and write back capability, (c) the computing means further having a virtual memory address translation mechanism for enforcing access constraints for segments or pages of memory, said constraints being selected from the group consisting of (i) memory shared but no caching allowed, (ii) memory protected and temporary exclusive caching with processor indicated write back, (iii) discretionary memory access controls with exclusive demanded authorization user access rights, 5 (iv) mandatory memory access controls in which a user may read values at an assigned security level or lower, write values at said assigned security level or higher, and only invoke combinable operations on values at the assigned security level. [received by the International Bureau on 16 June 1997 (16.06.97); original claims 1 and 3-6 amended ; remaining claims unchanged (2 pages)] 1. A method of operating a plurality of computing means comprising the steps of: (a) transforming a temporal logic formula into machine executable form; b (b) loading the machine executable form i r the temporal lpgic formula and data to e manipulated into said plurality of computing means; and (c) constructing a lattice to satisfy said temporal logic formula, the lattice having a plurality of nodes, each node representing a state value that is a value of a program variable at a given moment, the plurality of nodes being interconnected by C directed arcs, each directed arc representing a change in state affected by a computation associated with that arc. 2. The method of claim 1 wherein the state values are elements of strong types, including either constructed types or types defined using existential and bounded universal quantification. 5 3. The method of claim 1 wherein visibility of said program variables tn a particular state is restricted to omains. 4. The method of claim 1 in which the state values may occupy computing means located in disparate geographical locations utilizing remote procedure call semantics 0 5. The method of claim 1 and farther including the step of proving correctness of the lattice construction automatically. 6. The method of claim 1 and further including the step of proving correctness of the lattice construction manually, 7. A computing system comprising: 5 (a) a plurality of computing means operatively coupled by an interconnection network means for communicating with one another, the network means including means for performing global memory access routing, request combining and response decombintng; 0 (b) each of the computing means having an instruction set including combinable operations and write back capability, (c) the computing means wither having a virtual me or address translation mechanism foi enl icing access constraints for segments or pages of memory, said constraims being selected from the group consisting of: (i) memory shared but no caching allowed, 5 (ii) memory protected and temporary exclusive caching with processor indicated write back. (Hi) discretionary memory access controls with exclusive demanded authorization user access rights, (iv) mandatory memory access controls in which a user may 10 read values at an assigned security level or lower, write values at said assigned security level or higher, and only invoke combinable operations on values at _he assigned security level. STATEMENT UNDER ARTICLE 19 The replacement pages 233-234 containing claims 1-7 are intended to replace the originally submitted pages 233-234 containing claims 1-7. The claims contain modifications intended to clarify and more concisely describe the invention, including ditterenσβB from references cited. These claims are consistent with the claims of the application to which this application claims priority. The newly presented amended claims are believed to he within the scope of the invention described in the specification and, in addition to capturing the inventive concept more concisely, the amended claims enhance the definition over the references cited in category "X" in the International Search Report. The newly presented amended claims are believed to be -within the scope of the invention described in the specification and, in addition, and are believed to capture the inventive concept more concisely.

Inventors: LARSON BRIAN R, LARSON, BRIAN, R.
Applicants: LARSON BRIAN R, MULTITUDE CORP, LARSON, BRIAN, R., MULTITUDE CORPORATION
Application year: 1997
Main classification: G06F15
IPC codes: G06F9