The algorithm structure and V-matrices

This is a brief survey devoted to K-matrices, which can be used not only for describing the structure of an algorithm but can also provide some additional information about it. We show the way K-matrices appear in such applications as the roundoff error analysis, the fast computation of gradients, and the complexity evaluation of linear algorithms. A perfectly simple procedure for deriving a fast algorithm for computing gradients is proposed. We also discuss some lower bounds for linear algorithms based on a determinant inequality obtained in the paper. Structural properties of algorithms form the basis for research in several directions that appear at first glance to be distantly related. We can name here at least three such directions: the roundoff error analysis, the fast computation of gradients, and the complexity evaluation of linear algorithms. So far, these areas have not been considered interrelated because there were no cross-references from one area to another. The goal of this paper is to show that there is a natural unifying basis for these areas, namely F-matrices. It should be emphasized that K-matrices fully reflect the informational structure of an algorithm. Our intention was to write a survey of K-matrices and their application to the analysis of algorithms. In Section 1 we give the definition of F-matrices and a sketch of their use in the roundoff error analysis. Sections 2 and 3 are devoted to some applications of F-matrices that make use of different augmentations of F-matrices. In Section 2 we consider a very simple procedure for deriving a fast algorithm for computing gradients. In Section 3 we primarily discuss lower bounds for linear algorithms that are based on a new determinant inequality, Appendix contains the proof of the inequality. 1. FMATRICES AND ROUNDOFF ERRORS Let us consider a finite directed graph G(F,£), where F is the set of its vertices and E is the set of its arcs, and assign a vector pe e (D, which is treated later as a vector variable, to every arc e = (u,v) in the set E. Let inp(v) denote the set of vectors assigned to the arcs that enter a vertex v and let out(v) designate the set of vectors assigned to the arcs that leave the vertex v. We use entr(F) and quit(F) to denote subsets of F that contain vertices with no arcs coming in and going out, respectively, and let int(F) denote all the other vertices of F. Obviously, we have inp(v) = 0 for v € entr(F) and out(v) = 0 for v € quit(F). We thus obtain F= entr(F) u int(F) u quit(F) (1.1) where the set inp(v) contains just one arc for any v e quit(F). With every v e int(F) we associate the mapping /· inp(v)-out(v). (1.2) ' Institute of Numerical Mathematics, Academy of Sciences of Russia, Moscow 117334, Russia. 180 E.E. Tyrtyshnikov The mapping f calculates the values of the vector variables from out(v) for any 'admissible' values of the vector variables from inp(v). If the input values are not admissible, an abend is reported. In what follows, the algorithm A is understood to be the graph G = (V,E) equipped with the vector variables pe, eeE, and the mappings fv, v e int(K). The algorithm is easily seen to calculate the values of the output variables p, v\, v e quit(K), given some admissible values assigned to p, yy u e entr(K). Note that the same variable can be assigned to different arcs leaving a vertex. For simplicity we shall assume that all the arcs leaving a vertex are assigned to the same vector variable. Thus, u e int(F) 'computes' the value of a single vector variable p, . =pu and 'transmits' this value to all the corresponding vertices. We also assume that all fv are differentiable. Now we are in a position to give a definition of the F-matrix associated with the algorithm A. Let us number the vertices of entr(F) with integers from 1 to n and the vertices of int(F) with integers from n + 1 to n + N so that if i~u and / ** v we have / < / for any (u,v)e£. The F-matrix is now defined to be a block Nx(n+N) matrix (1-3) such that (1) the size of ΜΛ. is variables at the output of / and /; (2) wu=In, Ι = Λ + 1 , . ,ΑΖ+ΛΤ; (3) η. = 0 for ι Φ ] and (/,/) £ E\ (4) Mjy-cOJ/a/j. for (/,/)€£. The last condition needs clarification. Suppose that where r. and r· are the dimensions of the vector