on the convergence domains of the p cyclic sor abstract for the solution of the linear system ax b where a is block p cyclic the block sor iterative method is to be considered suppose that the block jacobi iteration matrix b associated with a has eigenvalues whose p th powers are all real of the same sign the problem of the determination of the precise convergence domains of the sor method in case a is also consistently ordered was solved by hadjidimos li and varga by using the schur cohn algorithm the same convergence domains were later recovered by other approaches too specifically wild and niethammer and also noutsos independently used hypocycloidal curves in this manuscript it is assumed that a is not consistently ordered but a t is by using the schur cohn algorithm we successfully determine not only i the precise sor convergence domains but also ii intervals for b the spectral radius of b that directly imply that the optimal value of the sor relaxation factor is equal to 1 in this work new results are obtained some well known ones are recovered or confirmed and a number of theoretical examples are investigated further it is worth noting that among the new results we derived something not quite expected specifically in many cases there exist pairs b for which the sor method associated with the matrix a we consider converges while the corresponding sor for the p cyclic consistently ordered matrix a t does not

an iterative substructuring algorithm for equilibrium equations the topic of iterative substructuring methods and more generally domain decomposition methods has been extensively studied over the past few years and the topic is well advanced with respect to first and second order elliptic problems however relatively little work has been done on more general constrained least squares problems or equivalent formulations involving equilibrium equations such as those arising for example in realistic structural analysis applications the potential is good for effective use of iterative algorithms on these problems but such methods are still far from being competitive with direct methods in industrial codes the purpose of this paper is to investigate an order reducing preconditioned conjugate gradient method proposed by barlow nichols and plemmons for solving problems of this type the relationships between this method and nullspace methods such as the force method for structures and the dual variable method for fluids are examined convergence properties are discussed in relation to recent optimality results for varga s theory ofp cyclic sor we suggest a mixed approach for solving equilibrium equations consisting of both direct reduction in the substructures and the conjugate gradient iterative algorithm to complete the computations

on the optimum relaxation factor associated with p cyclic matrices assume that the matrix coefficient of the nonsingular linear system ax b belongs to the class of the generalized consistently ordered p 9 9 matrices where p and 9 are relatively prime it is well known that under the additional assumption that the pth powers of the eigenvalues of the jacobi matrix t associated with a are nonnegative nonpositive the problem of determining the optimum relaxation factor that maximizes the asymptotic convergence rate of the successive overrelaxation method for the solution of ax b has been solved in many cases thus in the works by young by varga and by nichols and fox the problem has been solved in the nonnegative case for any p 9 in the nonpositive case in view of the work by kredell by niethammer de pillis and varga by galanis hadjidimos and noutsos and by wild and niethammer the corresponding problem seems to be more difficult it has been solved only for 9 p 1 the present work is a contribution towards the solution of the problem in the latter case in particular we study the case 9 1 p 3 with detailed results for

reconstruction of markov random fields from samples some observations and algorithms markov random fields are used to model high dimensional distributions in a number of applied areas much recent interest has been devoted to the reconstruction of the dependency structure from independent samples from the markov random fields we analyze a simple algorithm for reconstructing the underlying graph defining a markov random field on n nodes and maximum degree d given observations we show that under mild nondegeneracy conditions it reconstructs the generating graph with high probability using theta d epsilon 2 delta 4 log n samples where epsilon delta depend on the local interactions for most local interactions epsilon delta are of order exp o d our results are optimal as a function of n up to a multiplicative constant depending on d and the strength of the local interactions our results seem to be the first results for general models that guarantee that the generating model is reconstructed furthermore we provide explicit o n d 2 epsilon 2 delta

over and underrelaxation for linear systems with weakly cyclic jacobi matrices of index p abstract d young s results from 1954 concerning the application of the successive overrelaxation sor method to linear systems ax b with matrices possessing property a were generalized by r s varga in 1959 to systems with jacobi matrices ja which are weakly cyclic of index p if the eigenvalues of jpa are nonnegative then for the optimal parameter opt there holds 1 opt 1 1 p 1 here a different proof of varga s results is given and it is shown that 1 1 p 1 opt 1 holds if the eigenvalues of jpa are nonpositive further exact intervals for those parameters which yield convergence of the sor method are given in terms of chebyshev polynomials

the hopfield model on a sparse erdos renyi graph we analyze the storage capacity of the hopfield model on a sparse g n p random graph we show that it is proportional to pn in the entire regime where the corresponding random graph is asymptotically connected and for all value of c 0 03

extrapolated gauss seidel i and sor methods for least squares problems recently special attention has been given in the literature to the problems of accurately computing the least squares solution of very largescale over determined systems of linear equations which occur in geodetic applications in particular it has been suggested that one can solve such problems iteratively by applying the block sor successive overrelaxation and egs1 extrapolated gauss seidel 1 plus semi iterative methods to a linear system with coefficient matrix 2 cyclic or 3 cyclic the comparison of 2 block sor and 3 block sor was made in 1 and showed that the 2 block sor is better in 6 the authors also proved that 3 block egs1 si is better than 3 block sor here we first show that the 2 block dj double jacobi si gs si and egs1 si methods are equivalent and all of them are equivalent to the 3 block egs1 si method then we prove that the composite methods and 2 block sor have the same asymptotic rate of convergence but the former has a better average rate of convergence finally numerical experiments are reported and confirm that the 3 block egs1 si is better than the 2 block sor

a young eidson s type algorithm for complex p cyclic sor spectra abstract in a recent work of ours we have solved the problem of the minimization of the spectral radius of the iteration matrix of a p cyclic successive overrelaxation sor method for the solution of the linear system ax b when the matrix a is block p cyclic consistently ordered for what is known as the one point problem for any p 3 particular cases of the one point problem were solved by young varga kjellberg kredell russell and others in the present work we develop a theory using the results of our previous one and solve first the two point problem special cases of which were solved by wrigley eiermann niethammer ruttan noutsos and others secondly we generalize and extend our theory to cover the many point problem and develop a young eidson s type algorithm for its solution as possible application areas we mention among others the best block p cyclic repartitioning for the sor method are the solution of large scale systems arising in queueing network problems in markov analysis

a survey of the iterative methods for the solution of linear systems by extrapolation relaxation and other techniques abstract for the iterative solution of linear systems of algebraic equations ax b 1 with a c n n x b c n and det a 0 numerous methods exist although a classification of them seems not to be possible one may note that the first step for the construction of an iterative method usually begins with a splitting of a in 1 thus a is written as a m n where det m 0 and m is easily inverted so that 1 is equivalent to x tx c 2 t m 1n c m 1 the discussion will be restricted to the so called linear stationary iterative methods although some nonstationary ones will be mentioned and emphasis will be given to those which fall into the categories of extrapolation relaxation and similar ones for this it will be assumed that the spectrum t of t in 2 is contained in a well defined compact region r whose complement with respect to the complex plane is simply connected and 1 r under these assumptions and for each specific class of methods described each time an attempt will be made to present the optimum one the optimum is the one out of the class of methods for which the sequence of vectors yielded converges asymptotically to the unique solution of 1 as fast as possible

symmetric successive overrelaxation methods for rank deficient linear systems in this paper we develop symmetric successive overrelaxation symmetric sor or ssor methods for finding the least square solution of minimal norm to the linear system ax b where a is an mxn matrix of rank r the methods are obtained by first augmenting the system to a block 4x4 consistent system the augmented coefficient matrix is then split by a subproper ssor splitting we state and prove some theorems and by some numerical examples we show the number of iterations for ssor is less than sor and accelerated overrelaxation methods for finding the least square solution of minimal norm to the linear system

an algorithmic approach for the analysis of extrapolated iterative schemes applied to least squares problems abstract the problem of determining the optimal values of extrapolated iterative schemes as they apply to the solution of large scale least squares problems is addressed here based on algebraic and geometric eigenvalue properties of the accelerated gauss seidel ags we devise a simple algorithmic procedure which successfully yields the optimal values of the extrapolated ags eags comparisons with the optimal sor scheme reveal that the two optimal schemes strongly compete numerical examples are used to demonstrate our results

optimal successive overrelaxation iterative methods for p cyclic matrices we consider linear systems whose associated block jacobi matricesj p are weakly cyclic of indexp in a recent paper pierce hadjidimos and plemmons 13 proved that the block two cyclic successive overrelaxation sor iterative method is numerically more effective than the blockq cyclic sor method 2 2 finally as a counterpart to the result of pierce hadjidimos and plemmons we show that under this more general assumption the two cyclic sor method is not always superior to theq cyclic sor method 2 q p

optimal p cyclic sor for complex spectra abstract we consider the successive overrelaxation sor method for the solution of a linear system ax b when the matrix a has a block p p partitioned p cyclic form and its associated block jacobi matrix jp is weakly cyclic of index p following the pioneering work by young and varga in the 1950s many researchers have considered various cases for the spectrum jp and have determined optimal values for the relaxation factor 0 2 so that the sor method converges as fast as possible after the most recent work on the best block p cyclic repartitioning and that on the solution of large scale systems arising in queueing network problems in markov analysis the optimization of the convergence of the p cyclic sor for more complex spectra jp has become more demanding here we state the one point problem for the general p cyclic complex sor case the existence and the uniqueness of its solution are established by analyzing and developing further the theory of the associated hypocycloidal curves for the determination of the optimal parameter s an algorithm is presented and a number of illustrative numerical examples are given

bistable monostable transition in the ising model on two connected complex networks in this paper we investigate the behavior of the ising model on two sparsely connected complex networks the networks have the topology of a random graph or barabasi and albert scale free networks we extend our previous analysis and show that a bistable monostable phase transition occurs in such systems during this transition the magnetization undergoes a discontinuous jump we calculate the critical temperature analytically for regular random graphs and study a more general case using an iterative map corresponding to mean field dynamics the calculations are confirmed by numeric simulations based on monte carlo approach

towards the determination of the optimal p cyclic ssor abstract suppose that a c n n is a block p cyclic consistently ordered matrix and let b and s denote the block jacobi and the block symmetric successive overrelaxation ssor iteration matrices associated with a respectively extending previous work by hadjidimos and neumann the present authors have determined the exact regions of convergence of the ssor method in the b plane for any p 3 under the further assumption that the eigenvalues of b p are real of the same sign in the present work the investigation goes on further several questions are raised and among others the problem of the determination of the optimal regions of convergence in the spirit of niethammer and varga as well as that of the optimal relaxation factor are examined

on primitive bch codes with unequal error protection capabilities presents a class of binary primitive bch codes that have unequal error protection uep capabilities the authors use a previous result on the span of their minimum weight vectors to show that binary primitive bch codes containing second order punctured reed muller rm codes of the same minimum distance are binary cyclic uep codes the values of the error correction levels for this class of binary luep codes are estimated

multi group binary choice with social interaction and a random communication structure a random graph approach abstract we construct and analyze a random graph model for discrete choice with social interaction and several groups of equal size we concentrate on the case of two groups of equal sizes and we allow the interaction strength within a group to differ from the interaction strength between the two groups given that the resulting graph is sufficiently dense we show that with probability 1 the average decision in each of the two groups is the same as in the fully connected model in particular we show that there is a phase transition if the interaction among a group and between the groups is strong enough the average decision per group will either be positive or negative and the decision of the two groups will be correlated we also compute the free energy per particle in our model

on the solution of singular linear systems of algebraic equations by semiiterative methods for a square matrixt n n where i t is possibly singular we investigate the solution of the linear fixed point problemx tx c by applying semiiterative methods sim s to the basic iterationx0 n xk tck 1 c k 1 such problems arise if one splits the coefficient matrix of a linear systemax b of algebraic equations according toa m n m nonsingular which leads tox m 1nx m 1b tx c even ifx tx c is consistent there are cases where the basic iteration fails to converge namely ift possesses eigenvalues 1 with 1 or if 1 is an eigenvalue oft with nonlinear elementary divisors in these cases and also ifx tx c is incompatible we derive necessary and sufficient conditions implying that a sim tends to a vector hat x which can be described in terms of the drazin inverse of i t we further give conditions under which hat x is a solution or a least squares solution of i t x c

the analysis ofk step iterative methods for linear systems from summability theory using the theory of euler methods from summability theory we investigate general iterative methods for solving linear systems of equations in particular for a given euler method a regions of the complex plane is determined such that ak step iterative method converges if the eigenvalues of an iteration operatort are contained ins for a givens optimal methods are described and upper and lower bounds are derived for the associated asymptotic rate of convergence special attention is given to two step methods with complex parameters

convergence of a direct iterative method for large scale least squares problems abstract in 1975 chen and gentleman suggested a 3 block sor method for solving least squares problems based on a partitioning scheme for the observation matrix a into a a 1 a 2 where a1 is square and nonsingular in many cases a1 is obvious from the nature of the problem this combined direct iterative method was discussed further and applied to angle adjustment problems in geodesy where a1 is easily formed and is large and sparse by plemmons in 1979 recently niethammer de pillis and varga have rekindled interest in this method by correcting and extending the sor convergence interval the purpose of our paper is to discuss an alternative formulation of the problem leading to a 2 block sor method for this formulation it is shown that the resulting direct iterative method always converges for sufficiently small sor parameter in contrast to the 3 block formulation formulas for the optimum sor parameter and the resulting asymptotic convergence factor based upon a2a 11 2 are given furthermore it is shown that this 2 cyclic block sor method always gives better convergence results than the 3 cyclic one for the same amount of work per iteration the direct part of the algorithm requires only a sparse matrix factorization of a1 our purpose here is to establish theoretical convergence results in line with the purpose of the recent paper by niethammer de pillis and varga practical considerations of choosing a1 in certain applications and of estimating the resulting a2a 11 2 will be addressed elsewhere

a note on two block sor methods for sparse least squares problems abstract we compare two recently proposed block sor methods for the solution of large least squares problems with the conjugate gradient algorithm for solving the specially preconditioned normal equations by proving that all three methods are based on the same krylov sequence we show that the conjugate gradient algorithm is generally preferable to the two sor approaches

successive overrelaxation methods for solving the rank deficient linear squares problem abstract we develop successive overrelaxation sor methods for finding the least squares solution of minimal norm to the system ax b where a is an m n matrix of rank r the methods are obtained by first augmenting the system to a block 4 4 consistent system of linear equations the augmented coefficient matrix is then split by a subproper sor splitting an interval for the relaxation parameter in which the subproper sor iteration matrix is semiconvergent is determined along with the optimal relaxation parameter which minimizes the modulus of the controlling eigenvalue of the sor matrix since the scheme computes at first only a solution to the augmented system it is subsequently shown how to transform such a solution to the unique solution of minimal 2 norm analysis of the practical implementation of the algorithms developed here will be given elsewhere

a parallelizable sor like method systems with plus shaped and linear spectra abstract the classical sor method replaces an invertible cyclic linear system in b x f with a family of systems i n b x f where for some optimal scalar b the spectrum of b b is circular the eigenvalues of b are linearly placed falling on the real line moreover the spectral radius of b b is less than that of b similarly this paper explores a tensor mechanism which converts more general systems in b x f to equivalent systems i 2n b x f where the spectrum of b may be plus shaped the spectrum of b is circular and x col 0 x as in the sor theory the spectral radius of b is less than that of b among other things we recapture the results of the sor theory improve on some known results and extend our theory directly to matrices in b which need not be symmetric yet have real plus and minus valued eigenvalues

superior convergence domains for the p cyclic ssor majorizer abstract thus far for an n n complex nonsingular matrix a the symmetric successive overrelaxation ssor majorizing operator has been used to establish convergence properties of the ssor method mostly in the case where a is an h matrix in this paper we use actually a similarity transformation of the ssor majorizer to investigate convergence properties of the block ssor method when a is a block p cyclic matrix let ja denote the block jacobi method and let j a we establish regions in the plane where s a q a l a here s a is the block ssor iteration operator associated with a l a is the block successive overrelaxation sor iteration operator associated with a and q a is a convenient similarity transformation of the majorizing operator for s a of special interest to us are the values of for which the above inequality holds for the corresponding values of the relaxation parameter a 2 1 the latter being an important quantity in the sor ssor theory for h matrices

accelerate overrelaxation methods for rank deficient linear systems in this paper we apply accelerated overrelaxation aor methods to find the least square solution of minimal norm to the linear systemay bwhere a c m x n is a matrix of rank r and b c m we first augment the system to a block 4x4 consistent system and then split the augmented coefficient matrix by aor subproper splitting intervals for the two relaxation parameters where the aor iteration matrix is semiconvergent are presented also we provide a method to compute the least square solution of minimal norm to the system

block sor methods for rank deficient least squares problems many papers have discussed preconditioned block iterative methods for solving full rank least squares problems however very few papers studied iterative methods for solving rank deficient least squares problems miller and neumann 1987 proposed the 4 block sor method for solving the rank deficient problem here a 2 block sor method and a 3 block sor method are proposed to solve such problem the convergence of the block sor methods is studied the optimal parameters are determined comparison between the 2 block sor method and the 3 block sor method is given also

convergence of block iterative methods applied to sparse least squares problems abstract recently special attention has been given in the mathematical literature to the problems of accurately computing the least squares solutions of very large scale overdetermined systems of linear equations such as those arising in geodetical network problems in particular it has been suggested that one solve such problems iteratively by applying the block sor successive overrelaxation iterative method to a consistently ordered block jacobi matrix that is weakly cyclic of index 3 here we obtain new results theorem 1 giving the exact convergence and divergence domains for such iterative applications it is then shown how these results extend and correct the literature on such applications in addition analogous results theorem 2 are given for the case when the eigenvalues of the associated block jacobi matrix are nonnegative

determination of the optimal value of relaxation parameter in symmetric sor method for rectangular coefficient matrices in this paper we obtain the optimal value of relaxation parameter in symmetric sor ssor method by ssor method we find the least square solution of minimal norm to the linear system ax b where a is an m by n matrix

block cyclic sor for markov chains with p cyclic infinitesimal generator we introduce a new application of p cyclic iterations for arbitrary p 2 the block sor method for the computation of the steady state distribution of finite markov chains that possess p cyclic infinitesimal generators is considered it is shown that convergence in a sense more general than the usual may be obtained even if the sor iteration violates the usual conditions for semiconvergence necessary and sufficient conditions for convergence in this extended sense are derived they are then applied in the case where the pth power of the associated jacobi matrix of the system to be solved possesses only nonnegative eigenvalues exact convergence intervals and the optimal values are derived for this case in addition to the usual optimal in the interval 1 p p 1 other values that yield convergence in the extended sense are found to achieve the same optimal convergence rate numerical tests indicate that small perturbations of around the optimal value affect the convergence factor much less if these newly introduced optimal values are used

convergence domains of the ssor method for a class of generalized consistently ordered matrices abstract the functional equation relating the eigenvalues of the block symmetric successive overrelaxation ssor iteration matrix with those of the block jacobi iteration matrix found by chong and cai 1985 is used in order to obtain precise domains of convergence of the block ssor iteration method associated with a class of generalized consistently ordered gco k p k matrices a p 2 k 1 p 1 we show that the domain of convergence depends only on the relaxation parameter the spectral radius of the block jacobi iteration matrix and the value of l k p unlike the case k p k 1 p 1 p 3 which was studied in an earlier paper by the authors 1989 beyond certain critical values of l the domain of convergence does not grow monotonically with l and manifests various sorts of behavior however as in the case k p k 1 p 1 it is shown that the intersection of the convergence domains taken over all pairs p k coincides with the exact domain of convergence of the point ssor iteration method associated with nonsingular h matrices a that is matrices which can be diagonally scaled to being strictly diagonally dominant

successive overrelaxation sor and related methods abstract covering the last half of the 20th century we present some of the basic and well known results for the sor theory and related methods as well as some that are not as well known most of the earlier results can be found in the excellent books by varga matrix iterative analysis prentice hall englewood cliffs nj 1962 young iterative solution of large linear systems academic press new york 1971 and berman and plemmons nonnegative matrices in the mathematical sciences siam philadelphia pa 1994 while some of the most recent ones are given in the bibliography of this paper in this survey both the point and the block sor methods are considered for the solution of a linear system of the form ax b where a c n n and b c n 0 some general results concerning the sor and related methods are given and also some more specific ones in cases where a happens to possess some further property e g positive definiteness l m h matrix property p cyclic consistently ordered property etc

block aor iterative schemes for large scale least squares problems the problem of accelerating the convergence rate of iterative schemes as they apply to the solution of large scale least squares problems by means of different splittings is addressed here new convergence results together with explicit expressions for the optimal factors and their corresponding spectral radii are derived for certain schemes from the block accelerated overrelaxation aor family it is shown that for a class of least squares problems the proposed optimal block iterative scheme converges unconditionally and asymptotically faster than the optimal block successive overrelaxation sor method while in all other cases the two schemes are competitive numerical examples are used to illustrate our results

optimal p cyclic sor our purpose in this paper is to extend p cyclic sor theory to consistent singular systems and to apply the results to the solution of large scale systems arising em e g in queueing network problems in markov analysis markov chains and queueing models lead to structured singular linear systems and are playing an increasing role in the understanding of complex phenomena arising in computer communication and transportation systems for certain important classes of singular problems we develop a convergence theory for p cyclic sor and show how to repartition for optimal convergence results by kontovasilis plemmons and stewart on the concept of convergence of sor in an em extended sense are rigorously analyzed and applied to the solution of periodic markov chains with period p 2

k step iterative methods for solving nonlinear systems of equations let n n be frechet differentiable and let the equation 1 x phi left x right have at least one fixed point we considerk step stationary iterative methods 2 y m mu 0 phi left y m 1 right mu 1 y m 1 mu k y m k m geqq k with 0 1 k 1 using results for an affine mapping n n it is proven that 2 may converge locally even in cases where the usual iterationx m x m 1 belonging to 1 diverges these results are extended to nonstationary methods of type 2 and to cyclic mappings

optimality relationships forp cyclic sor the optimality question for blockp cyclic sor iterations discussed in young and varga is answered under natural conditions on the spectrum of the block jacobi matrix in particular it is shown that repartitioning a blockp cyclic matrix into a blockq cyclic form q p results in asymptotically faster sor convergence for the same amount of work per iteration as a consequence block 2 cyclic sor is optimal under these conditions