average consensus on general strongly connected digraphs we study the average consensus problem of multi agent systems for general network topologies with unidirectional information flow we propose two linear distributed algorithms deterministic and gossip respectively for the cases where the inter agent communication is synchronous and asynchronous in both cases the developed algorithms guarantee state averaging on arbitrary strongly connected digraphs in particular this graphical condition does not require that the network be balanced or symmetric thereby extending previous results in the literature the key novelty of our approach is to augment an additional variable for each agent called surplus whose function is to locally record individual state updates for convergence analysis we employ graph theoretic and nonnegative matrix tools plus the eigenvalue perturbation theory playing a crucial role

numerical integration of non linear two point boundary value problems using iterated deferred corrections i abstract in several recent papers the idea has been proposed of using implicit runge kutta formulae for the approximate integration of non linear two point boundary value problems these formulae often take a very special form so that computational efficiencies are obtained in certain well defined circumstances in this paper we examine the relative merits of these different classes of formulae and discuss their efficient implementation in a deferred correction framework a theoretical comparison of these formulae is important firstly because they call for different computational tasks to be performed and secondly because they are not always applicable to exactly the same classes of problems the most efficient of these formulae are then compared with methods based on finite differences and with methods based on spline collocation for some simple smooth test functions on the basis of operation counts and numerical experimentation it is concluded that runge kutta methods can be generally competitive with the other methods considered

the conditioning of boundary value problems in transferable differential algebraic equations the well conditioning of boundary value problems for transferable differential algebraic equations is shown to be related as in the case of explicit ordinary differential equations to the boundary conditions and the green function in this case other quantities related to the well transferability and the movement speed of the null space of the leading coefficient also appear in the bound the condition number of the related multiple shooting matrix is also bounded in terms of the quantities mentioned above

dynamics of vertex reinforced random walks we generalize a result from volkov ann probab 29 2001 66 91 and prove that on a large class of locally finite connected graphs of bounded degree g and symmetric reinforcement matrices a ai j i j g the vertex reinforced random walk vrrw eventually localizes with positive probability on subsets which consist of a complete d partite subgraph with possible loops plus its outer boundary we first show that in general any stable equilibrium of a linear symmetric replicator dynamics with positive payoffs on a graph g satisfies the property that its support is a complete d partite subgraph of g with possible loops for some d 1 this result is used here for the study of vrrws but also applies to other contexts such as evolutionary models in population genetics and game theory next we generalize the result of pemantle probab theory related fields 92 1992 117 136 and benaim ann probab 25 1997 361 392 relating the asymptotic behavior of the vrrw to replicator dynamics this enables us to conclude that given any neighborhood of a strictly stable equilibrium with support s the following event occurs with positive probability the walk localizes on s s where s is the outer boundary of s and the density of occupation of the vrrw converges with polynomial rate to a strictly stable equilibrium in this neighborhood

the factorization method for the numerical solution of two point boundary value problems for linear ode s the theoretical analysis and computational implementation of factorization based methods for the numerical solution of linear boundary value problems for ordinary differential equations are presented the methods are optimal with respect to certain clearly defined criteria numerical examples show the effectiveness of a general code based on the factorization method

split step orthogonal spline collocation methods for nonlinear schrodinger equations in one two and three dimensions abstract split step orthogonal spline collocation osc methods are proposed for one two and three dimensional nonlinear schrodinger nls equations with time dependent potentials firstly the nls equation is split into two nonlinear equations and one or more one dimensional linear equations commonly the nonlinear subproblems could be integrated directly and accurately but it fails when the time dependent potential cannot be integrated exactly in this case we propose three approximations by using quadrature formulae but the split order is not reduced discrete time osc schemes are applied for the linear subproblems in numerical experiments many tests are carried out to prove the reliability and efficiency of the split step osc ssosc methods solitons in one two and three dimensions are well simulated and conservative properties and convergence rates are demonstrated we also apply the ways of solving the nonlinear subproblems to the split step finite difference ssfd methods and the time splitting spectral tssp methods and the approximate ways still work well finally we apply the ssosc methods to solve some problems of bose einstein condensates

distributed subgradient methods for convex optimization over random networks we consider the problem of cooperatively minimizing the sum of convex functions where the functions represent local objective functions of the agents we assume that each agent has information about his local function and communicate with the other agents over a time varying network topology for this problem we propose a distributed subgradient method that uses averaging algorithms for locally sharing information among the agents in contrast to previous works on multi agent optimization that make worst case assumptions about the connectivity of the agents such as bounded communication intervals between nodes we assume that links fail according to a given stochastic process under the assumption that the link failures are independent and identically distributed over time possibly correlated across links we provide almost sure convergence results for our subgradient algorithm

a natural approach to the numerical integration of riccati differential equations this paper introduces a new class of methods which we call mobius schemes for the numerical solution of matrix riccati differential equations the approach is based on viewing the riccati equation in its natural geometric setting as a flow on the grassmannian of m dimensional subspaces of an n m dimensional vector space since the grassmannians are compact differentiable manifolds and the coefficients of the equation are assumed continuous there are no singularities or intrinsic instabilities in the associated flow the presence of singularities and numerical instabilities is an artifact of the coordinate system but since mobius schemes are based on the natural geometry they are able to deal with numerical instability and pass accurately through the singularities a number of examples are given to demonstrate these properties

distributed multi agent optimization with state dependent communication we study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents we assume that each agent knows only his own local objective function and constraint set and exchanges information with the other agents over a randomly varying network topology to update his information state we assume a state dependent communication model over this topology communication is markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents we study a projected multi agent subgradient algorithm under state dependent communication the state dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors we first show that the multi agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one under some assumptions on the stepsize sequence we provide convergence rate bounds on a disagreement metric between the agent estimates our bounds are time nonhomogeneous in the sense that they depend on the initial starting time despite this we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence

conditioning of collocation matrices and discrete green s functions the conditioning of spline collocation matrices is considered in the case of a boundary value problem for an mth order equation for bases with continuity built in which corresponds to most current implementations of splines for solving odes the collocation matrix a is related to the corresponding multiple shooting matrix for a first order system this gives the explicit form for a 1 as a discrete green function we briefly discuss the implications for mesh selection and how our results relate to previous ones for finite differences

vertex reinforced random walk on arbitrary graphs vertex reinforced random walk vrrw defined by pemantle is a random process in a continuously changing environment which is more likely to visit states it has visited before we consider vrrw on arbitrary graphs and show that on almost all of them vrrw visits only finitely many vertices with a positive probability we conjecture that on all graphs of bounded degree this happens with probability 1 and provide a proof only for trees of this type we distinguish between several different patterns of localization and explicitly describe the long run structure of vrrw which depends on whether a graph contains triangles or not while the results of this paper generalize those obtained by pemantle and volkov for z 1 ideas of proofs are different and typically based on a large deviation principle rather than a martingale approach

attracting edge property for a class of reinforced random walks using martingale techniques and comparison with the generalized urn scheme it is shown that the edge reinforced random walk on a graph of bounded degree with the weight function w k k 1 traverses crosses a random attracting edge at all large times if the graph is a triangle the above result is in agreement with a conjecture of sellke

invariant imbedding the box scheme and an equivalence between them we show for general linear two point boundary value problems that the box scheme and invariant imbedding are equivalent in the sense that a specific algorithm for solving the difference equations is valid if and only if an appropriate imbedding is valid a key element in the equivalence is the role played by pivoting to solve the linear equations and switching to get a finite imbedding part of the equivalence proof shows that imbedding is the completion of the box scheme multiple imbeddings are introduced and a lower bound for the critical length problem is obtained

a comparison of some global methods for solving two point boundary value problems a numerical comparison is made between three codes which implement global methods for solving two point boundary value problems the codes compared are a collocation method colsys a deferred correction method pasva 3 and a deferred correction method based on mono implicit runge kutta formulae hagron although the difficulty of conducting a fair comparison of complex codes is widely recognised it is nevertheless of great practical importance to try to gauge the relative performance of different codes and the three codes considered in this paper are particularly appropriate for numerical comparison by choosing a test set which includes both smooth and increasingly stiff problems some interesting characteristics of the codes are noted and some recommendations regarding their use are made

the solution of nonlinear schrodinger equations using orthogonal spline collocation abstract an orthogonal spline collocation semidiscretization previously applied in the numerical solution of the cubic schrodinger equation is extended to the solution of schrodinger equations which possess general power nonlinearities and nonlinear terms that contain derivatives the accuracy of the method is examined as well as its ability to conserve approximations to certain invariants associated with the theoretical solution the results of numerical experiments are presented in which the integration in time is performed using a routine from a software library particular attention is paid to the value and limitations of conservation as an indicator of accuracy

magnus integrators for solving linear quadratic differential games we consider magnus integrators to solve linear quadratic n player differential games these problems require to solve backward in time non autonomous matrix riccati differential equations which are coupled with the linear differential equations for the dynamic state of the game to be integrated forward in time we analyze different magnus integrators which can provide either analytical or numerical approximations to the equations they can be considered as time averaging methods and frequently are used as exponential integrators we show that they preserve some of the most relevant qualitative properties of the solution for the matrix riccati differential equations as well as for the remaining equations the analytical approximations allow us to study the problem in terms of the parameters involved some numerical examples are also considered which show that exponential methods are in general superior to standard methods

spline based computational technique for linear singularly perturbed boundary value problems a fourth order difference scheme via cubic spline in compression for singularly perturbed two point boundary value problems of the form ey p x y q x y r x y a a y b bis presented the convergence analysis is given finally computational efficiency of the method is verified through numerical examples

decentralized optimization over time varying directed graphs with row and column stochastic matrices in this article we provide a distributed optimization algorithm termed as tv mathcal ab that minimizes a sum of convex functions over time varying random directed graphs contrary to the existing work the algorithm we propose does not require eigenvector estimation to estimate the non mathbf 1 perron eigenvector of a stochastic matrix instead the proposed approach relies on a novel information mixing approach that exploits both row and column stochastic weights to achieve agreement toward the optimal solution when the underlying graph is directed we show that tv mathcal ab converges linearly to the optimal solution when the global objective is smooth and strongly convex and the underlying time varying graphs exhibit bounded connectivity i e a union of every c consecutive graphs is strongly connected we derive the convergence results based on the stability analysis of a linear system of inequalities along with a matrix perturbation argument simulations confirm the findings in this article

strong stability of compact discrete boundary value problems via exact discretizations this paper is concerned with strong stability inequalities with respect to discrete and continuous sobolev norms for general linear mth order two point boundary value problems and their discretizations by compact finite difference schemes the well known stability of compact consistent approximations on irregular grids is demonstrated here by a direct connection with the strong stability inequality of the continuous problem the connection is made by means of exact discretizations which are compact schemes that are exactly satisfied by the true solution of the original problem the analysis involves certain generalized splines related to the green s function for the problem

a riccati transformation method for solving linear bvps ii computational aspects in part i of this paper siamj numer anal 25 1988 pp 1055 1073 a riccati transformation method for solving linear boundary value problems for ordinary differential equations is presented here the implementation aspects of this method are discussed in particular a double sweep integration process which includes a strategy for solving a matrix riccati differential equation and forming a continuous representation of its solution is described the efficacy of the method is demonstrated on a number of singular perturbation problems particular attention is paid to the choice of scaling for the variables and its effect on the solution method the computational experience while not based upon a refined implementation indicates that the riccati method may well prove to be competitive with the other general methods

edge reinforced random walk on a ladder we prove that the edge reinforced random walk on the ladder z x 1 2 with initial weights a 3 4 is recurrent the proof uses a known representation of the edge reinforced random walk on a finite piece of the ladder as a random walk in a random environment this environment is given by a marginal of a multicomponent gibbsian process a transfer operator technique and entropy estimates from statistical mechanics are used to analyze this gibbsian process furthermore we prove spatially exponentially fast decreasing bounds for normalized local times of the edge reinforced random walk on a finite piece of the ladder uniformly in the size of the finite piece

a note on conditioning stability and collocation matrices the concept of a well conditioned bvp as considered in recent literature essentially corresponds to the concept of a uniformly stable ivp but this terminology suggests a shift from absolute to relative quantities which makes it more difficult to understand the conditioning of numerical discretizations especially for higher order differential operators in this note we wish to make it clear that the notion of well conditioning of bvps is not always a direct analogue of the u the condition number of a discretization of a well conditioned bvp generally grows unboundedly when higher accuracy is desired because it contains a discretization of a differential operator this condition number gets large particularly rapidly if a direct discretization of a higher order differential operator is attempted for collocation an implementation which uses a local runge kutta solution representation produces a much better conditioning because the repeated numerical differentiation is avoided instead the highest derivative is approximated locally by a discontinuous piece wise polynomial and local integrations yield lower solution derivatives upon imposing the collocation equations the highest derivative approximation can be locally eliminated resulting in a sophisticated discretization of the corresponding first order system

stability of a three point scheme for linear second order singularly perturbed bvps with turning points in this paper we study stability properties of a numerical method applied to linear second order perturbed boundary value problems with boundary or interior layers the method consists of a three point difference scheme with variable stepsize the stability of which may be investigated by studying that of an lu factorization for the coefficient matrix of a suitable tridiagonal system we also propose a mesh selection strategy for treating boundary and interior layers which furnishes a good mesh on which the three point scheme is stable

stable methods to solve the impedance matrix for radially inhomogeneous cylindrically anisotropic structures a stable approach for integrating the impedance matrix in cylindrical radial inhomogeneous structures is developed and studied a stroh like system using the time harmonic displacement traction state vector is used to derive the riccati matrix differential equation involving the impedance matrix it is found that the resulting equation is stiff and leads to exponential instabilities a stable scheme for integration is found in which a local expansion is performed by combining the matricant and impedance matrices this method offers a stable solution for fully anisotropic materials which was previously unavailable in the literature several approximation schemes for integrating the riccati equation in cylindrical coordinates are considered exponential magnus taylor series lagrange polynomials with numerical examples indicating that the exponential scheme performs best the impedance matrix is compared with solutions involving buchwald potentials in which the material is limited to piecewise constant transverse isotropy lastly a scattering example is considered and compared with the literature

on spline basis selection for solving differential equations the suitability of b splines as a basis for piecewise polynomial solution representation for solving differential equations is challenged two alternative local solution representations are considered in the context of collocating ordinary differential equations hermite type and lmonomial both are much easier and shorter to implement and somewhat more efficient than b splines a new condition number estimate for the b splines and hermite type representations is presented one choice of the hermite type representation is experimentally determined to produce roundoff errors at most as large as those for b splines the monomial representation is shown to have a much smaller condition number than the other ones and correspondingly produces smaller roundoff errors especially for extremely nonuniform meshes the operation counts for the two local representations considered are about the same the hermite type representation being slightly cheaper it is concluded that both representations are preferable

a well posed shooting method for transferable dae s consider a tpbvp for transferable nonlinear dae s in general the shooting equation has a singular jacobian a multiple shooting method which has a nonsingular jacobian and also produces consistent initial values for the integration is presented the estimation of the condition of the jacobian shows the well posedness of the method some illustrative examples are given

the order of accuracy for deferred corrections using uncentered end formulas a result of christiansen and russell siam j numer math 35 1980 pp 21 33 on the order of accuracy of an iterated deferred corrections algorithm of lentini and pereyra siam j numer math 14 1977 pp 91 111 is improved at the same time providing a realistic example for a theoretical framework developed by the author for proving accuracy results for deferred corrections

the conditioning of linear boundary value problems investigation is made into the sensitivity of solutions of linear boundary value problems to perturbations of the boundary condition we derive a useful quantity to decide for well or ill conditioning from this it is deduced which kind of requirements the boundary conditions should meet in order to have a well conditioned problem it is shown that this quantity can fruitfully be used to explain why e g the multiple shooting technique is stable in contrast to the single shooting one for certain well posed problems having solutions with different growth behavior the results are sustained by a number of examples

vertex reinforced jump processes on trees and finite graphs we study the continuous time process on the vertices of the b ary tree which jumps to each nearest neighbor vertex at the rate of the time already spent at that vertex times plus 1 where is a positive constant we show that for fixed b 1 if is large enough the process is transient and if is close enough to zero it is recurrent related results for some other graphs and trees are also proved

numerical integration of the differential riccati equation and some related issues abstract in this paper the problem of direct numerical integration of differential riccati equations dres and some related issues are considered the dre is an expression of a particular change of variables for a linear system of ordinary differential equations the error that an approximate solution of the dre induces on the original variables of the system is considered and it is related to geometrical properties of the system itself sharp bounds on the global error for the computed solution are also given in terms of local errors and geometrical properties of the original system nonstiff and stiff dres of unsymmetric and symmetric type are considered a useful matrix interpretation is given for many integration schemes such as the backward differentiation formulas bdf when applied to the dre this allows the matrix structure of the problem to be exploited in particular for stiff dres the resulting strategy allows for a saving of three orders of magnitude with respect to the standard reform

fast direct solvers for piecewise hermite bicubic orthogonal spline collocation equations fast direct methods are proposed for the solution of linear systems arising when orthogonal spline collocation with piecewise hermite bicubics is employed for the approximate solution of poisson s

on a fourth order finite difference method for singularly perturbed boundary value problems we present a fourth order finite difference method for singularly perturbed one dimensional reaction diffusion problem the problem is discretized using a bakhvalov type mesh we give a uniform convergence with respect to the perturbation parameter numerical examples are presented which demonstrate computationally the fourth order of the method

the close relationships between methods for solving two point boundary value problems the well conditioning of a boundary value problem is shown to be related to bounding two quantities one involving the boundary conditions and the other involving the green s function in the case of a solution dichotomy they are related to known stability results these results easily explain why difficulties arise using superposition and reduced superposition it is shown how multiple shooting overcomes these difficulties by relating its matrix conditioning to the underlying boundary value problem some factorization methods for the multiple shooting method are considered these show the close relationships between multiple shooting the stabilized march and invariant imbedding and one factorization leads to an efficient new way to implement each of these three methods many of the results apply for collocation and finite difference methods too finally it is shown both theroetically and computationally when matrix compactification can lead to difficulty

orthogonal cubic spline collocation method for the extended fisher kolmogorov equation a second order splitting combined with orthogonal cubic spline collocation method is formulated and analysed for the extended fisher kolmogorov equation with the help of lyapunov functional a bound in maximum norm is derived for the semidiscrete solution optimal error estimates are established for the semidiscrete case finally using the monomial basis functions we present the numerical results in which the integration in time is performed using radau 5 software library

the riccati transformation in the solution of boundary value problems the multiple shooting method and stabilized march have long been advocated for solving boundary value problems even though certain difficulties arise for problems with rapidly increasing and decreasing fundamental solution components we interpret these difficulties in terms of stability of the underlying problem and describe the stabilized march in terms of associated projections onto subspaces relating the riccati transformation to a specific implementation of the stabilized march leads naturally to an alternative method which involves only stable initial value problems and overcomes major difficulties of these other methods the potential of the method is demonstrated in some numerical examples

a computational study of finite element methods for second order linear two point boundary value problems a computational study of five finite element methods for the solution of a single second order linear ordinary differential equation subject to general linear separated boundary conditions is described in each method the approximate solution is a piecewise polynomial expressed in terms of a b spline basis and is determined by solving a system of linear algebraic equations with an almost block diagonal structure the aim of the investigation is twofold to determine if the theoretical orders of convergence of the methods are realized in practice and to compare the methods on the basis of cost for a given accuracy in this study three parametrized families of test problems containing problems of varying degrees of difficulty are used the conclusions drawn are rather straightforward collocation is the cheapest method for a given accuracy and the easiest to implement also for solving the linear algebraic equations the use of a special purpose solver which takes advantage of the structure of the equations is advisable 1 introduction finite element methods for two point boundary value problems for a single ordinary differential equation may take many forms the simplest type of method is probably collocation 3 where an approximate solution is sought in some finite space subject to the constraint that it satisfy the differential equation at certain specified points this type of method has been applied very successfully in the case of mixed order systems of boundary value problems see for example 1 other methods can be described based on the standard l2 galerkin approximation 9 or on a combination of this and the collocation approach 7 13 25 in addition different weak formulations of the boundary value problem lead to two other techniques the hx and galerkin methods 12 14 15 17 how one would apply these methods to a system of differential equations with the exception of the collocation case is not immediately clear in addition it is not apparent what the relative advantages are of these various methods even when applied to a single equation in this paper we restrict our attention to the case of a single linear differential equation our aim is to investigate numerically five finite element techniques for a second order linear two point boundary value problem with separated boundary conditions of a general nature in particular we consider i the organization of the methods ii the treatment of the boundary conditions

an alternating direction implicit method for orthogonal spline collocation linear systems an alternating direction implicit method is analyzed for the solution of linear systems arising in high order tensor product orthogonal spline collocation applied to some separable second order linear elliptic partial differential equations in rectangles on annxn partition with jordan s selection of the acceleration parameters the method requireso n 2 ln 2 n arithmetic operations to produce an approximation whose accuracy in theh 1 norm is that of the collocation solution

high order finite element methods for singularly perturbed elliptic and parabolic problems we develop a framework for applying high order finite element methods to singularly perturbed elliptic and parabolic differential systems that utilizes special quadrature rules to confine spurious effects such as excess diffusion and nonphysical oscillations to boundary and interior layers this approach is more suited for use with adaptive mesh refinement and order variation techniques than other problem dependent methods quadrature rules developed for two point convection diffusion and reaction diffusion problems are used with finite element software to solve examples involving ordinary and partial differential equations numerical artifacts are confined to layers for all combinations of meshes orders and singular perturbation parameters that were tested radau or lobatto quadrature used with the finite element method to solve respectively convection and reaction diffusion problems provide the benefits of the specialized quadrature formulas and are simpler to implement

decentralized quasi newton methods we introduce the decentralized broyden fletcher goldfarb shanno d bfgs method as a variation of the bfgs quasi newton method for solving decentralized optimization problems decentralized quasi newton methods are of interest in problems that are not well conditioned making first order decentralized methods ineffective and in which second order information is not readily available making second order decentralized methods impossible d bfgs is a fully distributed algorithm in which nodes approximate curvature information of themselves and their neighbors through the satisfaction of a secant condition we additionally provide a formulation of the algorithm in asynchronous settings convergence of d bfgs is established formally in both the synchronous and asynchronous settings and strong performance advantages relative to existing methods are shown numerically

numerical methods in seismology a survey is given of the numerical methods used for computing synthetic seismograms in particular the properties of finite differences finite elements integral transforms and ray tracing are described each of these methods is capable of giving good results for its own range of frequencies and wave numbers thus for complete seismograms one should be prepared to use all of the available tools

numerical solution of eigenvalue problems for linear boundary value odes abstract interrelationships between several popular approaches for solving eigenvalue problems for linear boundary value odes are given for linear eigenvalue problems the popular methods can be interpreted in a common framework this leads us to propose and justify alternative strategies the choice of numerical methods used here is motivated by the desire to solve eigenvalue problems for stiff odes in particular we consider a one step global method spline collocation and two initial value methods riccati and continuous orthonormalization to solve the orr sommerfeld equation a comparison of results for these methods using various implementation strategies is given

continuous time vertex reinforced jump processes we study the continuous time integer valued process which jumps to each of its two nearest neighbors at the rate of one plus the total time the process has previously spent at that neighbor we show that the proportion of the time before t which this process spends at integers j converges to positive random variables v j which sum to one and whose joint distribution is explicitly described we also show

second order splitting combined with orthogonal cubic spline collocation method for the kuramoto sivashinsky equation in this paper a second order splitting method is applied to the kuramoto sivashinsky equation and then an orthogonal cubic spline collocation procedure is employed to the approximate resulting system this semidiscrete method yields a system of differential algebraic equations daes of index 1 error estimates in l 2 and l c norms are obtained for the semidiscrete approximation for the time discretization the time integrator radau5 is used the results of numerical experiments are presented to validate the theoretical findings

linear system solvers for boundary value odes abstract we investigate the stability properties of several linear system solvers for solving boundary value odes we consider the compactification algorithm gaussian elimination with row partial pivoting and a qr algorithm applied to linear systems arising from solving bvps for which the matrix is block bidiagonal except for bordering along the last n rows and columns we will particularly compare auto s original linear solver an lu decomposition with partial pivoting and our implementation of the analogous qr algorithm to auto two other factors the underlying continuation strategy and mesh selection strategy may affect the stability of the linear system solver for ode continuation codes as well and are also discussed in our numerical investigations

convergence rates of distributed nesterov like gradient methods on random networks we consider distributed optimization in random networks where n nodes cooperatively minimize the sum i 1n fi x of their individual convex costs existing literature proposes distributed gradient like methods that are computationally cheap and resilient to link failures but have slow convergence rates in this paper we propose accelerated distributed gradient methods that 1 are resilient to link failures 2 computationally cheap and 3 improve convergence rates over other gradient methods we model the network by a sequence of independent identically distributed random matrices w k drawn from the set of symmetric stochastic matrices with positive diagonals the network is connected on average and the cost functions are convex differentiable with lipschitz continuous and bounded gradients we design two distributed nesterov like gradient methods that modify the d ng and d nc methods that we proposed for static networks we prove their convergence rates in terms of the expected optimality gap at the cost function let k and k be the number of per node gradient evaluations and per node communications respectively then the modified d ng achieves rates o logk k and o logk k and the modified d nc rates o 1 k2 and o 1 k2 where 0 is arbitrarily small for comparison the standard distributed gradient method cannot do better than 1 k2 3 and 1 k2 3 on the same class of cost functions even for static networks simulation examples illustrate our analytical findings

boundary value problems and dichotomic stability since the conditioning of a boundary value problem bvp is closely related to the existence of a dichotomic fundamental solution i e where one set of modes is increasing and a complementary set is decreasing it is important to have discretization methods that conserve this dichotomy property the conditions this imposes on such a method are investigated in this paper they are worked out in more detail for scalar second order equations the central difference scheme and for linear first order systems as well for the latter type both one step methods including collocation and multistep methods those that may be used in multiple shooting are examined

dextra a fast algorithm for optimization over directed graphs this paper develops a fast distributed algorithm termed dextra to solve the optimization problem when n agents reach agreement and collaboratively minimize the sum of their local objective functions over the network where the communication between the agents is described by a directed graph existing algorithms solve the problem restricted to directed graphs with convergence rates of o ln k sqrt k for general convex objective functions and o ln k k when the objective functions are strongly convex where k is the number of iterations we show that with the appropriate step size dextra converges at a linear rate o tau k for 0 given that the objective functions are restricted strongly convex the implementation of dextra requires each agent to know its local out degree simulation examples further illustrate our findings

on optimal shooting intervals we develop an adaptive multiple shooting strategy which is nearly optimal with respect to cpu time since the costs of integration are the most important components in this we investigate in some detail how the gridpoints are chosen by an adaptive integration routine we use this information to find out where the shooting points have to be selected we also show that our final strategy is stable in the sense that rounding errors can be kept below a given tolerance finally we pay attention to the question how the need for memory can be minimized

numerical solution of two point boundary value problems abstract this paper presents an original formulation of two point boundary value and eigenvalue problems expressed as a system of first order equations the fundamental difference between the new method and other methods based on a first order approach is the introduction of conditions of an integral character to supplement the simultaneous set of first order equations which are hence never regarded as an initial value problem the consideration of integral conditions leads to establish a class of linear multipoint schemes for the numerical solution of boundary value problems for ordinary differential equations furthermore the global character of the integral conditions nonlocality combined with the block structure of the system of algebraic equations allow dealing with stiff problems by means of the classical procedure of iterative refinement introduced by wilkinson the properties of the numerical schemes are illustrated by the solution of linear and nonlinear problems and by the accurate and efficient determination of some eigensolutions of a difficult problem of hydrodynamic stability the proposed method is conceptually simpler and numerically more convenient than existing initial value methods while still retaining all the advantages of a formulation based on a first order system

numerical methods for singular perturbation problems consider the two point boundary value problem for a stiff system of ordinary differential equations an adaptive method to solve these problems even when turning points are present is discussed

edge and vertex reinforced random walks with super linear reinforcement on infinite graphs in this paper we introduce a new simple but powerful general technique for the study of edge and vertex reinforced processes with super linear reinforcement based on the use of order statistics for the number of edge respectively of vertex traversals the technique relies on upper bound estimates for the number of edge traversals proved in a different context by cotar and limic ann appl probab 2009 for finite graphs with edge reinforcement we apply our new method both to edge and to vertex reinforced random walks with super linear reinforcement on arbitrary infinite connected graphs of bounded degree we stress that unlike all previous results for processes with super linear reinforcement we make no other assumption on the graphs for edge reinforced random walks we complete the results of limic and tarr es ann probab 2007 and we settle a conjecture of sellke technical report 94 26 purdue university 1994 by showing that for any reciprocally summable reinforcement weight function w the walk traverses a random attracting edge at all large times for vertex reinforced random walks we extend results previously obtained on z by volkov ann probab 2001 and by basdevant schapira and singh ann probab 2014 and on complete graphs by benaim raimond and schapira alea 2013 we show that on any infinite connected graph of bounded degree with reinforcement weight function w taken from a general class of reciprocally summable reinforcement weight functions the walk traverses two random neighbouring attracting vertices at all large times

on dichotomy and well conditioning in two point boundary value problems we investigate the close relationships between the stability constants and the growth behavior of the fundamental matrix to the general boundary value problem we also show that conditioning number is the right criterion to indicate possible error amplification of the perturbed boundary conditions

collocation for singular perturbation problems i first order systems with constant coefficients the application of collocation methods for the numerical solution of singularly perturbed ordinary differential equations is investigated collocation at gauss radau and lobatto points is considered for both initial and boundary value problems for first order systems with constant coefficients particular attention is paid to symmetric schemes for boundary value problems these problems may have boundary layers at both interval ends br our analysis shows that certain collocation schemes in particular those based on gauss or lobatto points do perform very well on such problems provided that a fine mesh with steps proportional to the layers width is used in the layers only and a coarse mesh just fine enough to resolve the solution of the reduced problem is used in between ways to construct appropriate layer meshes are proposed of all methods considered the lobatto schemes appear to be the most promising class of methods as they essentially retain their usual superconvergence power for the smooth reduced solution whereas gauss legendre schemes do not br we also investigate the conditioning of the linear systems of equations arizing in the discretization of the boundary value problem for a row equilibrated version of the discretized system we obtain a pleasantly small bound on the maximum norm condition number which indicates that these systems can be solved safely by gaussian elimination with scaled partial pivoting

accurate estimates for the fundamental solutions of discrete boundary value problems it is investigated how the solutions of a discretized ode can be related to solutions of the ode by using a perturbation analysis based on boundary value formulations in particular it is indicated how one can find estimates for discrete modes that correspond to certain continuous modes these estimates are fairly sharp in a relative sense as is shown by some examples as a consequence one can deduce that certain properties of the one solution space like a dichotomy carry over to the other

a preconditioned conjugate gradient method for nonselfadjoint or indefinite orthogonal spline collocation problems we study the computation of the orthogonal spline collocation solution of a linear dirichlet boundary value problem with a nonselfadjoint or an indefinite operator of the form lu sum a ij x u x ix j sum b i x u x i c x u we apply a preconditioned conjugate gradient method to the normal system of collocation equations with a preconditioner associated with a separable operator and prove that the resulting algorithm has a convergence rate independent of the partition step size we solve a problem with the preconditioner using an efficient direct matrix decomposition algorithm on a uniform n n partition the cost of the algorithm for computing the collocation solution within a tolerance epsilon is o n 2 ln n ln epsilon

on dichotomy and well conditioning in bvp we investigate the relationships between the stability bounds of the problem on the one hand and the growth behaviour of the fundamental solution on the other hand it is shown that if these stability bounds are moderate i e if the problem is well conditioned then the homogeneous solution space is dichotomic which means that it can be split into a subspace of nondecreasing and a complementary subspace of nonincreasing modes this is done by carefully examining the green s functions if these exhibit an exponential behaviour then the solution space is also exponentially dichotomic on the other hand we also show that exponential dichotomy implies moderate stability constants i e well conditioning from this it follows that both concepts are more or less equivalent

stable parallel elimination for boundary value odes a parallelizable and vectorizable algorithm for solving linear algebraic systems arising from two point boundary value odes is described the method is equivalent to gaussian elimination with row partial pivoting applied to a certain column reordered version of the usual almost block diagonal coefficient matrix we present analytical and numerical evidence to show that the algorithm is stable in most circumstances results from implementation on a shared memory multiprocessor and a vector processor are given the approach can be extended to handle problems with multipoint and integral conditions or with algebraic parameters

on the convergence of decentralized gradient descent consider the consensus problem of minimizing f x sum i 1 n f i x where x in mathbb r p and each f i is only known to the individual agent i in a connected network of n agents to solve this problem and obtain the solution all the agents collaborate with their neighbors through information exchange this type of decentralized computation does not need a fusion center offers better network load balance and improves data privacy this paper studies the decentralized gradient descent method a nedic and a ozdaglar ieee trans automat control 54 2009 pp 48 61 in which each agent i updates its local variable x i in mathbb r n by combining the average of its neighbors with a local negative gradient step alpha nabla f i x i the method is described by the iteration x i k 1 gets sum j 1 n w ij x j k alpha nabla f i x i k text for each agent i where w ij is nonzero only if i and j are neighbors or i j and the matrix

on best possible order of convergence estimates in the collocation method and galerkin s method for singularly perturbed boundary value problems for systems of first order ordinary differential equations the collocation method and galerkin method using parabolic splines are considered special adaptive meshes whose number of knots is independent of the small parameter of the problem are used unimprovable estimates in the l norm are obtained for the galerkin method these estimates are quasioptimal while for the collocation method they are suboptimal

decoupling and stability of algorithms for boundary value problems the ordinary differential equations occurring in linear boundary value problems characteristically have both stable and unstable solution modes therefore a stable numerical algorithm should avoid both forward and backward integration of solutions on large intervals it is shown that most methods like multiple shooting collocation invariant imbedding and difference methods derive their stability from the fact that they all decouple the continuous or the discrete problem sooner or later for instance when solving a linear system this decoupling is related to the dichotomy of the ordinary differential equations in fact it turns out that the inherent initial value instability is an important prerequisite for a stable utilization of the decoupled representations from which the solutions are computed how this stability is related to the use of the boundary conditions is also investigated

on the computation of solutions of boundary value problems on infinite intervals for solutions of linear boundary value problems defined on lbrack0 infty one has to study the stable or bounded solution manifold a characterization of these manifolds is investigated here a multiple shooting type algorithm is then developed to compute such solutions this algorithm is fully adaptive and also covers problems where the ode matrix does not tend to a limit as is usually assumed if the unstable manifold consists only of exponentially growing solutions if the latter manifold also contains polynomially growing solutions an extrapolation type approach is suggested the theory is illustrated by a number of examples

multilevel preconditioners for non self adjoint or indefinite orthogonal spline collocation problems efficient numerical algorithms are developed and analyzed that implement symmetric multilevel preconditioners for the solution of an orthogonal spline collocation osc discretization of a dirichlet boundary value problem with a non self adjoint or an indefinite operator the osc solution is sought in the hermite space of piecewise bicubic polynomials it is proved that the proposed additive and multiplicative preconditioners are uniformly spectrally equivalent to the operator of the normal osc equation the preconditioners are used with the preconditioned conjugate gradient method and numerical results are presented that demonstrate their efficiency

on the transience of processes defined on galton watson trees we introduce a simple technique for proving the transience of certain processes defined on the random tree mathcal g generated by a supercritical branching process we prove the transience for once reinforced random walks on mathcal g that is a generalization of a result of durrett kesten and limic probab theory related fields 122 2002 567 592 moreover we give a new proof for the transience of a family of biased random walks defined on mathcal g other proofs of this fact can be found in ann probab 16 1988 1229 1241 and ann probab 18 1990 931 958 as part of more general results a similar technique is applied to a vertex reinforced jump process a by product of our result is that this process is transient on the 3 ary tree davis and volkov probab theory related fields 128 2004 42 62 proved that a vertex reinforced jump process defined on the b ary tree is transient if b 4 and recurrent if b 1 the case b 2 is still open

achieving geometric convergence for distributed optimization over time varying graphs this paper considers the problem of distributed optimization over time varying graphs for the case of undirected graphs we introduce a distributed algorithm referred to as diging based on a combination of a distributed inexact gradient method and a gradient tracking technique the diging algorithm uses doubly stochastic mixing matrices and employs fixed step sizes and yet drives all the agents iterates to a global and consensual minimizer when the graphs are directed in which case the implementation of doubly stochastic mixing matrices is unrealistic we construct an algorithm that incorporates the push sum protocol into the diging structure thus obtaining the push diging algorithm push diging uses column stochastic matrices and fixed step sizes but it still converges to a global and consensual minimizer under the strong convexity assumption we prove that the algorithms converge at r linear geometric rates as long as the step sizes do not exceed some upper bounds we establish explicit est

indirect multiple shooting for nonlinear parabolic optimal control problems with control constraints we discuss the indirect multiple shooting approach for the solution of pde based parabolic optimal control problems with control constraints the method is formulated within an abstract function space setting and uses a space time galerkin finite element discretization the emphasis is on the embedding of indirect multiple shooting into the optimal control framework as well as the detailed description of the discretization within the pde context numerical results for linear and nonlinear model problems with and without control constraints illustrate the efficient use of indirect multiple shooting particularly in cases where other standard methods fail

on stability constants and condition numbers of discretisation methods discretizations lead to systems of algebraic equations of steadily growing size if the accuracy of the approximation is increasing inverse stability constants and condition numbers describe the sensitivity of the solution of these equations with respect to perturbations of the data starting from an abstract result in this paper the inverse stability constants of finite difference and collocation methods for bvodes as well as of ritz galerkin methods for elliptic variational problems are shown to converge to those of their continuous counterparts as the discretization becomes finer and finer for two choices of basis systems the exact asymptotic behavior of the collocation method condition number is given

extra an exact first order algorithm for decentralized consensus optimization recently there has been growing interest in solving consensus optimization problems in a multiagent network in this paper we develop a decentralized algorithm for the consensus optimization problem mathrm minimize x in mathbb r p bar f x frac 1 n sum i 1 n f i x which is defined over a connected network of n agents where each function f i is held privately by agent i and encodes the agent s data and objective all the agents shall collaboratively find the minimizer while each agent can only communicate with its neighbors such a computation scheme avoids a data fusion center or long distance communication and offers better load balance to the network this paper proposes a novel decentralized exact first order algorithm abbreviated as extra to solve the consensus optimization problem exact means that it can converge to the exact solution extra uses a fixed large step size which can be determined independently of the network size or topology the local variable of every a

distributed heavy ball a generalization and acceleration of first order methods with gradient tracking we study distributed optimization to minimize a sum of smooth and strongly convex functions recent work on this problem uses gradient tracking to achieve linear convergence to the exact global minimizer however a connection among different approaches has been unclear in this paper we first show that many of the existing first order algorithms are related with a simple state transformation at the heart of which lies a recently introduced algorithm known as mathcal ab we then present distributed heavy ball denoted as mathcal ab m that combines mathcal ab with a momentum term and uses nonidentical local step sizes by simultaneously implementing both row and column stochastic weights mathcal ab m removes the conservatism in the related work due to doubly stochastic weights or eigenvector estimation mathcal ab m thus naturally leads to optimization and average consensus over both undirected and directed graphs we show that mathcal ab m has a global r linear rate when the largest step size and momentum parameter are positive and sufficiently small we numerically show that mathcal ab m achieves acceleration particularly when the objective functions are ill conditioned

uniform fourth order difference scheme for a singular perturbation problem the numerical solution of a nonlinear singularly perturbed two point boundary value problem is studied the developed method is based on hermitian approximation of the second derivative on special discretization mesh numerical examples which demonstrate the effectiveness of the method are presented

conditions and conditioning stability and stabilization an overview is given of several concepts that are useful when dealing with numerical bvp in particular a generalisation of the well conditioning concept for nonlinear problems is considered this is done to be able to investigate a class of second order scalar nonlinear problems a detailed study of the solution structure is made for this class the results are applied to two specific problems korteweg de vries and burgers and a way is indicated to stabilize these ill posed bvp

a pde model for the dynamics of trail formation by ants abstract armies of ants are known to move in trails these trails are formed by a chemotactic force induced by pheromone secreted by the ants in this paper we develop a mathematical model consisting of two partial differential equations which explain when and how these trails are formed the first equation for the ants includes the chemotaxis effect of pheromone and the dispersion caused by overcrowding the second equation is a reaction diffusion equation for the pheromone concentration the strength of the chemotactic force plays a critical role in the analysis we prove that trails cannot be formed if is small while many trails exist if is large

staircase matrices and systems difference equations spline approximations and multiperiod linear programs all give rise to linear equation systems that have a characteristic staircase structure a staircase system s variables can be partitioned into a natural sequence of periods in such a way that every equation involves at most the variables of two adjacent periods this paper surveys the numerous properties of staircase systems and of the staircase matrices that comprise their coefficients particular attention is given to reduced and sparse staircase forms and to the varieties of gaussian elimination for solving staircase systems

some aspects of using iv software to solve bvp via a riccati transformation we consider a numerical method for the solution of linear boundary value problems bvps with separated boundary conditions bcs for ordinary differential equations odes by using a particular linear transformation the problem is reformulated as three initial value problems ivps one of them being a riccati differential equation the nonlinearity of the riccati de makes it necessary to deal with the eventuality of modifying the structure of integration so called multiple embedding equations must thus be derived and a marching algorithm is incorporated a stability analysis of the three ivps indicates clearly for what class of problems the method is suited and why stiff iv methods should then frequently be used this leads us to consider the undesirable effect of superstability of implicit schemes which can occur when integrating the riccati de we interpret this occurrence in terms of the original bvp and the resulting analysis suggests some final considerations including the possibility of corrective measures

convergence of asynchronous distributed gradient methods over stochastic networks we consider distributed optimization problems in which a number of agents are to seek the global optimum of a sum of cost functions through only local information sharing in this paper we are particularly interested in scenarios where agents are operating asynchronously over stochastic networks subject to random failures most existing algorithms require coordinated and decaying stepsizes to ensure zero gap between the estimated value of each agent and the exact optimum restricting it from asynchronous implementation and resulting in slower convergence results to deal with this issue we develop a new asynchronous distributed gradient method asyndgm based on consensus theory the proposed algorithm not only allows for asynchronous implementation in a completely distributed manner but also most importantly is able to seek the exact optimum even with constant stepsizes we will show that the assumption of boundedness of gradients which is widely used in the literature can be dropped by instead imposing the standard lipschitz continuity condition on gradients moreover we derive an upper bound of stepsize within which the proposed asyndgm can achieve a linear convergence rate for strongly convex functions with lipschitz gradients a canonical example of sensor fusion problems is provided to illustrate the effectiveness of the proposed algorithm

orthogonal spline collocation methods for schr o dinger type equations in one space variable we examine the use of orthogonal spline collocation for the semi discreti za tion of the cubic schr o dinger equation and the two dimensional parabolic equation of tappert in each case an optimal order l 2 estimate of the error in the semidiscrete approximation is derived for the cubic schr o dinger equation we present the results of numerical experiments in which the integration in time is performed using a routine from a software library

a riccati transformation method for solving linear bvps i theoretical aspects a method of solving linear two point boundary value problems bvps for first order systems of ordinary differential equations odes is considered of particular interest is the class of singularly perturbed bvps the method attempts to split the integration of the rapidly increasing decreasing solution components typical for such problems into different initial value problems ivps the case of separated boundary conditions bcs is considered and their ability to detect solution components that eventually increase decrease is exploited by using the riccati transformation the original bvp is then reformulated as three ivps one of them being a matrix riccati equation the riccati equation is nonlinear and a multiple embedding setting must be used the entire method is then fit theoretically into a multiple shooting like framework and the original bvp as a chain of subproblems is studied in this way a generalized decoupling result is presented which enables the study of stability to be brough