nonlinear control of constrained linear systems via predictive reference management a method based on conceptual tools of predictive control is described for solving set point tracking problems wherein pointwise in time input and or state inequality constraints are present it consists of adding to a primal compensated system a nonlinear device called command governor cg whose action is based on the current state set point and prescribed constraints the cg selects at any time a virtual sequence among a family of linearly parameterized command sequences by solving a convex constrained quadratic optimization problem and feeds the primal system according to a receding horizon control philosophy the overall system is proved to fulfill the constraints be asymptotically stable and exhibit an offset free tracking behavior provided that an admissibility condition on the initial state is satisfied though the cg can be tailored for the application at hand by appropriately choosing the available design knobs the required online computational load for the usual case of affine constraints is well tempered by the related relatively simple convex quadratic programming problem

global exponential periodicity of a class of recurrent neural networks with oscillating parameters and time varying delays in this paper we present the analytical results on the global exponential periodicity of a class of recurrent neural networks with oscillating parameters and time varying delays sufficient conditions are derived for ascertaining the existence uniqueness and global exponential periodicity of the oscillatory solution of such recurrent neural networks by using the comparison principle and mixed monotone operator method the periodicity results extend or improve existing stability results for the class of recurrent neural networks with and without time delays

global stability analysis in delayed cellular neural networks in this paper the author analyzes further problems of global stability for a class of cellular neural networks with delays by means of the lyapunov functional method inequalities a 2 b 2 g 2ab and a 3 b 3 c 3 g 3abc a b cg 0 analysis technique some stability criteria are obtained under more general conditions these criteria can be applied to design globally stable networks and thus have important significance in both theory and applications

novel criteria for exponential stability of nonlinear differential systems with delay general nonlinear differential systems with time varying delay are considered several explicit criteria for global exponential stability are presented two examples are given to illustrate the obtained results

faster convergence rates of relaxed peaceman rachford and admm under regularity assumptions in this paper we provide a comprehensive convergence rate analysis of the douglas rachford splitting drs peaceman rachford splitting prs and alternating direction method of multipliers admm algorithms under various regularity assumptions including strong convexity lipschitz differentiability and bounded linear regularity the main consequence of this work is that relaxed prs and admm automatically adapt to the regularity of the problem and achieve convergence rates that improve upon the tight worst case rates that hold in the absence of such regularity all of the results are obtained using simple techniques

on the linear convergence of the admm in decentralized consensus optimization in decentralized consensus optimization a connected network of agents collaboratively minimize the sum of their local objective functions over a common decision variable where their information exchange is restricted between the neighbors to this end one can first obtain a problem reformulation and then apply the alternating direction method of multipliers admm the method applies iterative computation at the individual agents and information exchange between the neighbors this approach has been observed to converge quickly and deemed powerful this paper establishes its linear convergence rate for the decentralized consensus optimization problem with strongly convex local objective functions the theoretical convergence rate is explicitly given in terms of the network topology the properties of local objective functions and the algorithm parameter this result is not only a performance guarantee but also a guideline toward accelerating the admm convergence

positive periodic solutions for a class of nonlinear delay equations abstract applying the theory of topological degree sufficient and realistic conditions are obtained for the existence of positive periodic solutions of a class of neutral delays equation from those conditions an algebraic criterion of existence for a more general neutral lotka volterra equation with several delays is obtained which extends and improves the previous results in addition this method is of great interest in many applications such as biomathematics

new exponential stability results for delayed neural networks with time varying delays abstract this paper presents new sufficient conditions for the uniqueness and exponential stability of the equilibrium point for delayed neural networks with time varying delays by employing a more general type of lyapunov krasovskii functional and using lmi linear matrix inequality we derive new results for exponential stability of the equilibrium point for delayed neural networks the results establish a relation between the delay time and the parameters of the network the results are compared with the previous results derived in the literature

nonlinear inexact uzawa algorithms for linear and nonlinear saddle point problems this paper proposes some nonlinear uzawa methods for solving linear and nonlinear saddle point problems a nonlinear inexact uzawa algorithm is first introduced for linear saddle point problems two different pcg techniques are allowed in the inner and outer iterations of the algorithm this algorithm is then extended for a class of nonlinear saddle point problems arising from some convex optimization problems with linear constraints for this extension some pcg method used in the inner iteration needs to be carefully constructed so that it converges in a certain energy norm instead of the usual l 2 norm it is shown that the new algorithm converges under some practical conditions and there is no need for any a priori estimates on the minimal and maximal eigenvalues of the two local preconditioned systems involved the two new methods perform more efficiently than the existing methods in the cases where no good preconditioners are available for the schur complements

preconditioning the augmented lagrangian method for instationary mean field games with diffusion we discuss the application of the augmented lagrangian method to the convex optimization problem of instationary variational mean field games with diffusion the problem is first discretized with space time tensor product piecewise polynomial bases this leads to a sequence of linear problems posed on the space time cylinder that are second order in the temporal variable and fourth order in the spatial variable to solve these large linear problems with the preconditioned conjugate gradients method we propose a preconditioner that is based on a temporal transformation coupled with a spatial multigrid this preconditioner is thus based on standard components and is particularly suitable for parallel computation it is conditionally parameter robust in the sense that the condition number of the preconditioned system is low for sufficiently fine temporal discretizations numerical examples illustrate the method

preconditioning techniques for large linear systems a survey this article surveys preconditioning techniques for the iterative solution of large linear systems with a focus on algebraic methods suitable for general sparse matrices covered topics include progress in incomplete factorization methods sparse approximate inverses reorderings parallelization issues and block and multilevel extensions some of the challenges ahead are also discussed an extensive bibliography completes the paper

methods for computing marginal data densities from the gibbs output abstract we introduce two estimators for estimating the marginal data density mdd from the gibbs output our methods are based on exploiting the analytical tractability condition which requires that some parameter blocks can be analytically integrated out from the conditional posterior densities this condition is satisfied by several widely used time series models an empirical application to six variate var models shows that the bias of a fully computational estimator is sufficiently large to distort the implied model rankings one of the estimators is fast enough to make multiple computations of mdds in densely parameterized models feasible

on the mathematical foundations of the self consistent clustering analysis for non linear materials at small strains abstract we investigate both mathematically and numerically the self consistent clustering analysis recently introduced by liu bessa liu and independently by wulfinghoff cavaliere reese we establish in the small strain setting and non softening material behavior existence and uniqueness of the solution to the discretized equations for fixed possibly anisotropic reference material by a constructive method furthermore we establish convergence of the solution to the discretized equation to the continuous solution upon refinement of the clusters thus we generalize the work of tang zhang liu to spatial dimensions larger than 1 we explore the specific structure of the lippmann schwinger equation governing clustering analysis proving strict equivalence to a problem of eyre milton type for the latter formulation existence and uniqueness are easily established based on recent progress in the understanding of polarization schemes in fft based computational homogenization methods last but not least for elasto viscoplastic constituents we clarify the relationship of the self consistent clustering analysis to the transformation field analysis of dvorak benveniste our theoretical considerations are confirmed by pertinent numerical experiments

nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems we consider projection algorithms for solving nonconvex feasibility problems in euclidean spaces of special interest are the method of alternating projections ap and the douglas rachford algorithm dr in the case of convex feasibility firm nonexpansiveness of projection mappings is a global property that yields global convergence of ap and for consistent problems dr a notion of local subfirm nonexpansiveness with respect to the intersection is introduced for consistent feasibility problems this together with a coercivity condition that relates to the regularity of the collection of sets at points in the intersection yields local linear convergence of ap for a wide class of nonconvex problems and even local linear convergence of nonconvex instances of the dr algorithm

tight coefficients of averaged operators via scaled relative graph abstract many iterative methods in optimization are fixed point iterations with averaged operators as such methods converge at an o 1 k rate with the constant determined by the averagedness coefficient establishing small averagedness coefficients for operators is of broad interest in this paper we show that the averagedness coefficients of the composition of averaged operators by ogura and yamada 2002 21 and the three operator splitting by davis and yin 2017 9 are tight the analysis relies on the scaled relative graph a geometric tool recently proposed by ryu et al 2019 25

exponential stability of delayed bi directional associative memory networks some sufficient conditions are derived for the global exponential stability in delayed bi directional associative memory bam networks by constructing suitable lyapunov functional and the inequality 2ab

global asymptotic stability of bam neural networks with mixed delays and impulses in this paper bam neural networks with mixed delays and impulses are considered a new set of sufficient conditions are derived by constructing suitable lyapunov functional with matrix theory for the global asymptotic stability of bam neural networks with mixed delays and impulses moreover an example is also provided to illustrate the effectiveness of the results

existence and global exponential stability of almost periodic solution for cellular neural networks with variable coefficients and time varying delays in this paper we study cellular neural networks with almost periodic variable coefficients and time varying delays by using the existence theorem of almost periodic solution for general functional differential equations introducing many real parameters and applying the lyapunov functional method and the technique of young inequality we obtain some sufficient conditions to ensure the existence uniqueness and global exponential stability of almost periodic solution the results obtained in this paper are new useful and extend and improve the existing ones in previous literature

alternating projections and douglas rachford for sparse affine feasibility the problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an np complete problem that is typically solved numerically via convex heuristics or nicely behaved nonconvex relaxations in this work we consider elementary methods based on projections for solving a sparse feasibility problem without employing convex heuristics it has been shown recently that locally the fundamental method of alternating projections must converge linearly to a solution to the sparse feasibility problem with an affine constraint in this paper we apply different analytical tools that allow us to show global linear convergence of alternating projections under familiar constraint qualifications these analytical tools can also be applied to other algorithms this is demonstrated with the prominent douglas rachford algorithm where we establish local linear convergence of this method applied to the sparse affine feasibility problem

metric selection in fast dual forward backward splitting the performance of fast forward backward splitting or equivalently fast proximal gradient methods depends on the conditioning of the optimization problem data this conditioning is related to a metric that is defined by the space on which the optimization problem is stated selecting a space on which the optimization data is better conditioned improves the performance of the algorithm in this paper we propose several methods with different computational complexity to find a space on which the algorithm performs well we evaluate the proposed metric selection procedures by comparing the performance to the case when the euclidean space is used for the most ill conditioned problem we consider the computational complexity is improved by two to three orders of magnitude we also report comparable to superior performance compared to state of the art optimization software

on the o 1 n convergence rate of the douglas rachford alternating direction method alternating direction methods adms have been well studied in the literature and they have found many efficient applications in various fields in this note we focus on the douglas rachford adm scheme proposed by glowinski and marrocco and we aim at providing a simple approach to estimating its convergence rate in terms of the iteration number the linearized version of this adm scheme which is known as the split inexact uzawa method in the image processing literature is also discussed

local linear convergence of the alternating direction method of multipliers on quadratic or linear programs we introduce a novel matrix recurrence yielding a new spectral analysis of the local transient convergence behavior of the alternating direction method of multipliers admm for the particular case of a quadratic program or a linear program we identify a particular combination of vector iterates whose convergence can be analyzed via a spectral analysis the theory predicts that admm should go through up to four convergence regimes such as constant step convergence or linear convergence ending with the latter when close enough to the optimal solution if the optimal solution is unique and satisfies strict complementarity

optimal parameter selection for the alternating direction method of multipliers admm quadratic problems the alternating direction method of multipliers admm has emerged as a powerful technique for large scale structured optimization despite many recent results on the convergence properties of admm a quantitative characterization of the impact of the algorithm parameters on the convergence times of the method is still lacking in this paper we find the optimal algorithm parameters that minimize the convergence factor of the admm iterates in the context of l 2 regularized minimization and constrained quadratic programming numerical examples show that our parameter selection rules significantly outperform existing alternatives in the literature

existence and global exponential stability of periodic solution for discrete time bam neural networks the discrete time analogues of bidirectional associative memory neural networks with periodic coefficients and distributed delays are formulated and studied and by using the continuation theorem of coincidence degree theory we derive the existence of periodic solution for the discrete time bam neural networks and by constructing a appropriate lyapunov type sequence we prove the global exponential stability of the periodic solution for the model it is shown that the discrete time analogues inherit the existence and global exponential stability of periodic solution for the continuous time bam neural networks an example is given to illustrate the effectiveness of the obtained results

analysis of the inexact uzawa algorithm for saddle point problems in this paper we consider the so called inexact uzawa algorithm for iteratively solving linear block saddle point problems such saddle point problems arise for example in finite element and finite difference discretizations of stokes equations the equations of elasticity and mixed finite element discretization of second order problems we consider both the linear and nonlinear variants of the inexact uzawa iteration we show that the linear method always converges as long as the preconditioners defining the algorithm are properly scaled bounds for the rate of convergence are provided in terms of the rate of convergence for the preconditioned uzawa algorithm and the reduction factor corresponding to the preconditioner for the upper left hand block in the case of nonlinear iteration the inexact uzawa algorithm is shown to converge provided that the nonlinear process approximating the inverse of the upper left hand block is of sufficient accuracy bounds for the nonlinear iteration are given in terms of this accuracy parameter and the rate of convergence of the preconditioned linear uzawa algorithm applications to the stokes equations and mixed finite element discretization of second order elliptic problems are discussed and finally the results of numerical experiments involving the algorithms are presented

linear convergence and metric selection for douglas rachford splitting and admm recently several convergence rate results for douglas rachford splitting and the alternating direction method of multipliers admm have been presented in the literature in this paper we show global linear convergence rate bounds for douglas rachford splitting and admm under strong convexity and smoothness assumptions we further show that the rate bounds are tight for the class of problems under consideration for all feasible algorithm parameters for problems that satisfy the assumptions we show how to select step size and metric for the algorithm that optimize the derived convergence rate bounds for problems with a similar structure that do not satisfy the assumptions we present heuristic step size and metric selection methods

on global asymptotic stability of recurrent neural networks with time varying delays in this paper by constructing a new lyapunov functional and using m matrix and topological degree tool problem of the global asymptotic stability gas is discussed for a class of recurrent neural networks with time varying delays some simple and new sufficient conditions are obtained ensuring existence uniqueness of the equilibrium point and its gas of the neural networks some previous works are improved in addition this condition does not require the activation functions to be differentiable bounded and monotone nondecreasing and the weight connected matrices to be symmetric the neural network model considered in this paper include the delayed hopfield neural networks bidirectional associative memory networks and delayed cellular neural networks as its special cases

global exponential periodicity of a class of bidirectional associative memory networks with finite distributed delays in this paper we study the periodic oscillatory behavior of a class of bidirectional associative memory bam networks with finite distributed delays a set of criteria are proposed for determining global exponential periodicity of the proposed bam networks which assume neither differentiability nor monotonicity of the activation function of each neuron in addition our criteria are easily checkable

stability and existence of periodic solutions of periodic cellular neural networks with time vary delays by using the continuation theorem of mawhins coincidence degree theory and constructing a suitable lyapunov function some new sufficient conditions are obtained ensuring existence and global asymptotical stability of periodic solution of cellular neural networks with periodic coefficients and delays which do not require the activation functions to be differentiable and monotone nondecreasing a numerical example is given to illustrate that the criteria are feasible these results are helpful to design globally asymptotically stable and periodic oscillatory cellular neural networks

computation of synchronized periodic solution in a bam network with two delays a bidirectional associative memory bam neural network with four neurons and two discrete delays is considered to represent an analytical method namely perturbation incremental scheme pis the expressions for the periodic solutions derived from hopf bifurcation are given by using the pis the result shows that the pis has higher accuracy than the center manifold reduction cmr with normal form for the values of time delay not far away from the hopf bifurcation point in terms of the pis the necessary and sufficient conditions of synchronized periodic solution arising from a hopf bifurcation are obtained and the synchronized periodic solution is expressed in an analytical form it can be seen that theoretical analysis is in good agreement with numerical simulation it implies that the provided method is valid and the obtained result is correct to the best of our knowledge the paper is the first one to introduce the pis to study the periodic solution derived from hopf bifurcation for a 4 d delayed system quantitatively

convergence rate of distributed admm over networks we propose a new distributed algorithm based on alternating direction method of multipliers admm to minimize sum of locally known convex functions using communication over a network this optimization problem emerges in many applications in distributed machine learning and statistical estimation our algorithm allows for a general choice of the communication weight matrix which is used to combine the iterates at different nodes we show that when functions are convex both the objective function values and the feasibility violation converge with rate o 1 t where t is the number of iterations we then show that when functions are strongly convex and have lipschitz continuous gradients the sequence generated by our algorithm converges linearly to the optimal solution in particular an epsilon optimal solution can be computed with o left sqrt kappa f log 1 epsilon right iterations where kappa f is the condition number of the problem our analysis highlights the effect of network and communication weights on the convergence rate through degrees of the nodes the smallest nonzero eigenvalue and operator norm of the communication matrix

on the equivalence of the primal dual hybrid gradient method and douglas rachford splitting the primal dual hybrid gradient pdhg algorithm proposed by esser zhang and chan and by pock cremers bischof and chambolle is known to include as a special case the douglas rachford splitting algorithm for minimizing the sum of two convex functions we show that conversely the pdhg algorithm can be viewed as a special case of the douglas rachford splitting algorithm

exponential stability and oscillation of hopfield graded response neural network both exponential and stochastic stabilities of the hopfield neural network are analyzed the results are especially useful for analyzing the stabilities of asymmetric neural networks a constraint on the connection matrix has been found under which the neural network has a unique and exponentially stable equilibrium given any connection matrix this constraint can be satisfied through the adjustment of the gains of the amplifiers and the resistances in the neural net circuit a one to one and smooth map between input currents and the equilibria of the neural network can be set up the above results can be applied to the master slave net to prove that the master net can find the best connection matrix for the slave net for the neural network disturbed by some noise the stochastic stability of the network is also analyzed a special asymmetric neural network formed by a closed chain of formal neurons is also studied for its stability and oscillation both stable and oscillatory dynamics are obtained in the closed chain network through the adjustment of the gains and resistances of the amplifiers

preconditioned douglas rachford splitting methods for convex concave saddle point problems we propose a preconditioned version of the douglas rachford splitting method for solving convex concave saddle point problems associated with fenchel rockafellar duality our approach makes it possible to use approximate solvers for the linear subproblem arising in this context we prove weak convergence in hilbert space under minimal assumptions in particular various efficient preconditioners are introduced in this framework for which only a few inner iterations are needed instead of computing an exact solution or controlling the error the method is applied to a discrete total variation denoising problem numerical experiments show that the proposed algorithms with appropriate preconditioners are very competitive with existing fast algorithms including the first order primal dual algorithm for saddle point problems of chambolle and pock

exponential stability and periodic oscillatory solution in bam networks with delays both exponential stability and periodic oscillatory solution of bidirectional associative memory bam networks with axonal signal transmission delays are considered by constructing suitable lyapunov functional and some analysis techniques some simple sufficient conditions are given ensuring the global exponential stability and the existence of periodic oscillatory solutions of bam with delays these conditions are presented in terms of system parameters and have important leading significance in the design and applications of globally exponentially stable and periodic oscillatory neural circuits for bam with delays in addition two examples are given to illustrate the results

global asymptotic stability of bi directional associative memory networks with distributed delays in this paper several sufficient conditions are derived for the global asymptotic stability in bi directional associative memory bam networks with distributed delays by constructing suitable lyapunov functional and the young inequality technique these conditions have an important leading significance in the design and applications of globally asymptotically stable neural circuits for bam with distributed delays

on the exponential stability and periodic solutions of delayed cellular neural networks a set of criteria is presented for the global exponential stability and the existence of periodic solutions of delayed cellular neural networks dcnns by constructing suitable lyapunov functionals introducing many parameters and combining with the elementary inequality technique these criteria have important leading significance in the design and applications of globally stable dcnns and periodic oscillatory dcnns in addition earlier results are extended and improved other results are contained two examples are given to illustrate the theory