periodic orbits in periodic discrete dynamics we study the combinatorial structure of periodic orbits of nonautonomous difference equations x n 1 f n x n in a periodically fluctuating environment we define the c set to be the set of minimal periods that are not multiples of the phase period we show that when the functions f n are rational functions the c set is a finite set in particular we investigate several mathematical models of single species without age structure and find that periodic oscillations are influenced by periodic environments to the extent that almost all periods are divisors or multiples of the phase period

stability of the gumowski mira equation with period two coefficient abstract we investigate the stability of solutions of the gumowski mira equation with a period two coefficient y n 1 y n b n y n 2 y n 1 n 0 1 with b n 0 for n 2 k 0 for n 2 k 1 k 0 1 and the initial values y 1 y 0 are real numbers

predator induced and mating limitation induced allee effects in a discrete time sims epidemic model we use a susceptible infected immune susceptible sims discrete time epidemic model that incorporates both predator induced and mating limitation induced allee effects in the demographic dynamics to study the impact of an infectious disease on population persistence or extinction when the allee effect is strong with or without the disease the total population goes extinct whenever the initial total population size is smaller than the allee threshold however when the allee effect is weak or the initial population sizes are high and the total population is either asymptotically constant or under geometric growth we use the next generation method to compute the basic reproductive number 0 and use it to predict the uniform persistence or extinction of the infective and immune populations

a constrained min max algorithm for rival models of the same economic system there are well established rival theories about the economy these have in turn led to the development of rival models purporting to represent the economic system the models are large systems of discrete time nonlinear dynamic equations observed data of the real system does not in general provide sufficient information for statistical methods to invalidate all but one of the rival models in such a circumstance there is uncertainty about which model to use in the formulation of policy prudent policy design would suggest that a model based policy should take into account all the rival models this is achieved as a pooling of the models the pooling that yields the policy which is robust to model choice is formulated as a constrained min max problem the minimization is over the decision variables and the maximization is over the rival models only equality constraints are considered

variable metric methods for unconstrainted optimization and nonlinear least squares variable metric or quasi newton methods are well known and commonly used in connection with unconstrained optimization since they have good theoretical and practical convergence properties although these methods were originally developed for small and moderate size dense problems their modifications based either on sparse partitioned or limited memory updates are very efficient on large scale sparse problems very significant applications of these methods also appear in nonlinear least squares approximation and nonsmooth optimization in this contribution we give an extensive review of variable metric methods and their use in various optimization fields

multilevel least change newton like methods for equality constrained optimization problems this paper deals with the application of multilevel least change newton like methods for solving twice continuously differentiable equality constrained optimization problems we define multilevel partial inverse least change updates multilevel least change newton like methods without derivatives and multilevel projections of fragments of the matrix for newton like methods without derivatives local andq superlinear convergence of these methods is proved the theorems here also imply local andq superlinear convergence of many standard newton like methods for nonconstrained and equality constraine optimization problems

periodic paths on nonautonomous graphs abstract we define nonautonomous graphs as a class of dynamic graphs in discrete time whose time dependence consists in connecting or disconnecting edges we study periodic paths in these graphs and the associated zeta functions based on the analytic properties of these zeta functions we obtain explicit formulae for the number of n periodic paths as the sum of the n th powers of some specific algebraic numbers

integrability and algebraic entropy of k periodic non autonomous lyness recurrences abstract this work deals with non autonomous lyness type recurrences of the form x n 2 a n x n 1 x n where a n n is a k periodic sequence of complex numbers with minimal period k we treat such non autonomous recurrences via the autonomous dynamical system generated by the birational mapping f a k f a k 1 f a 1 where f a is defined by f a x y y a y x for the cases k 1 2 3 6 the corresponding mappings have a rational first integral by calculating the dynamical degree we show that for k 4 and for k 5 generically the dynamical system is no longer rationally integrable we also prove that the only values of k for which the corresponding dynamical system is rationally integrable for all the values of the involved parameters are k 1 2 3 6

the necessary and sufficient conditions of existence of periodic solutions of nonautonomous difference equations the necessary and sufficient conditions of existence of periodic solutions of nonautonomous difference equations x n 1 f n x n x n 1 x n k where k is a nonnegative integer are studied a method of analysis of bifurcation of periodic solutions in the nonautonomous difference equations are presented we apply this result on the rational difference equation x n 1 a n b nx n x n 1

phase retrieval via wirtinger flow theory and algorithms we study the problem of recovering the phase from magnitude measurements specifically we wish to reconstruct a complex valued signal boldsymbol x in mathbb c n about which we have phaseless samples of the form y r left langle boldsymbol a r boldsymbol x rangle right 2 r 1 ldots m knowledge of the phase of these samples would yield a linear system this paper develops a nonconvex formulation of the phase retrieval problem as well as a concrete solution algorithm in a nutshell this algorithm starts with a careful initialization obtained by means of a spectral method and then refines this initial estimate by iteratively applying novel update rules which have low computational complexity much like in a gradient descent scheme the main contribution is that this algorithm is shown to rigorously allow the exact retrieval of phase information from a nearly minimal number of random measurements indeed the sequence of successive iterates provably converges to the solution at a geometric rate so that the proposed scheme is efficient both in terms of computational and data resources in theory a variation on this scheme leads to a near linear time algorithm for a physically realizable model based on coded diffraction patterns we illustrate the effectiveness of our methods with various experiments on image data underlying our analysis are insights for the analysis of nonconvex optimization schemes that may have implications for computational problems beyond phase retrieval

on the periodic logistic equation we show that the p periodic logistic equation xn 1 n mod pxn 1 i xn has cycles periodic solutions of minimal periods 1 p 2p 3p then we extend singer s theorem to periodic dierence equations and use it to show the p periodic logistic equation has at most p stable cycles also we present computational methods investigating the stable cycles in case p 2 and 3

attractors for discrete periodic dynamical systems a mathematical framework is introduced to study attractors of discrete nonautonomous dynamical systems which depend periodically on time a structure theorem for such attractors is established which says that the attractor of a time periodic dynamical system is the union of attractors of appropriate autonomous maps if the nonautonomous system is a perturbation of an autonomous map properties that the nonautonomous attractor inherits from the autonomous attractor are discussed examples from population biology are presented

on the convergence of a new trust region algorithm in this paper we present a new trust region algorithm for general nonlinear constrained optimization problems the algorithm is based on the l infty exact penalty function under very mild conditions global convergence results for the algorithm are given local convergence properties are also studied it is shown that the penalty parameter generated by the algorithm will be eventually not less than the l 1 norm of the lagrange multipliers at the accumulation point it is proved that the method is equivalent to the sequential quadratic programming method for all large k hence superlinearly convergent results of the sqp method can be applied numerical results are also reported

formulas for liapunov functions for systems of linear difference equations explicit quadratic liapunov functions that provide necessary and sufficient conditions for the asymptotic stability of the system of linear difference equations x t 1 a x t are constructed by transforming the original systems to y t 1 g y t where g is a companion matrix associated with the characteristic polynomial of a a necessary and sufficient condition for all roots of the characteristic polynomial to lie in the unit circle z 1 on the complex plane is also derived

on fienup methods for sparse phase retrieval alternating minimization or fienup methods have a long history in phase retrieval we provide new insights related to the empirical and theoretical analysis of these algorithms when used with fourier measurements and combined with convex priors in particular we show that fienup methods can be viewed as performing alternating minimization on a regularized nonconvex least squares problem with respect to amplitude measurements furthermore we prove that under mild additional structural assumptions on the prior semialgebraicity the sequence of signal estimates has a smooth convergent behavior toward a critical point of the nonconvex regularized least squares objective finally we propose an extension to fienup techniques based on a projected gradient descent interpretation and acceleration using inertial terms we demonstrate experimentally that this modification combined with an ell 1 prior constitutes a competitive approach for sparse phase retrieval

on the stability in periodic and almost periodic difference systems abstract in this paper we consider the system of difference equations x n 1 f n x n f n 0 0 n 0 1 2 this system has the trivial zero solution x n 0 sufficient conditions of its asymptotic stability are obtained in the cases when functions f n x are periodic and almost periodic in n

dynamic parrondo s paradox abstract this paper is focused on studying parrondo s paradox in non linear dynamics specifically how the periodic combination of the individual maps f and g can give rise to chaos or order we construct dynamical systems exhibiting the paradox for several notions of chaos derived from topological dynamics the effect of altering the order of the combination i e considering f g or g f is analyzed as well as the robustness of the parrondo effect under small perturbations conditions for avoiding parrondian dynamics are also obtained placing special emphasis on the notion of chaos given by turbulence

a note on quasi newton formulae for sparse second derivative matrices in order to apply quasi newton methods to solve unconstrained minimization calculations when the number of variables is very large it is usually necessary to make use of any sparsity in the second derivative matrix of the objective function therefore it is important to extend to the sparse case the updating formulae that occur in variable metric algorithms to revise the estimate of the second derivative matrix suitable extensions suggest themselves when the updating formulae are derived by variational methods 1 3 the purpose of the present paper is to give a new proof of a theorem of dennis and schnabel 1 that shows the effect of sparsity on updating formulae for second derivative estimates

discrete time sis epidemicmodel in a seasonal environment we study the combined effects of seasonal trends and diseases on the extinction and persistence of discretely reproducing populations we introduce the epidemic threshold parameter r0 for predicting disease dynamics in periodic environments typically in periodic environments r0 1 implies disease persistence on a cyclic attractor while r0 1 implies disease extinction we also explore the relationship between the demographic equation and the epidemic process in particular we show that in periodic environments it is possible for the infective population to be on a chaotic attractor while the demographic dynamics is nonchaotic

periodic sequences of simple maps can support chaos abstract in this paper we explore the parrondo s paradox when several dynamically simple maps are combined in a periodic way producing chaotic dynamics we show that the paradox is not commutative that is it depends on the way that the maps are iterated we also see that the paradox happens more frequently when the number of maps that we iterate increases

homogeneous polynomials as lyapunov functions in the stability research of solutions of difference equations the linear autonomous system of difference equations x n 1 ax n is considered where email protected r k a is a real nonsingular kxk matrix in this paper it has been proved that if w x is any homogeneous polynomial of m th degree in x then there exists a unique homogeneous polynomial v x of m th degree such that dv v ax v x w x if and only if l 1 i 1 l 2 i 2 l k i k 1 i 1 i 2 i k m i j 0 where l 1 l 2 l k are the eigenvalues of the matrix a the theorem on the instability has also been proved

folding and unfolding in periodic difference equations abstract given a p periodic difference equation x n 1 f n mod p x n where each f j is a continuous interval map j 0 1 p 1 we discuss the notion of folding and unfolding related to this type of non autonomous equations it is possible to glue certain maps of this equation to shorten its period which we call folding on the other hand we can unfold the glued maps so the original structure can be recovered or understood here we focus on the periodic structure under the effect of folding and unfolding in particular we analyze the relationship between the periods of periodic sequences of the p periodic difference equation and the periods of the corresponding subsequences related to the folded systems

spectral invariants of periodic nonautonomous discrete dynamical systems abstractfor an interval map the poles of the artin mazur zeta function providetopological invariants which are closely connected to topological entropy itis known that for a time periodic nonautonomous dynamical system f withperiod p the p th power f z p of its zeta function is meromorphic in theunit disc unlike in the autonomous case where the zeta function f z onlyhaves poles in the unit disc in the p periodic nonautonomous case f z p may have zeros in this paper we introduce the concept of spectral invariantsof p periodic nonautonomous discrete dynamical systems and study the roleplayed by the zeros of f z p in this context as we will see these zerosplay an important role in the spectral classi cation of these systems keywords nonautonomous discrete dynamical systems interval maps zeta functions spectral invariants topological entropy 1 motivation and statement of main resultsthe most part of mathematical models appearing in discrete dynamicalsystems for approaching applied situations coming from biology physics

orthogonal projections on convex sets for newton like methods this paper develops the fixed almost least change secant methods for solving of nonlinear problems the presented results generalize and extend some results given by dennis and walker siam j numer anal 18 1981 pp 949 987 among other things multilevel update algorithms are described

an extension of sharkovsky s theorem to periodic difference equations we present an extension of sharkovsky s theorem and its converse to periodic difference equations in addition we provide a simple method for constructing a p periodic difference equation having an r periodic geometric cycle with or without stability properties

on the structure of attractors for discrete periodically forced systems with application to population models abstract this work discusses the effects of periodic forcing on attracting cycles and more complicated attractors for autonomous systems of nonlinear difference equations results indicate that an attractor for a periodically forced dynamical system may inherit structure from an attractor of the autonomous unforced system and also from the periodicity of the forcing in particular a method is presented which shows that if the amplitude of the k periodic forcing is small enough then the attractor for the forced system is the union of k homeomorphic subsets examples from population biology and genetics indicate that each subset is also homeomorphic to the attractor of the original autonomous dynamical system

on the periodically perturbed logistic equation we study the logistic equation modified by a periodic time dependence the perturbation introduces bifurcation delays which can be calculated explicitly and we qualitatively explain how the bifurcation diagram deforms as the perturbation is increased

periodic systems of population models and enveloping functions we show that p cull s concept of enveloping functions can be applied to periodic systems of population models to ensure the existence of globally asymptotically stable attractors for such systems without ever considering the compositions of the maps involved we give conditions to ensure that a period n system of population models sharing a fixed point and enveloping function has a globally asymptotically stable trivial geometric cycle we also give conditions on two periodic systems of population models that do not share a fixed point or an enveloping function that ensures the existence of a globally attracting geometric 2 cycle as well as providing bounds for the location of the attractor a technique is outlined to establish the existence of a period n attractor for a period n system of population models that do not share a fixed point using the theorems for the period 2 case furthermore we give examples that show enveloping provides sufficient but not necessary conditions for the existence of globally asymptotically stable geometric cycles in such systems

phase retrieval using alternating minimization phase retrieval problems involve solving linear equations but with missing sign or phase for complex numbers information more than four decades after it was first proposed the seminal error reduction algorithm of gerchberg and saxton and fienup is still the popular choice for solving many variants of this problem the algorithm is based on alternating minimization i e it alternates between estimating the missing phase information and the candidate solution despite its wide usage in practice no global convergence guarantees for this algorithm are known in this paper we show that a resampling variant of this approach converges geometrically to the solution of one such problem finding a vector bf x from bf y bf a where bf y vert bf a t bf x vert and vert bf z vert denotes a vector of element wise magnitudes of bf z under the assumption that bf a is gaussian empirically we demonstrate that alternating minimization performs similar to recently proposed convex techniques for this problem which are based on lifting to a convex matrix problem in sample complexity and robustness to noise however it is much more efficient and can scale to large problems analytically for a resampling version of alternating minimization we show geometric convergence to the solution and sample complexity that is off by log factors from obvious lower bounds we also establish close to optimal scaling for the case when the unknown vector is sparse our work represents the first theoretical guarantee for alternating minimization albeit with resampling for any variant of phase retrieval problems in the non convex setting

phase retrieval with random gaussian sensing vectors by alternating projections we consider a phase retrieval problem where we want to reconstruct a n dimensional vector from its phaseless scalar products with m sensing vectors independently sampled from complex normal distributions we show that with a suitable initialization procedure the classical algorithm of alternating projections gerchberg saxton succeeds with high probability when m geq cn for some c 0 we conjecture that this result is still true when no special initialization procedure is used and present numerical experiments that support this conjecture

global stability of periodic orbits of non autonomous difference equations and population biology elaydi and yakubu showed that a globally asymptotically stable gas periodic orbit in an autonomous dierence equation must in fact be a fixed point whenever the phase space is connected in this paper we extend this result to periodic nonautonomous dierence equations via the concept of skew product dynamical systems we show that for a k periodic dierence equation if a periodic orbit of period r is gas then r must be a divisor of k in particular sub harmonic or long periodic oscillations cannot occur moreover if r divides k we construct a non autonomous dynamical system having minimum period k and which has a gas periodic orbit with minimum period r our methods are then applied to prove a conjecture by j cushing and s henson concerning a non autonomous beverton holt equation which arises in the study of the response of a population to a periodically fluctuating environmental force such as seasonal fluctuations in carrying capacity or demographic parameters like birth or death rates

asymptotic stability condition for a class of linear delay difference equations of higher order abstract in this paper we obtain a necessary and sufficient condition for the asymptotic stability of the zero solution of the linear delay difference equation x n 1 x n p j 1 n x n k j 1 l 0 n 0 1 2 by using root analysis for the characteristic equation here p is a real number and k l and n are positive integers with k n 1 l

local and superlinear convergence for truncated iterated projections methods least change secant updates can be obtained as the limit of iterated projections based on other secant updates we show that these iterated projections can be terminated or truncated after any positive number of iterations and the local and the superlinear rate of convergence are still maintained the truncated iterated projections method is used to find sparse and symmetric updates that are locally and superlinearly convergent

a sequential quadratic optimization algorithm with rapid infeasibility detection we present a sequential quadratic optimization sqo algorithm for nonlinear constrained optimization the method attains all of the strong global and fast local convergence guarantees of classical sqo methods but has the important additional feature that fast local convergence is guaranteed when the algorithm is employed to solve infeasible instances a two phase strategy carefully constructed parameter updates and a line search are employed to promote such convergence the first phase subproblem determines the reduction that can be obtained in a local model of an infeasibility measure when the objective function is ignored the second phase subproblem then seeks to minimize a local model of the objective while ensuring that the resulting search direction attains a reduction in the local model of the infeasibility measure that is proportional to that attained in the first phase the subproblem formulations and parameter updates ensure that near an optimal solution the algorithm reduces to a classica

a family of variable metric proximal methods we consider conceptual optimization methods combining two ideas the moreau yosida regularization in convex analysis and quasi newton approximations of smooth functions we outline several approaches based on this combination and establish their global convergence then we study theoretically the local convergence properties of one of these approaches which uses quasi newton updates of the objective function itself also we obtain a globally and superlinearly convergent bfgs proximal method at each step of our study we single out the assumptions that are useful to derive the result concerned

convergence theorems for least change secant update methods the purpose of this paper is to present a convergence analysis of least change secant methods in which part of the derivative matrix being approximated is computed by other means the theorems and proofs given here can be viewed as generalizations of those given by broyden dennis more j inst math appl 12 1973 pp 223 246 and by dennis more math comp 28 1974 pp 549 560 the analysis is done in the orthogonal projection setting of dennis schnabel siam rev 21 1980 pp 443 459 and many readers might feel that it is easier to understand the theorems here readily imply local and q superlinear convergence of all the standard methods in addition to proving these results for the first time for the sparse symmetric method of marwil and toint and the nonlinear least squares method of dennis gay welsch

phase recovery maxcut and complex semidefinite programming phase retrieval seeks to recover a signal x in mathbb c p x c p from the amplitude a x a x of linear measurements ax in mathbb c n a x c n we cast the phase retrieval problem as a non convex quadratic program over a complex phase vector and formulate a tractable relaxation called phasecut similar to the classical maxcut semidefinite program we solve this problem using a provably convergent block coordinate descent algorithm whose structure is similar to that of the original greedy algorithm in gerchberg and saxton optik 35 237 246 1972 where each iteration is a matrix vector product numerical results show the performance of this approach over three different phase retrieval problems in comparison with greedy phase retrieval algorithms and matrix completion formulations

on the local convergence of quasi newton methods for constrained optimization we consider the application of a general class of quasi newton methods to the solution of the classical equality constrained nonlinear optimization problem specifically we develop necessary and sufficient conditions for the q superlinear convergence of such methods and present a companion linear convergence theorem the essential conditions relate to the manner in which the hessian of the lagrangian function is approximated

stability in the family of ω limit sets of alternating systems let f and g be elements of c i with x i 0 1 we study the limit sets x f g generated by alternating trajectories of the form x f g x f x g f x f g f x as well as the sets f g x i x f g and l f g x f g x i in particular we show that 1 if g is constant on no interval j i then there exists a residual set s c i so that the maps c i c i k and l c i c i k taking f g to f g and l f g respectively are both continuous at f g whenever f s 2 the map i c i c i k taking x f g to x f g is in the second class of baire and for any g c i there exists a residual set t i c i so that is continuous at x f g whenever x f t 3 if f is constant on no interval j i then there exists a residual set d i c i so that x f g x g f f x f g where both x g f and f x f g are adding machines of type whenever x g d

a quasi newton strategy for the ssqp method for variational inequality and optimization problems the quasi newton strategy presented in this paper preserves one of the most important features of the stabilized sequential quadratic programming method the local convergence without constraint qualifications assumptions it is known that the primal dual sequence converges quadratically assuming only the second order sufficient condition in this work we show that if the matrices are updated by performing a minimization of a bregman distance which includes the classic updates the quasi newton version of the method converges superlinearly without introducing further assumptions also we show that even for an unbounded lagrange multipliers set the generated matrices satisfies a bounded deterioration property and the dennis more condition

attenuant cycles in periodically forced discrete time age structured population models in discrete time age structured population models a periodic environment is not always deleterious we show that it is possible to have the average of the age class populations over an attracting cycle in a periodic environment not less than the average of the carrying capacities in a corresponding constant environment in our age structured model a periodic environment does not increase the average total biomass no resonance however a periodic environment is disadvantageous for a population whenever there is no synchrony between the number of age classes and the period of the environment as in periodically forced models without age structure we show that periodically forced age structured population models support multiple attractors with complicated structures

solving systems of random quadratic equations via truncated amplitude flow this paper presents a new algorithm termed truncated amplitude flow taf to recover an unknown vector x from a system of quadratic equations of the form y i langle a i x rangle 2 where a i s are given random measurement vectors this problem is known to be np hard in general we prove that as soon as the number of equations is on the order of the number of unknowns taf recovers the solution exactly up to a global unimodular constant with high probability and complexity growing linearly with both the number of unknowns and the number of equations our taf approach adapts the amplitude based empirical loss function and proceeds in two stages in the first stage we introduce an orthogonality promoting initialization that can be obtained with a few power iterations stage two refines the initial estimate by successive updates of scalable truncated generalized gradient iterations which are able to handle the rather challenging nonconvex and nonsmooth amplitude based objective function in particular when vectors x and a i s are real valued our gradient truncation rule provably eliminates erroneously estimated signs with high probability to markedly improve upon its untruncated version numerical tests using synthetic data and real images demonstrate that our initialization returns more accurate and robust estimates relative to spectral initializations furthermore even under the same initialization the proposed amplitude based refinement outperforms existing wirtinger flow variants corroborating the superior performance of taf over state of the art algorithms

periodicity and stability of linear volterra difference systems abstract we give sufficient conditions for a system of volterra difference equations to have a unique periodic solution then a criterion for uniform asymptotic stability is established

a note on the lyapunov stability of periodic discrete time systems we characterize the stability of discrete time lyapunov equations with periodic coefficients the characterization can be seen as the analog of the classical stability theorem of lyapunov equations with constant coefficients it involves quantities readily computable with good accuracy

fixed point quasi newton methods this article studies iterative methods defined by x k 1 phi x k e k where x k in mathbb r n and e k lie on a space of parameters sufficient conditions are established for local convergence and for convergence at an ideal linear or superlinear rate a theory of least change secant update methods for this class of processes is developed several examples are given showing a wide range of applications of the new theory key words quasi

majorization minimization procedures and convergence of sqp methods for semi algebraic and tame programs in view of solving nonsmooth and nonconvex problems involving complex constraints like standard nlp problems we study general maximization minimization procedures produced by families of strongly convex subproblems using techniques from semi algebraic geometry and variational analysis in particular ojasiewicz inequality we establish the convergence of sequences generated by these types of schemes to critical points the broad applicability of this process is illustrated in the context of nlp in that case critical points coincide with kkt points when the data are semi algebraic or real analytic our method applies for instance to the study of various sequential quadratic programming sqp schemes the moving balls method the penalized sqp method and the extended sqp method under standard qualification conditions this provides to the best of our knowledge the first general convergence results for general nonlinear programming problems we emphasize the fact that unlike most works on this subjec

quasi newton updates with bounds we develop a quasi newton method which preserves known bounds on the jacobian matrix we show that this update can be computed with the same amount of work as competitive methods in particular we prove that the number of operations required to obtain this update is proportional to the number of nonzeros in the sparsity pattern of the jacobian matrix the method is also shown to share the local convergence properties of broyden s and schubert s method

a type of efficient feasible sqp algorithms for inequality constrained optimization in this paper motivated by zhu et al methods z b zhu k c zhang j b jian an improved sqp algorithm for inequality constrained optimization math meth oper res 58 2003 271 282 zhibin zhu jinbao jian an efficient feasible sqp algorithm for inequality constrained optimization nonlinear anal real world appl 10 2 2009 1220 1228 we propose a type of efficient feasible sqp algorithms to solve nonlinear inequality constrained optimization problems by solving only one qp subproblem with a subset of the constraints estimated as active a class of revised feasible descent directions are generated per single iteration these methods are implementable and globally convergent we also prove that the algorithms have superlinear convergence rate under some mild conditions