new step lengths in conjugate gradient methods consider any conjugate gradient method for finding a zero point of a given gradient whose function is implicit we propose two different types of conditions for selecting the step length using the gradient information only one is used for re proving known convergence results under the same gradient lipschitz assumption moreover if the gradient is merely continuous then we are still able to get some interesting convergence results the other also allows for convergence of the resulting conjugate gradient methods with an application to convergence analysis of the fletcher reeves conjugate gradient method preliminary numerical experiments show the efficiency of our proposed step length rules in practice

benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

a new nonmonotone spectral residual method for nonsmooth nonlinear equations abstract in this paper a new spectral residual method is proposed to solve systems of large scale nonlinear equations where the steplength is obtained by minimizing the residue of an approximate secant equation especially the new steplength can be directly applied into solving strictly convex quadratic function combined with a new nonmonotone line search strategy a new derivative free algorithm called a nonmonotone spectral residual algorithm nsra is developed under mild assumptions global convergence is established for locally lipschitz continuous nonlinear systems compared with the state of the art algorithms available in the literatures the new algorithm is more efficient in solving large scale benchmark test problems

a new efficient conjugate gradient method for unconstrained optimization we propose a nonlinear conjugate gradient method for unconstrained optimization based on solving a new optimization problem our optimization problem combines the good features of the linear conjugate gradient method using some penalty parameters we show that the new method is a subclass of dai liao family the fact that enables us to analyze the family closely as a consequence we obtain an optimal bound for dai liao parameter the global convergence of the new method is investigated under mild assumptions numerical results show that the new method is efficient and robust and outperforms cg descent

dynamical scaling of surface growth in simple lattice models we present extensive simulations of the atomistic edwards wilkinson ew and restricted edwards wilkinson rew models in 2 1 dimensions dynamic finite size scaling analyses of the interfacial width and structure factor provide the estimates for the dynamic exponent z 1 65 0 05 for the ew model and z 2 0 0 1 for the rew model the stochastic contribution to the interface velocity u due to the deposition and diffusion of particles is characterized for both the models using a blocking procedure for the ew model the time displaced temporal correlations in u show nonexponential decay while the temporal correlations decay exponentially for the rew model dynamical scaling of the temporal correlation function for the ew model yields a value of z which is consistent with the estimate obtained from finite size scaling of the interfacial width and structure factor

globally convergent polak ribiere polyak conjugate gradient methods under a modified wolfe line search it is well known that global convergence has not been established for the polak ribiere polyak prp conjugate gradient method using the standard wolfe conditions in the convergence analysis of prp method with wolfe line search the sufficient descent condition and the restriction b k 0 are indispensable see 4 7 this paper shows that these restrictions could be relaxed under some suitable conditions by using a modified wolfe line search global convergence results were established for the prp method some special choices for b k which can ensure the search direction s descent property were also discussed in this paper preliminary numerical results on a set of large scale problems were reported to show that the prp method s computational efficiency is encouraging

on projection algorithms for solving convex feasibility problems due to their extraordinary utility and broad applicability in many areas of classical mathematics and modern physical sciences most notably computerized tomography algorithms for solving convex feasibility problems continue to receive great attention to unify generalize and review some of these algorithms a very broad and flexible framework is investigated several crucial new concepts which allow a systematic discussion of questions on behaviour in general hilbert spaces and on the quality of convergence are brought out numerous examples are given

fixed point optimization algorithms for distributed optimization in networked systems this paper considers a networked system with a finite number of users and deals with the problem of minimizing the sum of all users objective functions over the intersection of all users constraint sets onto which the projection cannot be easily implemented the main objective of this paper is to devise distributed optimization algorithms which enable each user to find the solution of the problem without using other users objective functions and constraint sets to reach this goal we first introduce easily implementable nonexpansive mappings of which the intersection of the fixed point sets is equal to the constraint set in the problem we formulate the problem as a convex minimization problem over the intersection of the fixed point sets of the nonexpansive mappings we then present an iterative algorithm based on the conventional incremental subgradient methods which use the projection for solving the problem the algorithm can be implemented by using nonexpansive mappings other than the project

a globally convergent version of the polak ribiere conjugate gradient method in this paper we propose a new line search algorithm that ensures global convergence of the polak ribiere conjugate gradient method for the unconstrained minimization of nonconvex differentiable functions in particular we show that with this line search every limit point produced by the polak ribiere iteration is a stationary point of the objective function moreover we define adaptive rules for the choice of the parameters in a way that the first stationary point along a search direction can be eventually accepted when the algorithm is converging to a minimum point with positive definite hessian matrix under strong convexity assumptions the known global convergence results can be reobtained as a special case from a computational point of view we may expect that an algorithm incorporating the step size acceptance rules proposed here will retain the same good features of the polak ribiere method while avoiding pathological situations

a nonlinear conjugate gradient method with a strong global convergence property conjugate gradient methods are widely used for unconstrained optimization especially large scale problems the strong wolfe conditions are usually used in the analyses and implementations of conjugate gradient methods this paper presents a new version of the conjugate gradient method which converges globally provided the line search satisfies the standard wolfe conditions the conditions on the objective function are also weak being similar to those required by the zoutendijk condition

nonlinear conjugate gradient methods with structured secant condition for nonlinear least squares problems in this paper we deal with conjugate gradient methods for solving nonlinear least squares problems several newton like methods have been studied for solving nonlinear least squares problems which include the gauss newton method the levenberg marquardt method and the structured quasi newton methods on the other hand conjugate gradient methods are appealing for general large scale nonlinear optimization problems by combining the structured secant condition and the idea of dai and liao 2001 20 the present paper proposes conjugate gradient methods that make use of the structure of the hessian of the objective function of nonlinear least squares problems the proposed methods are shown to be globally convergent under some assumptions finally some numerical results are given

a combined smoothing and regularization method for monotone second order cone complementarity problems the second order cone complementarity problem soccp is a wide class of problems containing the nonlinear complementarity problem ncp and the second order cone programming problem socp recently fukushima luo and tseng siam j optim 12 2001 pp 436 460 extended some merit functions and their smoothing functions for ncp to soccp moreover they derived computable formulas for the jacobians of the smoothing functions and gave conditions for the jacobians to be invertible in this paper we propose a globally and quadratically convergent algorithm which is based on smoothing and regularization methods for solving monotone soccp in particular we study strong semismoothness and jacobian consistency which play an important role in establishing quadratic convergence of the algorithm furthermore we examine the effectiveness of the algorithm by means of numerical experiments

on three term conjugate gradient algorithms for unconstrained optimization this paper presents a project for three term conjugate gradient algorithms development the search direction of the algorithms from this class has three terms and is computed as modifications of the classical conjugate gradient algorithms to satisfy both the descent and the conjugacy conditions an example of three term conjugate gradient algorithm from this class as modifications of the classical and well known hestenes and stiefel or of the cg descent by hager and zhang conjugate gradient algorithms satisfying both the descent and the conjugacy conditions is presented these properties are independent of the line search also the algorithm could be considered as a modification of the memoryless bfgs quasi newton method the new approximation of the minimum is obtained by the general wolfe line search and using by now a standard acceleration technique developed by andrei the proposed three term conjugate gradient algorithm substantially outperforms the well known hestenes and stiefel conjugate gradient algorithm as well as the more elaborated cg descent algorithm a number of 5 applications from minpack 2 test problems collection with 10 6 variables prove that the suggested three term conjugate gradient algorithm is top performer versus cg descent

a modified hestenes stiefel conjugate gradient method with sufficient descent condition and conjugacy condition in this paper by taking a little modification to the hestenes stiefel method we propose a new way to construct descent directions satisfying the sufficient descent condition also an adaptive conjugacy condition and a intrinsic self restarting mechanism are revealed a dynamical adjustment can be regarded as the inheritance and development of properties of standard hestenes stiefel method furthermore we establish global convergence for general nonconvex objective function under mild condition numerical results show that our presented methods can be efficient for solving large scale test problems and therefore is promising

a new conjugate gradient method with guaranteed descent and an efficient line search a new nonlinear conjugate gradient method and an associated implementation based on an inexact line search are proposed and analyzed with exact line search our method reduces to a nonlinear version of the hestenes stiefel conjugate gradient scheme for any inexact line search our scheme satisfies the descent condition bf g k sf t bf d k le frac 7 8 bf g k 2 moreover a global convergence result is established when the line search fulfills the wolfe conditions a new line search scheme is developed that is efficient and highly accurate efficiency is achieved by exploiting properties of linear interpolants in a neighborhood of a local minimizer high accuracy is achieved by using a convergence criterion which we call the approximate wolfe conditions obtained by replacing the sufficient decrease criterion in the wolfe conditions with an approximation that can be evaluated with greater precision in a neighborhood of a local minimum than the usual sufficient decrease criterion numerical comparisons are given with both l bfgs and conjugate gradient methods using the unconstrained optimization problems in the cute library

symmetry based determination of space time functions in nonequilibrium growth processes we study the space time correlation and response functions in nonequilibrium growth processes described by linear stochastic langevin equations exploiting exclusively the existence of space and time dependent symmetries of the noiseless part of these equations we derive expressions for the universal scaling functions of two time quantities which are found to agree with the exact expressions obtained from the stochastic equations of motion the usefulness of the space time functions is illustrated through the investigation of two atomistic growth models the family model and the restricted family model which are shown to belong to a unique universality class in 1 1 and 2 1 space dimensions this corrects earlier studies which claimed that in 2 1 dimensions the two models belong to different universality classes

from microscopic taxation and redistribution models to macroscopic income distributions abstract we present here a general framework expressed by a system of nonlinear differential equations suitable for the modeling of taxation and redistribution in a closed society this framework allows one to describe the evolution of income distribution over the population and to explain the emergence of collective features based on knowledge of the individual interactions by making different choices of the framework parameters we construct different models whose long time behavior is then investigated asymptotic stationary distributions are found which enjoy similar properties as those observed in empirical distributions in particular they exhibit power law tails of pareto type and their lorenz curves and gini indices are consistent with some real world ones

stochastic effects in a discretized kinetic model of economic exchange linear stochastic models and discretized kinetic theory are two complementary analytical techniques used for the investigation of complex systems of economic interactions the former employ langevin equations with an emphasis on stock trade the latter is based on systems of ordinary differential equations and is better suited for the description of binary interactions taxation and welfare redistribution we propose a new framework which establishes a connection between the two approaches by introducing random fluctuations into the kinetic model based on langevin and fokker planck formalisms numerical simulations of the resulting model indicate positive correlations between the gini index and the total wealth that suggest a growing inequality with increasing income further analysis shows in the presence of a conserved total wealth a simultaneous decrease in inequality as social mobility increases in conformity with economic data

a practical pr conjugate gradient method only using gradient abstract in this paper we consider the polak ribiere or polak ribiere plus conjugate gradient method for solving optimality condition of an unconstrained minimization problem we give two new steplength rules only using gradient and under gradient lipschitz assumption prove this method s global convergence correspondingly then we develop a practical polak ribiere plus method whose steplength is located by one inequality only using gradient and report promising numerical results on high accuracy solution for some standard test problems when compared to the state of art methods in this research direction importantly our work provides a new idea of devising a practical version of the celebrated polak ribiere or polak ribiere plus method

acceleration method for convex optimization over the fixed point set of a nonexpansive mapping the existing algorithms for solving the convex minimization problem over the fixed point set of a nonexpansive mapping on a hilbert space are based on algorithmic methods such as the steepest descent method and conjugate gradient methods for finding a minimizer of the objective function over the whole space and attach importance to minimizing the objective function as quickly as possible meanwhile it is of practical importance to devise algorithms which converge in the fixed point set quickly because the fixed point set is the set with the constraint conditions that must be satisfied in the problem this paper proposes an algorithm which not only minimizes the objective function quickly but also converges in the fixed point set much faster than the existing algorithms and proves that the algorithm with diminishing step size sequences strongly converges to the solution to the convex minimization problem we also analyze the proposed algorithm with each of the fletcher reeves polak ribiere polyak hestenes stiefel and dai yuan formulas used in the conventional conjugate gradient methods and show that there is an inconvenient possibility that their algorithms may not converge to the solution to the convex minimization problem we numerically compare the proposed algorithm with the existing algorithms and show its effectiveness and fast convergence

n step quadratic convergence of the mprp method with a restart strategy abstract it is well known that the prp conjugate gradient method with exact line search is globally and linearly convergent if a restart strategy is used the convergence rate of the method can be an n step superlinear quadratic convergence recently zhang et al l zhang w zhou d h li a descent modified polak ribiere polyak conjugate gradient method and its global convergence ima j numer anal 26 2006 629 640 developed a modified prp mprp method that is globally convergent if an inexact line search is used in this paper we investigate the convergence rate of the mprp method with inexact line search we first show that the mprp method with armijo line search or wolfe line search is linearly convergent we then show that the mprp method with a restart strategy still retains n step superlinear quadratic convergence if the initial steplength is appropriately chosen we also do some numerical experiments the results show that the restart mprp method does converge quadratically moreover it is more efficient than the non restart method

improved fletcher reeves and dai yuan conjugate gradient methods with the strong wolfe line search abstract the conjugate gradient methods cgms are very effective iterative methods for solving large scale unconstrained optimization the aim of this work is to improve the fletcher reeves and dai yuan cgms first based on the conjugate parameters of the fletcher reeves fr method and the dai yuan dy method and combining the second inequality of the strong wolfe line search two new conjugate parameters are constructed second using the two new conjugate parameters another fr type conjugate parameter is presented third utilizing the strong wolfe line search to yield the steplength three improved cgms are proposed for large scale unconstrained optimization under usual assumptions the improved methods are all proved to possess sufficient descent property and global convergence finally three group experiments and their corresponding performance profiles are reported which show that the proposed methods are very promising

new conjugacy condition and related new conjugate gradient methods for unconstrained optimization the conjugate gradient cg method has played a special role in solving large scale nonlinear optimization due to the simplicity of their iterations and their very low memory requirements based on a new quasi newton equation proposed in z wei g li l qi new quasi newton methods for unconstrain optimization preprint z wei g yu g yuan z lian the superlinear convergence of a modified bfgs type method for unconstrained optimization comput optim appl 29 3 2004 315 332 we establish a new conjugacy condition for cg methods and propose several new cg methods it is a interesting feature that these new cg methods take both the gradient and function value information under some suitable conditions the global convergence is achieved for these methods the numerical results show that one of our new cg methods is very encouraging

computational method for solving a stochastic linear quadratic control problem given an unsolvable stochastic algebraic riccati equation we discuss a stochastic linear quadratic control problem in which a stochastic algebraic riccati equation derived from the problem is unsolvable the riccati equation has no solution when the state and control weighting matrices in the objective function of the problem are indefinite and the conventional methods cannot solve the problem when the riccati equation itself is unsolvable we first show that the optimal value of the problem is finite next we formulate a compromise solution to the stochastic algebraic riccati equation and show that the problem can be solved via this compromise solution under certain assumptions moreover we propose a novel computational method for finding the compromise solution based on iterative techniques for a convex optimization problem over the fixed point set of a certain nonexpansive mapping numerical examples demonstrate the effectiveness of this method

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

an adaptive three term conjugate gradient method based on self scaling memoryless bfgs matrix abstract due to its simplicity and low memory requirement conjugate gradient methods are widely used for solving large scale unconstrained optimization problems in this paper we propose a three term conjugate gradient method the search direction is given by a symmetrical perry matrix which contains a positive parameter the value of this parameter is determined by minimizing the distance of this matrix and the self scaling memoryless bfgs matrix in the frobenius norm the sufficient descent property of the generated directions holds independent of line searches the global convergence of the given method is established under wolfe line search for general non convex functions numerical experiments show that the proposed method is promising

conjugate gradient methods based on secant conditions that generate descent search directions for unconstrained optimization conjugate gradient methods have been paid attention to because they can be directly applied to large scale unconstrained optimization problems in order to incorporate second order information of the objective function into conjugate gradient methods dai and liao 2001 proposed a conjugate gradient method based on the secant condition however their method does not necessarily generate a descent search direction on the other hand hager and zhang 2005 proposed another conjugate gradient method which always generates a descent search direction in this paper combining dai liao s idea and hager zhang s idea we propose conjugate gradient methods based on secant conditions that generate descent search directions in addition we prove global convergence properties of the proposed methods finally preliminary numerical results are given

convergence properties of algorithms for nonlinear optimization this paper reviews some of the most successful methods for unconstrained constrained and nondifferentiable optimization calculations particular attention is given to the contribution that theoretical analysis has made to the development of algorithms it seems that practical considerations provide the main new ideas and that subsequent theoretical studies give improvements to algorithms coherence to the subject and better understanding

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

properties and numerical performance of quasi newton methods with modified quasi newton equations quasi newton qn equation plays a core role in contemporary nonlinear optimization the usual qn equation employs only the gradients but ignores the available function value information in this paper we derive a class of modified qn equations with a vector parameter which use both available gradient and function value information the modified quasi newton methods maintain most properties of the usual quasi newton methods meanwhile they achieve a higher order accuracy in approximating the second order curvature of the problem functions than the usual ones do numerical experiments are reported which support the theoretical analyses and show the advantages of the modified qn methods over the usual ones

a smoothing conjugate gradient method for solving systems of nonsmooth equations many problems in real world are reduced to systems of nonsmooth equations and hence many researchers study numerical methods for solving systems of nonsmooth equations as numerical methods for solving systems of nonsmooth equations newton like methods are known as efficient numerical methods however these methods are not necessarily applied directly to large scale problems because these methods need to store matrices in this paper we propose a smoothing method which is based on the nonlinear conjugate gradient method and does not store any matrices for solving systems of nonsmooth equations in addition we prove the global convergence property of the proposed method under standard assumptions finally we give some preliminary numerical results

global convergence of a modified fletcher reeves conjugate gradient method with armijo type line search in this paper we are concerned with the conjugate gradient methods for solving unconstrained optimization problems it is well known that the direction generated by a conjugate gradient method may not be a descent direction of the objective function in this paper we take a little modification to the fletcher reeves fr method such that the direction generated by the modified method provides a descent direction for the objective function this property depends neither on the line search used nor on the convexity of the objective function moreover the modified method reduces to the standard fr method if line search is exact under mild conditions we prove that the modified method with armijo type line search is globally convergent even if the objective function is nonconvex we also present some numerical results to show the efficiency of the proposed method

fixed point optimization algorithm and its application to power control in cdma data networks we discuss the variational inequality problem for a continuous operator over the fixed point set of a nonexpansive mapping one application of this problem is a power control for a direct sequence code division multiple access data network for such a power control each user terminal has to be able to quickly transmit at an ideal power level such that it can get a sufficient signal to interference plus noise ratio and achieve the required quality of service iterative algorithms to solve this problem should not involve auxiliary optimization problems and complicated computations to ensure this we devise a fixed point optimization algorithm for the variational inequality problem and perform a convergence analysis on it we give numerical examples of the algorithm as a power control

general four step discrete time zeroing and derivative dynamics applied to time varying nonlinear optimization abstract time varying nonlinear optimization tvno problems are considered as important issues in various scientific disciplines and industrial applications in this paper the continuous time derivative dynamics ctdd model is developed for obtaining the real time solutions of tvno problems furthermore aiming to remedy the weaknesses of ctdd model a continuous time zeroing dynamics ctzd model is presented and investigated for potential digital hardware realization by using bilinear transform a general four step zhang et al discretization zead formula is proposed and applied to the discretization of both ctdd and ctzd models a general four step discrete time derivative dynamics general four step dtdd model and a general four step discrete time zeroing dynamics general four step dtzd model are proposed on the basis of this general four step zead formula further theoretical analyses indicate that the general four step dtzd model is zero stable consistent and convergent with the truncation error of o g 4 which denotes a vector with every entries being o g 4 with g denoting the sampling period theoretical analyses also indicate that the maximal steady state residual error mssre has an o g 4 pattern confirmedly the efficacy and accuracy of the general four step dtdd and dtzd models are further illustrated by numerical examples

a note on convergence of prp method with new nonmonotone line search we in this note drop a lipschitz constant in a grippo lucidi type step length rule recently proposed by shi and shen z shi j shen convergence of prp method with new nonmonotone line search applied mathematics and computation 181 1 2006 423 431 and the original convergence remains valid

notes on the dai yuan yuan modified spectral gradient method in this paper we give some notes on the two modified spectral gradient methods which were developed in 10 these notes present the relationship between their stepsize formulae and some new secant equations in the quasi newton method in particular we also introduce another two new choices of stepsize by using an efficient nonmonotone line search technique we propose some new spectral gradient methods under some mild conditions we show that these proposed methods are globally convergent numerical experiments on a large number of test problems from the cuter library are also reported which show that the efficiency of these proposed methods

convergence properties of nonlinear conjugate gradient methods recently important contributions on convergence studies of conjugate gradient methods were made by gilbert and nocedal siam j optim 2 1992 pp 21 42 they introduce a sufficient descent condition to establish global convergence results although this condition is not needed in the convergence analyses of newton and quasi newton methods gilbert and nocedal hint that the sufficient descent condition which was enforced by their two stage line search algorithm may be crucial for ensuring the global convergence of conjugate gradient methods this paper shows that the sufficient descent condition is actually not needed in the convergence analyses of conjugate gradient methods consequently convergence results on the fletcher reeves and polak ribiere type methods are established in the absence of the sufficient descent condition to show the differences between the convergence properties of fletcher reeves and polak ribiere type methods two examples are constructed showing that neither the boundedness of the level set nor the restriction beta k geq 0 can be relaxed for the polak ribiere type methods

iterative algorithm for triple hierarchical constrained nonconvex optimization problem and its application to network bandwidth allocation in this paper we discuss a variational inequality problem with the gradient of a nonconvex objective function in which the constraint set composed of the absolute set and the subsidiary sets is not feasible we formulate a compromise solution of the problem by using a solution to a variational inequality for the gradient over a subset of the absolute set with the elements closest to the subsidiary sets in terms of the norm the objective of this paper is to enable us to discuss a network bandwidth allocation problem with a nonconcave utility function in which the compoundable constraints about the preferable transmission rate fall in the infeasible region for such a bandwidth allocation the ideal bandwidth must be quickly shared among traffic sources so as to satisfy the capacity constraints absolute constraints and all sources demands regarding the transmission rate subsidiary constraints as much as possible iterative algorithms for the bandwidth allocation should thus not involve auxiliary opti

relaxation to steady states and dynamical exponents in deposition models abstract considering some deposition models with limited mobility we show that the typical decay of the interface width to its saturation value is exponential which defines the crossover or saturation time we present a method to calculate a characteristic time 0 proportional to and estimate the dynamical exponent z in one dimensional substrates of lengths l 2048 the method is applied to the family model the restricted solid on solid rsos model and the ballistic deposition effective exponents z l converge to asymptotic values consistent with the corresponding continuum theories for the two dimensional family model the expected dynamic scaling hypothesis suggests a particular definition of 0 that leads to z 2 improving previous calculations based on data collapse methods for the two dimensional rsos model we obtain z 1 6 and

a descent extension of the polak ribiere polyak conjugate gradient method based on an eigenvalue analysis a descent class of two parameter extension of the conjugate gradient method proposed by polak and ribiere 1969 and polyak 1969 is suggested it is interesting that the one parameter class of descent conjugate gradient methods proposed by yu et al 2008 is a member of the suggested class global convergence analysis for the methods of the suggested class is made briefly preliminary numerical results are reported they demonstrate proper choices for the parameters of the suggested class of the conjugate gradient methods that may lead to a promising computational performance

a simple three term conjugate gradient algorithm for unconstrained optimization a simple three term conjugate gradient algorithm which satisfies both the descent condition and the conjugacy condition is presented this algorithm is a modification of the hestenes and stiefel algorithm hestenes and stiefel 1952 10 or that of hager and zhang hager and zhang 2005 23 in such a way that the search direction is descent and it satisfies the conjugacy condition these properties are independent of the line search also the algorithm could be considered as a modification of the memoryless bfgs quasi newton method the new approximation of the minimum is obtained by the general wolfe line search now using a standard acceleration technique developed by andrei 2009 27 for uniformly convex functions under standard assumptions the global convergence of the algorithm is proved numerical comparisons of the suggested three term conjugate gradient algorithm versus six other three term conjugate gradient algorithms using a set of 750 unconstrained optimization problems show that all these computational schemes have similar performances the suggested one being slightly faster and more robust the proposed three term conjugate gradient algorithm substantially outperforms the well known hestenes and stiefel conjugate gradient algorithm as well as the more elaborate cg descent algorithm using five applications from the minpack 2 test problem collection averick et al 1992 25 with 10 6 variables we show that the suggested three term conjugate gradient algorithm is the top performer versus cg descent

the dai liao nonlinear conjugate gradient method with optimal parameter choices minimizing two different upper bounds of the matrix which generates search directions of the nonlinear conjugate gradient method proposed by dai and liao two modified conjugate gradient methods are proposed under proper conditions it is briefly shown that the methods are globally convergent when the line search fulfills the strong wolfe conditions numerical comparisons between the implementations of the proposed methods and the conjugate gradient methods proposed by hager and zhang and dai and kou are made on a set of unconstrained optimization test problems of the cuter collection the results show the efficiency of the proposed methods in the sense of the performance profile introduced by dolan and more

hypercube stacking a potts spin model for surface growth we present a deposition and evaporation model for surface growth under a solid on solid constraint we generalize the ising spin representation of a two dimensional surface by blote and hilhorst to ad dimensional surface of a d 1 dimensional hypercubic lattice the allowed surface configurations correspond to the degenerate ground states of a chirald state potts model we describe a vectorized multisite coding implementation for the corresponding kinetic potts spin model ford 2 andd 3 for thed 2 equilibrium surface our simulation results show excellent agreement with an exact analysis

langevin equations for fluctuating surfaces exact langevin equations are derived for the height fluctuations of surfaces driven by the deposition of material from a molecular beam we consider two types of model deposition models where growth proceeds by the deposition and instantaneous local relaxation of particles with no subsequent movement and models with concurrent random deposition and surface diffusion starting from a chapman kolmogorov equation the deposition relaxation and hopping rules of these models are first expressed as transition rates within a master equation for the joint height probability density function the kramers moyal van kampen expansion of the master equation in terms of an appropriate largeness parameter yields according to a limit theorem due to kurtz stoch proc appl 6 223 1978 a fokker planck equation that embodies the statistical properties of the original lattice model the statistical equivalence of this fokker planck equation solved in terms of its associated langevin equation and solutions of the chapman kolmogorov equation as determined by kinetic monte carlo kmc simulations of the lattice transition rules is demonstrated explicitly by comparing the surface roughness and the lateral height correlations obtained from the two formulations for the edwards wilkinson proc roy soc london ser a 381 17 1982 and wolfvillain europhys lett 13 389 1990 deposition models and for a model with random deposition and surface diffusion in each case as the largeness parameter is increased the langevin equation converges to the surface roughness and lateral height correlations produced by kmc simulations for all times including the crossover between different scaling regimes we conclude by examining some of the wider implications of these results including applications to heteroepitaxial systems and the passage to the continuum limit

acceleration of conjugate gradient algorithms for unconstrained optimization conjugate gradient methods are important for large scale unconstrained optimization this paper proposes an acceleration of these methods using a modification of steplength the idea is to modify in a multiplicative manner the steplength a k computed by wolfe line search conditions by means of a positive parameter h k in such a way to improve the behavior of the classical conjugate gradient algorithms it is shown that for uniformly convex functions the convergence of the accelerated algorithm is still linear but the reduction in function values is significantly improved numerical comparisons with some conjugate gradient algorithms using a set of 750 unconstrained optimization problems some of them from the cute library show that the accelerated computational scheme outperform the corresponding conjugate gradient algorithms

multi step quasi newton methods for optimization abstract quasi newton methods update at each iteration the existing hessian approximation or its inverse by means of data deriving from the step just completed we show how multi step methods employing in addition data from previous iterations may be constructed by means of interpolating polynomials leading to a generalization of the secant or quasi newton equation the issue of positive definiteness in the hessian approximation is addressed and shown to depend on a generalized version of the condition which is required to hold in the original single step methods the results of extensive numerical experimentation indicate strongly that computational advantages can accrue from such an approach by comparison with single step methods particularly as the dimension of the problem increases

another improved wei yao liu nonlinear conjugate gradient method with sufficient descent property abstract recently zhang 13 take a little modification to the wei yao liu nonlinear conjugate gradient method proposed by wei et al 10 such that the modified method called nprp method in zhang 13 satisfies sufficient descent condition with greater parameter 0 1 2 in the strong wolfe line search and converges globally for nonconvex minimization with the strong wolfe line search in this paper we take a little modification to the nprp method such that the modified nprp method possesses better properties than the nprp method in zhang 13 firstly the modified nprp method possesses the sufficient descent property without any line searches secondly the modified nprp method converges globally for nonconvex minimization with wolfe line search or armijo line search moreover we extend these results to the hestenes stiefel hs method and prove that the modified hs method also possesses sufficient descent property and global convergence with the standard wolfe conditions numerical results are reported by utilizing some test problems in the cute library

an efficient dai liao type conjugate gradient method by reformulating the cg parameter in the search direction equation abstract based on the modified bfgs method proposed by li and fukushima 2001 we present a new value of the parameter t in dai liao conjugate gradient method under mild assumptions we establish the global convergence property of the proposed method numerical results on some test problems in the cutest library illustrate computational efficiency of the new method

convergence of prp method with new nonmonotone line search abstract in this paper we develop a new nonmonotone line search for prp conjugate gradient method polak ribiere polyak for minimizing functions having lipschitz continuous partial derivatives the nonmonotone line search can guarantee the global convergence of original prp method under some mild conditions numerical experiments show that prp method with the new nonmonotone line search is available and efficient in practical computation

a norm descent derivative free algorithm for solving large scale nonlinear symmetric equations abstract in this paper we propose a norm descent derivative free algorithm for solving large scale nonlinear symmetric equations without involving any information of the gradient or jacobian matrix by using some approximate substitutions the proposed algorithm is extended from an efficient three term conjugate gradient method for solving unconstrained optimization problems and inherits some nice properties such as simple structure low storage requirements and symmetric property under some appropriate conditions the global convergence is proved finally the numerical experiments and comparisons show that the proposed algorithm is very effective for large scale problems

a new conjugate gradient algorithm for training neural networks based on a modified secant equation conjugate gradient methods have been established as excellent neural network training methods due to the simplicity of their iteration numerical efficiency and their low memory requirements in this work we propose a conjugate gradient neural network training algorithm which guarantees sufficient descent using any line search avoiding thereby the usually inefficient restarts moreover it approximates the second order curvature information of the error surface with a high order accuracy by utilizing a new modified secant condition under mild conditions we establish that the global convergence of our proposed method experimental results provide evidence that our proposed method is in general superior to the classical conjugate gradient training methods and has a potential to significantly enhance the computational efficiency and robustness of the training process

a three term conjugate gradient method with sufficient descent property for unconstrained optimization conjugate gradient methods are widely used for solving large scale unconstrained optimization problems because they do not need the storage of matrices in this paper we propose a general form of three term conjugate gradient methods which always generate a sufficient descent direction we give a sufficient condition for the global convergence of the proposed method moreover we present a specific three term conjugate gradient method based on the multistep quasi newton method finally some numerical results of the proposed method are given

on hager and zhang s conjugate gradient method with guaranteed descent abstract in this paper we point out that hager zhang family of conjugate gradient methods are based on the spectral scaling secant equation are really members of dai liao type of gradient methods with common constant parameters we propose a possible efficient parameter for hager zhang s conjugate gradient method with guaranteed descent cg descent and present a conjugate gradient method cg descent1 the cg descent1 method preserves the same global convergence properties as cg descent method of hager and zhang and is numerically tested compared with cg descent method and other conjugate gradient methods considering the difference of the search directions approximation to the search direction of the memoryless bfgs method we give the reason why the proposed method is more efficient than cg descent method under some conditions and further find an optimal conjugate gradient method and an approximation optimal conjugate method in hager zhang family of conjugate gradient methods

two modified dy conjugate gradient methods for unconstrained optimization problems abstract in this paper we study the unconstrained optimization problems and the two modified dy conjugate gradient methods ddy1 method and ddy2 method are proposed under the dy conjugate gradient method by using the standard wolfe line search we prove the global convergence of the two methods the search direction of ddy1 method is descent with the standard wolfe line search the search direction generated by the ddy2 method is sufficient descent in which the property does not depend on any line search preliminary numerical results show that the two methods are effective

non linear kinetics underlying generalized statistics abstract the purpose of the present effort is threefold firstly it is shown that there exists a principle that we call kinetical interaction principle kip underlying the non linear kinetics in particle systems independently on the picture kramers boltzmann used to describe their time evolution secondly the kip imposes the form of the generalized entropy associated to the system and permits to obtain the particle statistical distribution both as stationary solution of the non linear evolution equation and as the state which maximizes the generalized entropy thirdly the kip allows on one hand to treat all the classical or quantum statistical distributions already known in the literature in a unifying scheme and on the other hand suggests how we can introduce naturally new distributions finally as a working example of the approach to the non linear kinetics here presented a new non extensive statistics is constructed and studied starting from a one parameter deformation of the exponential function holding the relation f x f x 1

wealth condensation in a simple model of economy abstract we introduce a simple model of economy where the time evolution is described by an equation capturing both exchange between individuals and random speculative trading in such a way that the fundamental symmetry of the economy under an arbitrary change of monetary units is insured we investigate a mean field limit of this equation and show that the distribution of wealth is of the pareto power law type the pareto behaviour of the tails of this distribution appears to be robust for finite range models as shown using both a mapping to the random directed polymer problem as well as numerical simulations in this context a phase transition between an economy dominated by a few individuals and a situation where the wealth is more evenly spread out is found an interesting outcome is that the distribution of wealth tends to be very broadly distributed when exchanges are limited either in amplitude or topologically favouring exchanges and less surprisingly increasing taxes seems to be an efficient way to reduce inequalities

an improved wei yao liu nonlinear conjugate gradient method for optimization computation in this paper we take a little modification to the wei yao liu nonlinear conjugate gradient method proposed by wei et al z wei s yao l liu the convergence properties of some new conjugate gradient methods appl math comput 183 2006 1341 1350 such that the modified method possesses better convergence properties in fact we prove that the modified method satisfies sufficient descent condition with greater parameter email protected 0 12 in the strong wolfe line search and converges globally for nonconvex minimization we also extend these results to the hestenes stiefel method and prove that the modified hs method is globally convergent for nonconvex functions with the standard wolfe conditions numerical results are reported by using some test problems in the cute library

edwards wilkinson equation from lattice transition rules continuum equations of motion for the height fluctuations of lattice growth models are derived from their transition rules by regularizing and coarse graining the associated discrete langevin equations for models with random deposition followed by instantaneous relaxation to a neighboring site based on identifying the local height minimum our methodology yields the edwards wilkinson equation the application of this procedure to other growth models is discussed

a modified bfgs method and its global convergence in nonconvex minimization in this paper we propose a modification of the bfgs method for unconstrained optimization a remarkable feature of the proposed method is that it possesses a global convergence property even without convexity assumption on the objective function under certain conditions we also establish superlinear convergence of the method

the edwards wilkinson model revisited large scale simulations of dynamic scaling in 2 1 dimensions abstract using extensive simulations of surface growth in the 2 1 dimensional atomistic model of edwards and wilkinson ew we have calculated both interfacial width and the structure factor to extract the dynamic exponent z in contrast to theoretical expectations finite size scaling of the data for both surface properties is achieved using z 1 63 instead of the predicted value of 2 0 a restricted variant of the ew model gives z 2 0 clarifying these non trivial differences in dynamical correlations now represents an intriguing theoretical challenge

a nonsmooth version of newton s method newton s method for solving a nonlinear equation of several variables is extended to a nonsmooth case by using the generalized jacobian instead of the derivative this extension includes the b derivative version of newton s method as a special case convergence theorems are proved under the condition of semismoothness it is shown that the gradient function of the augmented lagrangian forc2 nonlinear programming is semismooth thus the extended newton s method can be used in the augmented lagrangian method for solving nonlinear programs

new version of the three term conjugate gradient method based on spectral scaling conjugacy condition that generates descent search direction in this paper a general form of three term conjugate gradient method is presented in which the search directions simultaneously satisfy the dai liao conjugacy condition and sufficient descent property in addition the choice for an optimal parameter is suggested in the sense that the condition number of the iteration matrix could arrives at its minimum which can be regarded as the inheritance and development of the spectral scaling quasi newton equation different from the existent methods a new update strategy in constructing the search direction is proposed to establish the global convergence for the general function numerical results show our algorithm is practical and effective for the test problems

new properties of forward backward splitting and a practical proximal descent algorithm abstract in this paper we discuss a proximal descent algorithm for finding a zero of the sum of two maximal monotone operators in a real hilbert space some new properties of forward backward splitting are given which extend the well known properties of the usual projection then they are used to analyze the weak convergence of the proximal descent algorithm without assuming lipschitz continuity of the forward operator we also give a new technique of choosing trial values of the step length involved in an armijo like condition which returns the not necessarily decreasing step length self adaptively rudimentary numerical experiments show that it is effective in practical implementations

a nonlinear conjugate gradient algorithm with an optimal property and an improved wolfe line search in this paper we seek the conjugate gradient direction closest to the direction of the scaled memoryless bfgs method and propose a family of conjugate gradient methods for unconstrained optimization an improved wolfe line search is also proposed which can avoid a numerical drawback of the original wolfe line search and guarantee the global convergence of the conjugate gradient method under mild conditions to accelerate the algorithm we introduce adaptive restarts along negative gradients based on the extent to which the function approximates some quadratic function during previous iterations numerical experiments with the cuter collection show that the proposed algorithm is promising

a new class of spectral conjugate gradient methods based on a modified secant equation for unconstrained optimization conjugate gradient methods have played a special role for solving large scale optimization problems due to the simplicity of their iteration convergence properties and their low memory requirements in this work we propose a new class of spectral conjugate gradient methods which ensures sufficient descent independent of the accuracy of the line search moreover an attractive property of our proposed methods is that they achieve a high order accuracy in approximating the second order curvature information of the objective function by utilizing the modified secant condition proposed by babaie kafaki et al s babaie kafaki r ghanbari n mahdavi amiri two new conjugate gradient methods based on modified secant equations journal of computational and applied mathematics 234 2010 1374 1386 further a global convergence result for general functions is established provided that the line search satisfies the wolfe conditions our numerical experiments indicate that our proposed methods are preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness