new step lengths in conjugate gradient methods consider any conjugate gradient method for finding a zero point of a given gradient whose function is implicit we propose two different types of conditions for selecting the step length using the gradient information only one is used for re proving known convergence results under the same gradient lipschitz assumption moreover if the gradient is merely continuous then we are still able to get some interesting convergence results the other also allows for convergence of the resulting conjugate gradient methods with an application to convergence analysis of the fletcher reeves conjugate gradient method preliminary numerical experiments show the efficiency of our proposed step length rules in practice

benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

on an open problem concerning regular magic squares of odd order abstract in this paper we give an answer to an open problem posed by m z lee et al 2012 2 more precisely we prove that the classical regular magic square of odd order produced by the centroskew s circulant matrix with the assignment a j j 1 j 1 2 n 1 2 is always nonsingular moreover an explicit formula for computing the eigenvalues of classical regular magic squares of odd order is given

a new class of supermemory gradient methods in this paper we propose a new class of supermemory gradient methods for unconstrained optimization problems trust region approach is used in the new algorithms to guarantee the global convergence in each iteration the new algorithms generate a suitable trust region radius automatically and obtain the next iterate by solving a simple subproblem these algorithms converge stably and averagely due to using more iterative information at each iteration and can be reduced to quasi newton methods when the iterate is close to the optimal solution numerical results show that this new class of supermemory gradient methods is effective in practical computation

a conjugate gradient method with descent direction for unconstrained optimization a modified conjugate gradient method is presented for solving unconstrained optimization problems which possesses the following properties i the sufficient descent property is satisfied without any line search ii the search direction will be in a trust region automatically iii the zoutendijk condition holds for the wolfe powell line search technique iv this method inherits an important property of the well known polak ribiere polyak prp method the tendency to turn towards the steepest descent direction if a small step is generated away from the solution preventing a sequence of tiny steps from happening the global convergence and the linearly convergent rate of the given method are established numerical results show that this method is interesting

a globally convergent version of the polak ribiere conjugate gradient method in this paper we propose a new line search algorithm that ensures global convergence of the polak ribiere conjugate gradient method for the unconstrained minimization of nonconvex differentiable functions in particular we show that with this line search every limit point produced by the polak ribiere iteration is a stationary point of the objective function moreover we define adaptive rules for the choice of the parameters in a way that the first stationary point along a search direction can be eventually accepted when the algorithm is converging to a minimum point with positive definite hessian matrix under strong convexity assumptions the known global convergence results can be reobtained as a special case from a computational point of view we may expect that an algorithm incorporating the step size acceptance rules proposed here will retain the same good features of the polak ribiere method while avoiding pathological situations

a nonlinear conjugate gradient method with a strong global convergence property conjugate gradient methods are widely used for unconstrained optimization especially large scale problems the strong wolfe conditions are usually used in the analyses and implementations of conjugate gradient methods this paper presents a new version of the conjugate gradient method which converges globally provided the line search satisfies the standard wolfe conditions the conditions on the objective function are also weak being similar to those required by the zoutendijk condition

detection power estimation efficiency and predictability in event related fmri experimental designs for event related functional magnetic resonance imaging can be characterized by both their detection power a measure of the ability to detect an activation and their estimation efficiency a measure of the ability to estimate the shape of the hemodynamic response randomized designs offer maximum estimation efficiency but poor detection power while block designs offer good detection power at the cost of minimum estimation efficiency periodic single trial designs are poor by both criteria we present here a theoretical model of the relation between estimation efficiency and detection power and show that the observed trade off between efficiency and power is fundamental using the model we explore the properties of semirandom designs that offer intermediate trade offs between efficiency and power these designs can simultaneously achieve the estimation efficiency of randomized designs and the detection power of block designs at the cost of increasing the length of an experiment by less than a factor of 2 experimental designs can also be characterized by their predictability a measure of the ability to circumvent confounds such as habituation and anticipation we examine the relation between detection power estimation efficiency and predictability and show that small increases in predictability can offer significant gains in detection power with only a minor decrease in estimation efficiency

two new conjugate gradient methods based on modified secant equations following the approach proposed by dai and liao we introduce two nonlinear conjugate gradient methods for unconstrained optimization problems one of our proposed methods is based on a modified version of the secant equation proposed by zhang deng and chen and zhang and xu and the other is based on the modified bfgs update proposed by yuan an interesting feature of our methods is their account of both the gradient and function values under proper conditions we show that one of the proposed methods is globally convergent for general functions and that the other is globally convergent for uniformly convex functions to enhance the performance of the line search procedure we also propose a new approach for computing the initial steplength to be used for initiating the procedure we provide a comparison of implementations of our methods with the efficient conjugate gradient methods proposed by dai and liao and hestenes and stiefel numerical test results show the efficiency of our proposed methods

spectra of order 4 special magic squares abstract a new parameterization for order 4 regular or associative magic square matrices leads to general formulas for their eigenvalues eigenvectors and singular value decomposition known transformations extend these results to order 4 pandiagonal and bent diagonal magic squares the effect of various transformations on the eigenvalues and singular values of these special magic squares is considered numerical examples are presented and numerical values are obtained from simple formulas for the eigenvalues and singular values of each of the 48 natural pandiagonal regular and bent diagonal magic squares of order 4 and their reflections

a gradient related algorithm with inexact line searches in this paper a new gradient related algorithm for solving large scale unconstrained optimization problems is proposed the new algorithm is a kind of line search method the basic idea is to choose a combination of the current gradient and some previous search directions as a new search direction and to find a step size by using various inexact line searches using more information at the current iterative step may improve the performance of the algorithm this motivates us to find some new gradient algorithms which may be more effective than standard conjugate gradient methods uniformly gradient related conception is useful and it can be used to analyze global convergence of the new algorithm the global convergence and linear convergence rate of the new algorithm are investigated under diverse weak conditions numerical experiments show that the new algorithm seems to converge more stably and is superior to other similar methods in many situations

a new conjugate gradient method with guaranteed descent and an efficient line search a new nonlinear conjugate gradient method and an associated implementation based on an inexact line search are proposed and analyzed with exact line search our method reduces to a nonlinear version of the hestenes stiefel conjugate gradient scheme for any inexact line search our scheme satisfies the descent condition bf g k sf t bf d k le frac 7 8 bf g k 2 moreover a global convergence result is established when the line search fulfills the wolfe conditions a new line search scheme is developed that is efficient and highly accurate efficiency is achieved by exploiting properties of linear interpolants in a neighborhood of a local minimizer high accuracy is achieved by using a convergence criterion which we call the approximate wolfe conditions obtained by replacing the sufficient decrease criterion in the wolfe conditions with an approximation that can be evaluated with greater precision in a neighborhood of a local minimum than the usual sufficient decrease criterion numerical comparisons are given with both l bfgs and conjugate gradient methods using the unconstrained optimization problems in the cute library

a note about wyl s conjugate gradient method and its applications this paper reviews the development of different versions of nonlinear conjugate gradient methods shortly the special attentions were given to the wyl method which was proposed by zengxin wei et al the convergence properties of some conjugate gradient methods applied mathematics and computation 183 2006 1341 1350 and its applications

the proof of the sufficient descent condition of the wei yao liu conjugate gradient method under the strong wolfe powell line search abstract in z wei et al the convergence properties of some conjugate gradient methods appl math comput 183 2006 1341 1350 wei et al proposed a new conjugate gradient method called wyl method which has good numerical experiments under the sufficient descent condition and wwp line search the given method s convergent results have been established by wei et al so the sufficient descent condition is important for the wyl method in this paper we prove that by restricting the parameter 1 4 under the swp line search condition the wyl method possess the sufficient descent condition which deduces the global convergent results of the wyl method under swp line search

convergence of the polak ribiere polyak conjugate gradient method abstract in this paper we consider the global convergence of the polak ribiere polyak abbreviated prp conjugate gradient method for unconstrained optimization problems a new armijo type line search is proposed for the original prp method and some convergence properties are given under some mild conditions the new armijo type line search can make the prp method choose a suitable initial step size so as to decrease the function evaluations at each iteration and improve the performance of the prp method numerical results show that the prp method with the new armijo type line search is more efficient than other similar methods in practical computation

a construction of regular magic squares of odd order abstract a magic square is an n n array of numbers whose rows columns and the two diagonals sum to a regular magic square satisfies the condition that the entries symmetrically placed with respect to the center sum to 2 n using circulant matrices we describe a construction of regular classical magic squares that are nonsingular for all odd orders a similar construction is given that produces regular classical magic squares that are singular for odd composite orders this paper is an extension of 3

the global convergence of the polak ribiere polyak conjugate gradient algorithm under inexact line search for nonconvex functions abstract powell 1984 and dai 2003 constructed respectively a counterexample to show that the polak ribiere polyak prp conjugate gradient algorithm fails to globally converge for nonconvex functions even when the exact line search technique is used which implies similar failure of the weak wolfe powell wwp inexact line search technique does another inexact line search technique exist that can ensure global convergence for nonconvex functions this paper gives a positive answer to this question and proposes a modified wwp mwwp inexact line search technique an algorithm is presented that uses the mwwp inexact line search technique the next point x k 1 generated by the prp formula is accepted if a positive condition holds and otherwise x k 1 is defined by a technique for projection onto a parabolic surface the proposed prp conjugate gradient algorithm is shown to possess global convergence under inexact line search for nonconvex functions numerical performance of the proposed algorithm is shown to be competitive with those of similar algorithms

a practical pr conjugate gradient method only using gradient abstract in this paper we consider the polak ribiere or polak ribiere plus conjugate gradient method for solving optimality condition of an unconstrained minimization problem we give two new steplength rules only using gradient and under gradient lipschitz assumption prove this method s global convergence correspondingly then we develop a practical polak ribiere plus method whose steplength is located by one inequality only using gradient and report promising numerical results on high accuracy solution for some standard test problems when compared to the state of art methods in this research direction importantly our work provides a new idea of devising a practical version of the celebrated polak ribiere or polak ribiere plus method

n step quadratic convergence of the mprp method with a restart strategy abstract it is well known that the prp conjugate gradient method with exact line search is globally and linearly convergent if a restart strategy is used the convergence rate of the method can be an n step superlinear quadratic convergence recently zhang et al l zhang w zhou d h li a descent modified polak ribiere polyak conjugate gradient method and its global convergence ima j numer anal 26 2006 629 640 developed a modified prp mprp method that is globally convergent if an inexact line search is used in this paper we investigate the convergence rate of the mprp method with inexact line search we first show that the mprp method with armijo line search or wolfe line search is linearly convergent we then show that the mprp method with a restart strategy still retains n step superlinear quadratic convergence if the initial steplength is appropriately chosen we also do some numerical experiments the results show that the restart mprp method does converge quadratically moreover it is more efficient than the non restart method

a new liu storey type nonlinear conjugate gradient method for unconstrained optimization problems although the liu storey ls nonlinear conjugate gradient method has a similar structure as the well known polak ribiere polyak prp and hestenes stiefel hs methods research about this method is very rare in this paper based on the memoryless bfgs quasi newton method we propose a new ls type method which converges globally for general functions with the grippo lucidi line search moreover we modify this new ls method such that the modified scheme is globally convergent for nonconvex minimization if the strong wolfe line search is used numerical results are also reported

analysis of fmri time series revisited again friston et al 1995 neuroimage 2 45 53 presented a method for detecting activations in fmr1 time series based on the general linear model and a heuristic analysis of the effective degrees of freedom in this communication we present corrected results that replace those of the previous paper and solve the same problem without recourse to heuristic arguments specifically we introduce a proper and unbiased estimator for the error terms and provide a more generally correct expression for the effective degrees of freedom the previous estimates of error variance were biased and in some instances could have led to a 10 20 overestimate of z values although the previous results are almost correct for the random regressors chosen for validation the present theoretical results are exact for any covariate or waveform we comment on some aspects of experimental design and data analysis in the light of the theoretical framework discussed here

the convergence rate of a restart mfr conjugate gradient method with inexact line search abstract in this paper we investigate the convergence rate of the modified fletcher reeves mfr method proposed by zhang et al l zhang w j zhou d h li global convergence of a modified fletcher revves conjugate gradient method with armijo type line search numer math 104 2006 561 572 under reasonable conditions we show that the mfr method with some inexact line search will be n step superlinearly and even quadratically convergent if some restart technique is used some numerical results are also reported to verify the theoretical results

group inverses of m matrices associated with nonnegative matrices having few eigenvalues abstract we continue here earlier investigations of the structure of group generalized inverses a of singular and irreducible m matrices a ri m where m is an n n nonnegative and irreducible matrix whose spectral radius is r and where m is assumed to belong to one of several special classes of nonnegative matrices we now focus on cases where m has only a few distinct eigenvalues we are particularly interested in instances of such matrices m which cause all or almost all of the off diagonal entries of the corresponding a to be nonpositive we then apply our results to the study of the sign pattern of the off diagonal entries of the group generalized inverse of a ri m where m comes from one of the following families of nonnegative matrices 1 the magic square of order n 4k generated by matlab 2 the doubly regular tournament matrices the latter type is an example of a special type of a nonnegative matrix m for which the group inverse of the associated m matrix satisfies a a t for some 0

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

convergence properties of algorithms for nonlinear optimization this paper reviews some of the most successful methods for unconstrained constrained and nondifferentiable optimization calculations particular attention is given to the contribution that theoretical analysis has made to the development of algorithms it seems that practical considerations provide the main new ideas and that subsequent theoretical studies give improvements to algorithms coherence to the subject and better understanding

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

qn like variable storage conjugate gradients both conjugate gradient and quasi newton methods are quite successful at minimizing smooth nonlinear functions of several variables and each has its advantages in particular conjugate gradient methods require much less storage to implement than a quasi newton code and therefore find application when storage limitations occur they are however slower so there have recently been attempts to combine cg and qn algorithms so as to obtain an algorithm with good convergence properties and low storage requirements one such method is the code conmin due to shanno and phua it has proven quite successful but it has one limitation it has no middle ground in that it either operates as a quasi newton code using o n 2 storage locations or as a conjugate gradient code using 7n locations but it cannot take advantage of the not unusual situation where more than 7n locations are available but a quasi newton code requires an excessive amount of storage

memory gradient method with goldstein line search in this paper we present a multi step memory gradient method with goldstein line search for unconstrained optimization problems and prove its global convergence under some mild conditions we also prove the linear convergence rate of the new method when the objective function is uniformly convex numerical results show that the new algorithm is suitable to solve large scale optimization problems and is more stable than other similar methods in practical computation

stopping criteria for linesearch methods without derivatives in this paper acceptability criteria for the linesearch stepsize are introduced which require only function values simple algorithm models based on these criteria are presented some modifications of criteria based on the knowledge of the directional derivative are also illustrated

the convergence properties of some new conjugate gradient methods abstract in this paper a new conjugate gradient formula k is given to compute the search directions for unconstrained optimization problems general convergence results for the proposed formula with some line searches such as the exact line search the wolfe powell line search and the grippo lucidi line search are discussed under the above line searches and some assumptions the global convergence properties of the given methods are discussed the given formula k 0 and has the similar form with k prp preliminary numerical results show that the proposed methods are efficient

a barzilai and borwein scaling conjugate gradient method for unconstrained optimization problems in this paper we combine the conjugate gradient method with the barzilai and borwein gradient method and propose a barzilai and borwein scaling conjugate gradient method for nonlinear unconstrained optimization problems the new method does not require to compute and store matrices associated with hessian of the objective functions and has an advantage of less computational efforts moreover the descent direction property and the global convergence are established when the line search fulfills the wolfe conditions the limited numerical experiments and comparisons show that the proposed algorithm is potentially efficient

convergence of liu storey conjugate gradient method abstract the conjugate gradient method is a useful and powerful approach for solving large scale minimization problems liu and storey developed a conjugate gradient method which has good numerical performance but no global convergence result under traditional line searches such as armijo wolfe and goldstein line searches in this paper a convergent version of liu storey conjugate gradient method ls in short is proposed for minimizing functions that have lipschitz continuous partial derivatives by estimating the lipschitz constant of the derivative of objective functions we can find an adequate step size at each iteration so as to guarantee the global convergence and improve the efficiency of ls method in practical computation

global convergence of a modified fletcher reeves conjugate gradient method with armijo type line search in this paper we are concerned with the conjugate gradient methods for solving unconstrained optimization problems it is well known that the direction generated by a conjugate gradient method may not be a descent direction of the objective function in this paper we take a little modification to the fletcher reeves fr method such that the direction generated by the modified method provides a descent direction for the objective function this property depends neither on the line search used nor on the convexity of the objective function moreover the modified method reduces to the standard fr method if line search is exact under mild conditions we prove that the modified method with armijo type line search is globally convergent even if the objective function is nonconvex we also present some numerical results to show the efficiency of the proposed method

a modified conjugate gradient algorithm with cyclic barzilai borwein steplength for unconstrained optimization for solving large scale unconstrained minimization problems the nonlinear conjugate gradient method is welcome due to its simplicity low storage efficiency and nice convergence properties among all the methods in the framework the conjugate gradient descent algorithm cg descent is very popular in which the generated directions descend automatically and this nice property is independent of any line search used in this paper we generalize cg descent with two barzilai borwein steplength reused cyclically we show that the resulting algorithm owns attractive sufficient descent property and converges globally under some mild conditions we test the proposed algorithm by using a large set of unconstrained problems with high dimensions in cuter library the numerical comparisons with the state of the art algorithm cg descent illustrate that the proposed method is effective competitive and promising

global convergence of the polak ribiere polyak conjugate gradient method with an armijo type inexact line search for nonconvex unconstrained optimization problems we propose two algorithms for nonconvex unconstrained optimization problems that employ polak ribiere polyak conjugate gradient formula and new inexact line search techniques we show that the new algorithms converge globally if the function to be minimized has lipschitz continuous gradients preliminary numerical results show that the proposed methods for particularly chosen line search conditions are very promising

on properties of special magic square matrices abstract by treating regular or associative pandiagonal and most perfect mp magic squares as matrices we find a number of interesting properties and relationships in addition we introduce a new class of quasi regular qr magic squares which includes regular and mp magic squares these four classes of magic squares are called special we prove that qr magic squares have signed pairs of eigenvalues just as do regular magic squares according to a well known theorem of mattingly this leads to the fact that odd powers of qr magic squares are magic squares which also can be established directly from the qr condition since all pandiagonal magic squares of order 4 are mp they are qr also we show that all pandiagonal magic squares of order 5 are qr but higher order ones may or may not be in addition we prove that odd powers of mp magic squares are mp a simple proof is given of the known result that natural or classic pandiagonal and regular magic squares of singly even order do not exist we consider the reflection of a regular magic square about its horizontal or vertical centerline and prove that signed pairs of eigenvalues of the reflected square differ from those of the original square by the factor i a similar result is found for mp magic squares and a subclass of qr magic squares the paper begins with mathematical definitions of the special magic squares then a number of useful matrix transformations between them are presented next following a brief summary of the spectral analysis of matrices the spectra of these special magic squares are considered and the results mentioned above are established a few numerical examples are presented to illustrate our results

two modified hs type conjugate gradient methods for unconstrained optimization problems abstract based on the modified secant equation we propose two new hs type conjugate gradient formulas their forms are similar to the original hs conjugate gradient formula and inherit all nice properties of the hs method by utilizing the technique of the three term hs method in zhang et al 2007 15 without the requirement of truncation and convexity of the objective function we show that one with wolfe line search and the other with armijo line search are globally convergent moreover under some mild conditions the linear convergence rate of the two modified methods is established the numerical results show that the proposed methods are efficient

estimating efficiency a priori a comparison of blocked and randomized designs this technical note deals with a priori estimation of efficiency of functional magnetic resonance imaging fmri designs the efficiency of an estimator is a measure of how reliable it is and depends on error variance the variance not modeled by explanatory variables in the design matrix and the design variance a function of the explanatory variables and the contrast tested changes in the experimental design can induce changes in the variance of estimated responses this translates into changes in the standard error of the response estimate or equivalently into changes in efficiency one consequence is that statistics testing for the same activation in different contexts i e experimental designs can change substantially even if the activation and error variance are exactly the same we demonstrate this effect using an event related fmri study of single word reading during blocked and randomized trial presentations furthermore we show that the error variance can change with the experimental design this highlights a problem with a priori comparison of efficiency for two or more experimental designs which usually assumes identical error variance

new properties of a nonlinear conjugate gradient method this paper provides several new properties of the nonlinear conjugate gradient method in 5 firstly the method is proved to have a certain self adjusting property that is independent of the line search and the function convexity secondly under mild assumptions on the objective function the method is shown to be globally convergent with a variety of line searches thirdly we find that instead of the negative gradient direction the search direction defined by the nonlinear conjugate gradient method in 5 can be used to restart any optimization method while guaranteeing the global convergence of the method some numerical results are also presented

globally convergent modified perry s conjugate gradient method abstract conjugate gradient methods are probably the most famous iterative methods for solving large scale optimization problems in scientific and engineering computation characterized by the simplicity of their iteration and their low memory requirements in this paper we propose a new conjugate gradient method which is based on the mbfgs secant condition by modifying perry s method our proposed method ensures sufficient descent independent of the accuracy of the line search and it is globally convergent under some assumptions numerical experiments are also presented

magic square spectra abstract to study the eigenvalues of low order singular and non singular magic squares we begin with some aspects of general square matrices additional properties follow for general semimagic squares same row and column sums with further properties for general magic squares semimagic with same diagonal sums parameterizations of general magic squares for low orders are examined including factorization of the linesum eigenvalue from the characteristic polynomial for nth order natural magic squares with matrix elements 1 n 2 we find examples of some remarkably singular cases all cases of the regular or associative or symmetric type antipodal pair sum of 1 n 2 with n 1 zero eigenvalues have been found in the only complete sets of these squares in fourth and fifth order both the jordan form and singular value decomposition svd have been useful in this study which examines examples up to 8th order in fourth order these give examples illustrating a theorem by mattingly that even order regular magic squares have a zero eigenvalue with odd algebraic multiplicity m we find 8 cases with m 3 which have a non diagonal jordan form the regular group of 48 squares is completed by 40 squares with m 1 which are diagonable a surprise finding is that the eigenvalues of 16 fourth order pandiagonal magic squares alternate between m 1 diagonable and m 3 non diagonable on rotation by 2 two 8th order natural magic squares one regular and the other pandiagonal are also examined found to have m 5 and to be diagonable mattingly also proved that odd order regular magic squares have a zero eigenvalue with even multiplicity m 0 2 4 analyzing results for natural fifth order magic squares from exact backtracking calculations we find 652 with m 2 and four with m 4 there are also 20 604 singular seventh order natural ultramagic simultaneously regular and pandiagonal squares with m 2 demonstrating that the co existence of regularity and pandiagonality permits singularity the singular odd order examples studied are all non diagonable

further studies on the wei yao liu nonlinear conjugate gradient method the wei yao liu wyl method proposed by wei et al z wei s yao l liu the convergence properties of some new conjugate gradient methods appl math comput 183 2006 1341 1350 has been proved to converge globally for uniformly convex functions with exact line search and nonconvex functions with the strong wolfe line search if the parameter email protected 0 1 4 in this paper we further study the global convergence properties of the wyl method we show that this method still converges globally for nonconvex optimization when the strong wolfe line search with the parameter s 1 4 or some armijo type line searches are used

a three parameter family of nonlinear conjugate gradient methods in this paper we propose a three parameter family of conjugate gradient methods for unconstrained optimization the three parameter family of methods not only includes the already existing six practical nonlinear conjugate gradient methods but subsumes some other families of nonlinear conjugate gradient methods as its subfamilies with powell s restart criterion the three parameter family of methods with the strong wolfe line search is shown to ensure the descent property of each search direction some general convergence results are also established for the three parameter family of methods this paper can also be regarded as a brief review on nonlinear conjugate gradient methods

globally convergent conjugate gradient algorithms the paper examines a method of implementing an angle test for determining when to restart conjugate gradient methods in a steepest descent direction the test is based on guaranteeing that the cosine of the angle between the search direction and the negative gradient is within a constant multiple of the cosine of the angle between the fletcher reeves search direction and the negative gradient this guarantees convergence for the fletcher reeves method is known to converge numerical results indicate little if anything is lost in efficiency and indicate gains may well be possible for large problems

preconditioned nonlinear conjugate gradient methods based on a modified secant equation this paper includes a twofold result for the nonlinear conjugate gradient ncg method in large scale unconstrained optimization first we consider a theoretical analysis where preconditioning is embedded in a strong convergence framework of an ncg method from the literature mild conditions to be satisfied by the preconditioners are defined in order to preserve ncg convergence as a second task we also detail the use of novel matrixfree preconditioners for ncg our proposals are based on quasinewton updates and either satisfy the secant equation or a secantlike condition at some of the previous iterates we show that in some sense the preconditioners we propose also approximate the inverse of the hessian matrix in particular the structures of our preconditioners depend on lowrank updates used along with different choices of specific parameters the lowrank updates are obtained as byproduct of ncg iterations the results of an extended numerical experience using large scale cutest problems is reported showing that our preconditioners can considerably improve the performance of ncg methods

convergence of prp method with new nonmonotone line search abstract in this paper we develop a new nonmonotone line search for prp conjugate gradient method polak ribiere polyak for minimizing functions having lipschitz continuous partial derivatives the nonmonotone line search can guarantee the global convergence of original prp method under some mild conditions numerical experiments show that prp method with the new nonmonotone line search is available and efficient in practical computation

the convergence of conjugate gradient method with nonmonotone line search abstract the conjugate gradient method is a useful and powerful approach for solving large scale minimization problems liu and storey developed a conjugate gradient method which has good numerical performance but no global convergence under traditional line searches such as armijo line search wolfe line search and goldstein line search in this paper we propose a new nonmonotone line search for liu storey conjugate gradient method ls in short the new nonmonotone line search can guarantee the global convergence of ls method and has a good numerical performance by estimating the lipschitz constant of the derivative of objective functions in the new nonmonotone line search we can find an adequate step size and substantially decrease the number of functional evaluations at each iteration numerical results show that the new approach is effective in practical computation

a new super memory gradient method with curve search rule in this paper we propose a new super memory gradient method with curve search rule for unconstrained optimization problems the method uses previous multi step iterative information and some curve search rules to generate new iterative points at each iteration this makes the new method converge stably and be more suitable for solving large scale optimization problems than other similar methods we analyze the global convergence and convergence rate under some mild conditions numerical experiments show that some new algorithms are available and effective in practical computation

some global convergence properties of the wei yao liu conjugate gradient method with inexact line search abstract in z wei s yao l liu the convergence properties of some new conjugate gradient methods applied mathematics and computation 183 2006 1341 1350 wei et al proposed a new conjugate gradient method called wyl method which has good numerical experiments and some excellent properties such as k wyl 0 in this paper we prove that while t k 1 c 2 l g k 2 d k 2 the sufficient descent condition can be satisfied at each iteration of the wyl method based on this the global convergence of the wyl method with the standard armijo line search and the generalized wolfe powell line search is established

a prp based residual method for large scale monotone nonlinear equations this paper presents a derivative free residual method for solving large scale monotone nonlinear equations which may be nonsmooth the method is constructed by replacing the gradients of the unmodified polak ribiere polyak prp nonlinear conjugate gradient method with the residuals and combining the hyperplane projection technique under suitable conditions we show that the proposed method converges globally in the sense that the whole iterative sequence converges to a solution of the problem some numerical results are reported to show its efficiency

a sufficient descent nonlinear conjugate gradient method for solving m tensor equations abstract tensor equations is a kind of important tensor optimization problems with higher order nonlinear equations which are widely used in engineering and economics this paper is concerned with solving m tensor equations we transform m tensor equations to nonlinear unconstrained optimization problems then a sufficient descent nonlinear conjugate gradient method with inexact line search is proposed for solving the transformed unconstrained optimization problem under only one mild assumption the global convergence of the proposed method is proved finally to show the effectiveness of the proposed nonlinear conjugate gradient method we compare it with three term conjugate gradient method and newton method the numerical results show that the proposed nonlinear conjugate gradient method is potentially efficient

on nonsingular regular magic squares of odd order abstract using centroskew matrices we provide a necessary and sufficient condition for a regular magic square to be nonsingular using latin squares and circulant matrices we describe a method of construction of nonsingular regular magic squares of order n where n is an odd prime power

a family of hybrid conjugate gradient methods for unconstrained optimization conjugate gradient methods are an important class of methods for unconstrained optimization especially for large scale problems recently they have been much studied this paper proposes a three parameter family of hybrid conjugate gradient methods two important features of the family are that i it can avoid the propensity of small steps namely if a small step is generated away from the solution point the next search direction will be close to the negative gradient direction and ii its descent property and global convergence are likely to be achieved provided that the line search satisfies the wolfe conditions some numerical results with the family are also presented

a new descent algorithm with curve search rule in the paper we present a new descent algorithm with curve search rule for unconstrained minimization problems at each iteration the next iterative point is determined by means of a curve search rule it is particular that the search direction and the step size is determined simultaneously at each iteration of the new algorithm the algorithm similarly to conjugate gradient methods avoids the computation and storage of some matrices associated with the hessian of objective functions unlike ode methods it is not required to solve ordinary differential equations at each iteration although the convergence rate of our algorithm is not as fast as that of newton like methods it is suitable to solve large scale minimization problems

convergence of descent method without line search line search method is sometimes a useful and efficient technique for solving unconstrained optimization problems especially for solving small and middle scale problems however a line search procedure is necessary at each iteration leading to a significant amount of computation in order to reduce the evaluations of objective functions and gradients the line search procedure should be avoided in algorithm design in this paper we propose a new descent method without line search for unconstrained optimization problems the algorithm is required to estimate some parameters at each iteration we analyze theoretically the global convergence of the new algorithm under mild conditions these theoretical conclusions can help us in designing new efficient methods for optimization problems

a new class of spectral conjugate gradient methods based on a modified secant equation for unconstrained optimization conjugate gradient methods have played a special role for solving large scale optimization problems due to the simplicity of their iteration convergence properties and their low memory requirements in this work we propose a new class of spectral conjugate gradient methods which ensures sufficient descent independent of the accuracy of the line search moreover an attractive property of our proposed methods is that they achieve a high order accuracy in approximating the second order curvature information of the objective function by utilizing the modified secant condition proposed by babaie kafaki et al s babaie kafaki r ghanbari n mahdavi amiri two new conjugate gradient methods based on modified secant equations journal of computational and applied mathematics 234 2010 1374 1386 further a global convergence result for general functions is established provided that the line search satisfies the wolfe conditions our numerical experiments indicate that our proposed methods are preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness

a new family of conjugate gradient methods in this paper we develop a new class of conjugate gradient methods for unconstrained optimization problems a new nonmonotone line search technique is proposed to guarantee the global convergence of these conjugate gradient methods under some mild conditions in particular polak ribiere polyak and liu storey conjugate gradient methods are special cases of the new class of conjugate gradient methods by estimating the local lipschitz constant of the derivative of objective functions we can find an adequate step size and substantially decrease the function evaluations at each iteration numerical results show that these new conjugate gradient methods are effective in minimizing large scale non convex non quadratic functions

a new globalization technique for nonlinear conjugate gradient methods for nonconvex minimization abstract it is well known that the hs method and the prp method may not converge for nonconvex optimization even with exact line search some globalization techniques have been proposed for instance the prp globalization technique and the grippo lucidi globalization technique for the prp method in this paper we propose a new efficient globalization technique for general nonlinear conjugate gradient methods for nonconvex minimization this new technique utilizes the information of the previous search direction sufficiently under suitable conditions we prove that the nonlinear conjugate gradient methods with this new technique are globally convergent for nonconvex minimization if the line search satisfies wolfe conditions or armijo condition extensive numerical experiments are reported to show the efficiency of the proposed technique