benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

a limited memory bfgs type method for large scale unconstrained optimization in this paper a new numerical method for solving large scale unconstrained optimization problems is presented it is derived from a modified bfgs type update formula by wei li and qi it is observed that the update formula can be extended to the framework of limited memory scheme with hardly more storage or arithmetic operations under some suitable conditions the global convergence property is established the implementations of the method on a set of cute problems indicate that this extension is beneficial to the performance of the algorithm

a new class of nonmonotone conjugate gradient training algorithms in this paper we propose a new class of conjugate gradient algorithms for training neural networks which is based on a new modified nonmonotone scheme proposed by shi and wang 2011 the utilization of a nonmonotone strategy enables the training algorithm to overcome the case where the sequence of iterates runs into the bottom of a curved narrow valley a common occurrence in neural network training process our proposed class of methods ensures sufficient descent avoiding thereby the usual inefficient restarts and it is globally convergent under mild conditions our experimental results provide evidence that the proposed nonmonotone conjugate gradient training methods are efficient outperforming classical methods proving more stable efficient and reliable learning

a globally convergent version of the polak ribiere conjugate gradient method in this paper we propose a new line search algorithm that ensures global convergence of the polak ribiere conjugate gradient method for the unconstrained minimization of nonconvex differentiable functions in particular we show that with this line search every limit point produced by the polak ribiere iteration is a stationary point of the objective function moreover we define adaptive rules for the choice of the parameters in a way that the first stationary point along a search direction can be eventually accepted when the algorithm is converging to a minimum point with positive definite hessian matrix under strong convexity assumptions the known global convergence results can be reobtained as a special case from a computational point of view we may expect that an algorithm incorporating the step size acceptance rules proposed here will retain the same good features of the polak ribiere method while avoiding pathological situations

a nonlinear conjugate gradient method with a strong global convergence property conjugate gradient methods are widely used for unconstrained optimization especially large scale problems the strong wolfe conditions are usually used in the analyses and implementations of conjugate gradient methods this paper presents a new version of the conjugate gradient method which converges globally provided the line search satisfies the standard wolfe conditions the conditions on the objective function are also weak being similar to those required by the zoutendijk condition

projected gradient methods for nonnegative matrix factorization nonnegative matrix factorization nmf can be formulated as a minimization problem with bound constraints although bound constrained optimization has been studied extensively in both theory and practice so far no study has formally applied its techniques to nmf in this letter we propose two projected gradient methods for nmf both of which exhibit strong optimization properties we discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach a simple matlab code is also provided

an extension of the fletcher reeves method to linear equality constrained optimization problem in this paper combining the feasible direction method with the conjugate gradient method we propose a feasible fletcher reeves conjugate gradient method for solving linear equality constrained optimization problem the directions generated by the method are feasible and descent for the objective function under mild conditions we show that the proposed method with exact line search is globally convergent numerical experiments are also given to show the efficiency of the method

a new conjugate gradient method with guaranteed descent and an efficient line search a new nonlinear conjugate gradient method and an associated implementation based on an inexact line search are proposed and analyzed with exact line search our method reduces to a nonlinear version of the hestenes stiefel conjugate gradient scheme for any inexact line search our scheme satisfies the descent condition bf g k sf t bf d k le frac 7 8 bf g k 2 moreover a global convergence result is established when the line search fulfills the wolfe conditions a new line search scheme is developed that is efficient and highly accurate efficiency is achieved by exploiting properties of linear interpolants in a neighborhood of a local minimizer high accuracy is achieved by using a convergence criterion which we call the approximate wolfe conditions obtained by replacing the sufficient decrease criterion in the wolfe conditions with an approximation that can be evaluated with greater precision in a neighborhood of a local minimum than the usual sufficient decrease criterion numerical comparisons are given with both l bfgs and conjugate gradient methods using the unconstrained optimization problems in the cute library

a spectral conjugate gradient method for solving large scale unconstrained optimization abstract this paper establishes a spectral conjugate gradient method for solving unconstrained optimization problems where the conjugate parameter and the spectral parameter satisfy a restrictive relationship the search direction is sufficient descent without restarts in per iteration moreover this feature is independent of any line searches under the standard wolfe line searches the global convergence of the proposed method is proved when k k f r holds the preliminary numerical results are presented to show effectiveness of the proposed method

the barzilai and borwein gradient method for the large scale unconstrained minimization problem the barzilai and borwein gradient method for the solution of large scale unconstrained minimization problems is considered this method requires few storage locations and very inexpensive computations furthermore it does not guarantee descent in the objective function and no line search is required recently the global convergence for the convex quadratic case has been established however for the nonquadratic case the method needs to be incorporated in a globalization scheme in this work a nonmonotone line search strategy that guarantees global convergence is combined with the barzilai and borwein method this strategy is based on the nonmonotone line search technique proposed by grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 numerical results to compare the behavior of this method with recent implementations of the conjugate gradient method are presented these results indicate that the global barzilai and borwein method may allow some significant reduction in the number of line searches and also in the number of gradient evaluations

spectral modified polak ribiere polyak projection conjugate gradient method for solving monotone systems of nonlinear equations abstract in this paper we present a modification of polak ribi e re polyak prp conjugate gradient method for solving system of monotone nonlinear equations which is a combination of spectral conjugate gradient method and the hyperplane projection technique the method is based on two methods for unconstrained optimization proposed by wan et al 2011 and sun 2015 we obtained a new search direction by the use of a different formula for the conjugate gradient parameter the search direction satisfies the sufficient descent condition and the global convergence of the method is established under some assumptions preliminary numerical comparison with some existing methods shows the efficiency of the proposed method

n step quadratic convergence of the mprp method with a restart strategy abstract it is well known that the prp conjugate gradient method with exact line search is globally and linearly convergent if a restart strategy is used the convergence rate of the method can be an n step superlinear quadratic convergence recently zhang et al l zhang w zhou d h li a descent modified polak ribiere polyak conjugate gradient method and its global convergence ima j numer anal 26 2006 629 640 developed a modified prp mprp method that is globally convergent if an inexact line search is used in this paper we investigate the convergence rate of the mprp method with inexact line search we first show that the mprp method with armijo line search or wolfe line search is linearly convergent we then show that the mprp method with a restart strategy still retains n step superlinear quadratic convergence if the initial steplength is appropriately chosen we also do some numerical experiments the results show that the restart mprp method does converge quadratically moreover it is more efficient than the non restart method

p adic distribution of the unification of the bernoulli euler and genocchi polynomials abstract the aim of this paper is to construct p adic distribution on x c p of the unification of the bernoulli euler and genocchi polynomials y n x k a b which is given by n k a b j dp n z p a b dp n p dp n n k a jb y n dp n j dp n k a dp n b where y n x k a b are defined by 1 1 we give some applications related to these functions and distribution

some q extensions of the apostol bernoulli and the apostol euler polynomials of order n and the multiple hurwitz zeta function abstract in this paper we first investigate several further interesting properties of the multiple hurwitz lerch zeta function n z s a which was introduced recently by choi et al j choi d s jang h m srivastava a generalization of the hurwitz lerch zeta function integral transform spec funct 19 2008 we then introduce and investigate some q extensions of the multiple hurwitz lerch zeta function n z s a the apostol bernoulli polynomials b k n x of order n and the apostol euler polynomials e k n x of order n relevant connections of the results presented here with those obtained in earlier works are also indicated precisely

the convergence rate of a restart mfr conjugate gradient method with inexact line search abstract in this paper we investigate the convergence rate of the modified fletcher reeves mfr method proposed by zhang et al l zhang w j zhou d h li global convergence of a modified fletcher revves conjugate gradient method with armijo type line search numer math 104 2006 561 572 under reasonable conditions we show that the mfr method with some inexact line search will be n step superlinearly and even quadratically convergent if some restart technique is used some numerical results are also reported to verify the theoretical results

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

convergence properties of algorithms for nonlinear optimization this paper reviews some of the most successful methods for unconstrained constrained and nondifferentiable optimization calculations particular attention is given to the contribution that theoretical analysis has made to the development of algorithms it seems that practical considerations provide the main new ideas and that subsequent theoretical studies give improvements to algorithms coherence to the subject and better understanding

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

a family of p adic twisted interpolation functions associated with the modified bernoulli numbers the main purpose of this paper is to construct a family of modified p adic twisted functions which interpolate the modified twisted q bernoulli polynomials and the generalized twisted q bernoulli numbers at negative integers we also give some applications and examples related to these functions and numbers

a barzilai and borwein scaling conjugate gradient method for unconstrained optimization problems in this paper we combine the conjugate gradient method with the barzilai and borwein gradient method and propose a barzilai and borwein scaling conjugate gradient method for nonlinear unconstrained optimization problems the new method does not require to compute and store matrices associated with hessian of the objective functions and has an advantage of less computational efforts moreover the descent direction property and the global convergence are established when the line search fulfills the wolfe conditions the limited numerical experiments and comparisons show that the proposed algorithm is potentially efficient

two descent hybrid conjugate gradient methods for optimization in this paper we propose two new hybrid nonlinear conjugate gradient methods which produce sufficient descent search direction at every iteration this property depends neither on the line search used nor on the convexity of the objective function under suitable conditions we prove that the proposed methods converge globally for general nonconvex functions the numerical results show that both hybrid methods are efficient for the given test problems from the cute library

global convergence of a modified fletcher reeves conjugate gradient method with armijo type line search in this paper we are concerned with the conjugate gradient methods for solving unconstrained optimization problems it is well known that the direction generated by a conjugate gradient method may not be a descent direction of the objective function in this paper we take a little modification to the fletcher reeves fr method such that the direction generated by the modified method provides a descent direction for the objective function this property depends neither on the line search used nor on the convexity of the objective function moreover the modified method reduces to the standard fr method if line search is exact under mild conditions we prove that the modified method with armijo type line search is globally convergent even if the objective function is nonconvex we also present some numerical results to show the efficiency of the proposed method

application of singular spectrum analysis to the smoothing of raw kinematic signals motion capture systems currently used in biomechanical analysis introduce systematic measurement errors that appear in the form of noise in recorded displacement signals the noise is unacceptably amplified when differentiating displacements to obtain velocities and accelerations to avoid this phenomenon it is necessary to smooth the displacement signal prior to differentiation in order to eliminate the noise introduced by the experimental system the use of singular spectrum analysis ssa is presented in this paper as an alternative to traditional digital filtering methods ssa is a novel non parametric technique based on principles of multivariate statistics the original time series is decomposed into a number of additive time series each of which can be easily identified as being part of the modulated signal or as being part of the random noise several examples that demonstrate the superiority of this technique over other methods used in biomechanical analysis are presented in this paper

new properties of a nonlinear conjugate gradient method this paper provides several new properties of the nonlinear conjugate gradient method in 5 firstly the method is proved to have a certain self adjusting property that is independent of the line search and the function convexity secondly under mild assumptions on the objective function the method is shown to be globally convergent with a variety of line searches thirdly we find that instead of the negative gradient direction the search direction defined by the nonlinear conjugate gradient method in 5 can be used to restart any optimization method while guaranteeing the global convergence of the method some numerical results are also presented

globally convergent modified perry s conjugate gradient method abstract conjugate gradient methods are probably the most famous iterative methods for solving large scale optimization problems in scientific and engineering computation characterized by the simplicity of their iteration and their low memory requirements in this paper we propose a new conjugate gradient method which is based on the mbfgs secant condition by modifying perry s method our proposed method ensures sufficient descent independent of the accuracy of the line search and it is globally convergent under some assumptions numerical experiments are also presented

three term conjugate gradient method for the convex optimization problem over the fixed point set of a nonexpansive mapping many constrained sets in problems such as signal processing and optimal control can be represented as a fixed point set of a certain nonexpansive mapping and a number of iterative algorithms have been presented for solving a convex optimization problem over a fixed point set this paper presents a novel gradient method with a three term conjugate gradient direction that is used to accelerate conjugate gradient methods for solving unconstrained optimization problems it is guaranteed that the algorithm strongly converges to the solution to the problem under the standard assumptions numerical comparisons with the existing gradient methods demonstrate the effectiveness and fast convergence of this algorithm

carlitz s q bernoulli and q euler numbers and polynomials and a class of generalized q hurwitz zeta functions in this paper we systematically recover the identities for the q eta numbers h k and the q eta polynomials h k x presented by carlitz l carlitz q bernoulli numbers and polynomials duke math j 15 1948 987 1000 which we define here via generating series rather than via the difference equations of carlitz following a method developed by kaneko et al m kaneko n kurokawa m wakayama a variation of euler s approach to the riemann zeta function kyushu j math 57 2003 175 192 for a canonical q extension of the riemann zeta function we investigate a similarly constructed q extension of the hurwitz zeta function the details of this investigation disclose some interesting connections among q eta polynomials carlitz s q bernoulli polynomials b k x e polynomials and the q bernoulli polynomials that emerge from the q extension of the hurwitz zeta function discussed here

a norm descent derivative free algorithm for solving large scale nonlinear symmetric equations abstract in this paper we propose a norm descent derivative free algorithm for solving large scale nonlinear symmetric equations without involving any information of the gradient or jacobian matrix by using some approximate substitutions the proposed algorithm is extended from an efficient three term conjugate gradient method for solving unconstrained optimization problems and inherits some nice properties such as simple structure low storage requirements and symmetric property under some appropriate conditions the global convergence is proved finally the numerical experiments and comparisons show that the proposed algorithm is very effective for large scale problems

some generalizations of the apostol genocchi polynomials and the stirling numbers of the second kind abstract recently the authors introduced some generalizations of the apostol bernoulli polynomials and the apostol euler polynomials see q m luo h m srivastava j math anal appl 308 2005 290 302 and q m luo taiwanese j math 10 2006 917 925 the main object of this paper is to investigate an analogous generalization of the genocchi polynomials of higher order that is the so called apostol genocchi polynomials of higher order for these generalized apostol genocchi polynomials we establish several elementary properties provide some explicit relationships with the apostol bernoulli polynomials and the apostol euler polynomials and derive various explicit series representations in terms of the gaussian hypergeometric function and the hurwitz or generalized zeta function we also deduce their special cases and applications which are shown here to lead to the corresponding results for the genocchi and euler polynomials of higher order by introducing an analogue of the stirling numbers of the second kind that is the so called stirling numbers of the second kind we derive some basic properties and formulas and consider some interesting applications to the family of the apostol type polynomials furthermore we also correct an error in a previous paper q m luo h m srivastava comput math appl 51 2006 631 642 and pose two open problems on the subject of our investigation

a unified presentation of the generating functions of the generalized bernoulli euler and genocchi polynomials the goal of this paper is to unify and extend the generating functions of the generalized bernoulli polynomials the generalized euler polynomials and the generalized genocchi polynomials associated with the positive real parameters a and b and the complex parameter b by using this generating function we derive recurrence relations and other properties for these polynomials by applying the mellin transformation to the generating function of the unification of bernoulli euler and genocchi polynomials we construct a unification of the zeta functions furthermore we give many properties and applications involving the functions and polynomials investigated in this paper

a class of modified fr conjugate gradient method and applications to non negative matrix factorization conjugate gradient method as an efficient method is used to solve unconstrained optimization problems in this paper we propose a class of modified fletcherreeves conjugate gradient method with armijo type line search which generates the direction is descent for the objective function under mild conditions we give that the proposed method is descent and globally convergent moreover the method is applied to nonnegative matrix factorization and the experimental results demonstrate the validity of the proposed method

improved accuracy of biomechanical motion data obtained during impacts using a time frequency low pass filter biomechanical motion data involving impacts are not adequately represented using 31 conventional low pass filters cf time frequency filters tff are a viable alternative but 32 have been largely overlooked by movement scientists we modified georgakis and 33 subramaniam s 2009 fractional fourier filter mfrff and demonstrated it performed better 34 than cfs for obtaining lower leg accelerations during football instep kicking the mfrff 35 displayed peak marker accelerations comparable to a reference accelerometer during foot to 36 ball impact peak error 5 0 11 4 whereas cfs severely underestimated these peaks 37 30 70 error during the non impact phases the mfrff performed comparably to cfs 38 using an appropriate 12 20hz cut off frequency rmse 37 3 7 6 m s2 vs 42 1 11 4 39 m s2 respectively since accuracy of segmental kinematics is fundamental for understanding 40 human movement the mfrff should be applied to a range of biomechanical impact scenarios 41 e g locomotion landing and striking motions to enhance the efficacy of study in these areas

a sufficient descent nonlinear conjugate gradient method for solving m tensor equations abstract tensor equations is a kind of important tensor optimization problems with higher order nonlinear equations which are widely used in engineering and economics this paper is concerned with solving m tensor equations we transform m tensor equations to nonlinear unconstrained optimization problems then a sufficient descent nonlinear conjugate gradient method with inexact line search is proposed for solving the transformed unconstrained optimization problem under only one mild assumption the global convergence of the proposed method is proved finally to show the effectiveness of the proposed nonlinear conjugate gradient method we compare it with three term conjugate gradient method and newton method the numerical results show that the proposed nonlinear conjugate gradient method is potentially efficient

an improved wei yao liu nonlinear conjugate gradient method for optimization computation in this paper we take a little modification to the wei yao liu nonlinear conjugate gradient method proposed by wei et al z wei s yao l liu the convergence properties of some new conjugate gradient methods appl math comput 183 2006 1341 1350 such that the modified method possesses better convergence properties in fact we prove that the modified method satisfies sufficient descent condition with greater parameter email protected 0 12 in the strong wolfe line search and converges globally for nonconvex minimization we also extend these results to the hestenes stiefel method and prove that the modified hs method is globally convergent for nonconvex functions with the standard wolfe conditions numerical results are reported by using some test problems in the cute library

new version of the three term conjugate gradient method based on spectral scaling conjugacy condition that generates descent search direction in this paper a general form of three term conjugate gradient method is presented in which the search directions simultaneously satisfy the dai liao conjugacy condition and sufficient descent property in addition the choice for an optimal parameter is suggested in the sense that the condition number of the iteration matrix could arrives at its minimum which can be regarded as the inheritance and development of the spectral scaling quasi newton equation different from the existent methods a new update strategy in constructing the search direction is proposed to establish the global convergence for the general function numerical results show our algorithm is practical and effective for the test problems

a nonlinear conjugate gradient algorithm with an optimal property and an improved wolfe line search in this paper we seek the conjugate gradient direction closest to the direction of the scaled memoryless bfgs method and propose a family of conjugate gradient methods for unconstrained optimization an improved wolfe line search is also proposed which can avoid a numerical drawback of the original wolfe line search and guarantee the global convergence of the conjugate gradient method under mild conditions to accelerate the algorithm we introduce adaptive restarts along negative gradients based on the extent to which the function approximates some quadratic function during previous iterations numerical experiments with the cuter collection show that the proposed algorithm is promising

a new class of spectral conjugate gradient methods based on a modified secant equation for unconstrained optimization conjugate gradient methods have played a special role for solving large scale optimization problems due to the simplicity of their iteration convergence properties and their low memory requirements in this work we propose a new class of spectral conjugate gradient methods which ensures sufficient descent independent of the accuracy of the line search moreover an attractive property of our proposed methods is that they achieve a high order accuracy in approximating the second order curvature information of the objective function by utilizing the modified secant condition proposed by babaie kafaki et al s babaie kafaki r ghanbari n mahdavi amiri two new conjugate gradient methods based on modified secant equations journal of computational and applied mathematics 234 2010 1374 1386 further a global convergence result for general functions is established provided that the line search satisfies the wolfe conditions our numerical experiments indicate that our proposed methods are preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness

a variant spectral type fr conjugate gradient method and its global convergence abstract in this paper we propose a variant fr vfr formula k vfr and corresponding spectral type conjugate gradient method svfr such that the direction generated is always a descent direction for the objective function we also extend k vfr to k such that k k fr and have similar conclusions under appropriate conditions we prove that the proposed method is globally convergent under not only wolfe line search but also armijo type line search numerical experiments show the svfr method performs well

time frequency analysis and filtering of kinematic signals with impacts using the wigner function accurate estimation of the second derivative biomechanical signals are represented in the time frequency domain using the wigner distribution function filtering of this representation for the case of a non stationary displacement signal with impact is studied smoothed displacement data are then double differentiated and compared with references accelerometer data it is shown that this technique is able to remove noise from these signals in a better way than conventional filtering techniques currently used in biomechanics

a new globalization technique for nonlinear conjugate gradient methods for nonconvex minimization abstract it is well known that the hs method and the prp method may not converge for nonconvex optimization even with exact line search some globalization techniques have been proposed for instance the prp globalization technique and the grippo lucidi globalization technique for the prp method in this paper we propose a new efficient globalization technique for general nonlinear conjugate gradient methods for nonconvex minimization this new technique utilizes the information of the previous search direction sufficiently under suitable conditions we prove that the nonlinear conjugate gradient methods with this new technique are globally convergent for nonconvex minimization if the line search satisfies wolfe conditions or armijo condition extensive numerical experiments are reported to show the efficiency of the proposed technique