a new class of nonmonotone conjugate gradient training algorithms in this paper we propose a new class of conjugate gradient algorithms for training neural networks which is based on a new modified nonmonotone scheme proposed by shi and wang 2011 the utilization of a nonmonotone strategy enables the training algorithm to overcome the case where the sequence of iterates runs into the bottom of a curved narrow valley a common occurrence in neural network training process our proposed class of methods ensures sufficient descent avoiding thereby the usual inefficient restarts and it is globally convergent under mild conditions our experimental results provide evidence that the proposed nonmonotone conjugate gradient training methods are efficient outperforming classical methods proving more stable efficient and reliable learning

non monotone trust region algorithms for nonlinear optimization subject to convex constraints this paper presents two new trust region methods for solving nonlinear optimization problems over convex feasible domains these methods are distinguished by the fact that they do not enforce strict monotonicity of the objective function values at successive iterates the algorithms are proved to be convergent to critical points of the problem from any starting point extensive numerical experiments show that this approach is competitive with the lancelot package

the proof of the sufficient descent condition of the wei yao liu conjugate gradient method under the strong wolfe powell line search abstract in z wei et al the convergence properties of some conjugate gradient methods appl math comput 183 2006 1341 1350 wei et al proposed a new conjugate gradient method called wyl method which has good numerical experiments under the sufficient descent condition and wwp line search the given method s convergent results have been established by wei et al so the sufficient descent condition is important for the wyl method in this paper we prove that by restricting the parameter 1 4 under the swp line search condition the wyl method possess the sufficient descent condition which deduces the global convergent results of the wyl method under swp line search

convergence of the polak ribiere polyak conjugate gradient method abstract in this paper we consider the global convergence of the polak ribiere polyak abbreviated prp conjugate gradient method for unconstrained optimization problems a new armijo type line search is proposed for the original prp method and some convergence properties are given under some mild conditions the new armijo type line search can make the prp method choose a suitable initial step size so as to decrease the function evaluations at each iteration and improve the performance of the prp method numerical results show that the prp method with the new armijo type line search is more efficient than other similar methods in practical computation

a modified conjugate gradient algorithm with cyclic barzilai borwein steplength for unconstrained optimization for solving large scale unconstrained minimization problems the nonlinear conjugate gradient method is welcome due to its simplicity low storage efficiency and nice convergence properties among all the methods in the framework the conjugate gradient descent algorithm cg descent is very popular in which the generated directions descend automatically and this nice property is independent of any line search used in this paper we generalize cg descent with two barzilai borwein steplength reused cyclically we show that the resulting algorithm owns attractive sufficient descent property and converges globally under some mild conditions we test the proposed algorithm by using a large set of unconstrained problems with high dimensions in cuter library the numerical comparisons with the state of the art algorithm cg descent illustrate that the proposed method is effective competitive and promising

on the asymptotic behaviour of some new gradient methods the barzilai borwein bb gradient method and some other new gradient methods have shown themselves to be competitive with conjugate gradient methods for solving large dimension nonlinear unconstrained optimization problems little is known about the asymptotic behaviour even when applied to n dimensional quadratic functions except in the case that n 2 we show in the quadratic case how it is possible to compute this asymptotic behaviour and observe that as n increases there is a transition from superlinear to linear convergence at some value of n 4 depending on the method by neglecting certain terms in the recurrence relations we define simplified versions of the methods which are able to predict this transition the simplified methods also predict that for larger values of n the eigencomponents of the gradient vectors converge in modulus to a common value which is a similar to a property observed to hold in the real methods some unusual and interesting recurrence relations are analysed in the course of the study

a hypersingular boundary integral method for two dimensional screen and crack problems we analyze hypersingular integral equations on a curved open smooth arc in r 2 that model either curved cracks in an elastic medium or the scattering of acoustic and elastic waves at a hard screen by using the mellin transformation we obtain sharp regularity results for the solution of these equations in sobolev spaces in the form of singular expansions in particular we show that the expansions do not contain logarithmic singularities

testing for structural breaks in factor copula models abstract we propose new fluctuation tests for detecting structural breaks in factor copula models and analyse the behaviour under the null hypothesis of no change in the model the joint copula is given by the copula of random variables which arise from a factor model this is particularly useful for analysing data with high dimensions parameters are estimated with the simulated method of moments smm the discontinuity of the smm objective function complicates the derivation of a functional limit theorem for the parameters we analyse the behaviour of the tests in monte carlo simulations and a real data application it turns out that our test is more powerful than nonparametric tests for copula constancy in high dimensions

wave propagation in bifurcated waveguides of square lattice strips the propagation of waves in square lattice strips as waveguides with discrete analogues of hard and soft outer boundaries is considered in the presence of a bifurcating lattice row which is either soft or hard by an application of the discrete fourier transform the related boundary value problem for both types of the bifurcated waveguides is formulated as a discrete wiener hopf equation the exact solution of the latter is found in the form of a contour integral on the unit circle in complex plane closed form expressions for the approximate solution in the far field are provided using the normal mode expansion a general expression for the reflectance and transmittance is obtained the sum in the absence of dissipation is established as unity it is shown that the limiting wave field as the waveguide width tends to infinity coincides with that on an infinite lattice the proof is presented in the dissipative case the discrete paradigm of the bifurcated waveguides introduced in the paper has natural applications in engineering and science

convergence of prp method with new nonmonotone line search abstract in this paper we develop a new nonmonotone line search for prp conjugate gradient method polak ribiere polyak for minimizing functions having lipschitz continuous partial derivatives the nonmonotone line search can guarantee the global convergence of original prp method under some mild conditions numerical experiments show that prp method with the new nonmonotone line search is available and efficient in practical computation

the convergence of conjugate gradient method with nonmonotone line search abstract the conjugate gradient method is a useful and powerful approach for solving large scale minimization problems liu and storey developed a conjugate gradient method which has good numerical performance but no global convergence under traditional line searches such as armijo line search wolfe line search and goldstein line search in this paper we propose a new nonmonotone line search for liu storey conjugate gradient method ls in short the new nonmonotone line search can guarantee the global convergence of ls method and has a good numerical performance by estimating the lipschitz constant of the derivative of objective functions in the new nonmonotone line search we can find an adequate step size and substantially decrease the number of functional evaluations at each iteration numerical results show that the new approach is effective in practical computation

simulation and the asymptotics of optimization estimators on demontre un theoreme de limite centrale general pour des estimateurs definis par minimisation de la longueur d une fonction de critere aleatoire a valeurs vectorielles aucune hypothese de regularite suffisante n est imposee sur la fonction de critere pour que les resultats puissent etre appliques a une classe d estimlateurs de simulation assez large des analyses completes de deux estimateurs de simulation l un introduit par pakes l autre par mcfadden illustrent l application des theoremes generaux

frank s family of bivariate distributions summary this paper examines the properties of a new class of bivariate distributions whose members are stochastically ordered and likelihood ratio dependent the proposed class can be used to construct bivariate families of distributions whose marginals are arbitrary and which include the frechet bounds as well as the distribution corresponding to independent variables three nonparametric estimators of the association parameter are suggested and monte carlo experiments are used to compare their small sample behaviour to that of the maximum likelihood estimate

on the fundamental solutions for the difference helmholtz operator in this paper we construct the difference approximation of the helmholtz operator on a square grid and we analyze asymptotic properties of the difference operator fundamental solutions we show that for a sommerfeld type asymptotic condition there exists a fundamental solution that is determined in a unique way

efficient estimation of semiparametric multivariate copula models we propose a sieve maximum likelihood estimation procedure for a broad class of semiparametric multivariate distributions a joint distribution in this class is characterized by a parametric copula function evaluated at nonparametric marginal distributions this class of distributions has gained popularity in diverse fields due to its flexibility in separately modeling the dependence structure and the marginal behaviors of a multivariate random variable and its circumvention of the curse of dimensionality associated with purely nonparametric multivariate distributions we show that the plug in sieve maximum likelihood estimators mles of all smooth functionals including the finite dimensional copula parameters and the unknown marginal distributions are semiparametrically efficient and that their asymptotic variances can be estimated consistently moreover prior restrictions on the marginal distributions can be easily incorporated into the sieve maximum likelihood estimation procedure to achieve fu

a new globalization technique for nonlinear conjugate gradient methods for nonconvex minimization abstract it is well known that the hs method and the prp method may not converge for nonconvex optimization even with exact line search some globalization techniques have been proposed for instance the prp globalization technique and the grippo lucidi globalization technique for the prp method in this paper we propose a new efficient globalization technique for general nonlinear conjugate gradient methods for nonconvex minimization this new technique utilizes the information of the previous search direction sufficiently under suitable conditions we prove that the nonlinear conjugate gradient methods with this new technique are globally convergent for nonconvex minimization if the line search satisfies wolfe conditions or armijo condition extensive numerical experiments are reported to show the efficiency of the proposed technique

global convergence of nonmonotone descent methods for unconstrained optimization problems global convergence results are established for unconstrained optimization algorithms that utilize a nonmonotone line search procedure this procedure allows the user to specify a flexible forcing function and includes the nonmonotone armijo rule the nonmonotone goldstein rule and the nonmonotone wolfe rule as special cases

a semiparametric estimation procedure of dependence parameters in multivariate families of distributions summary this paper investigates the properties of a semiparametric method for estimating the dependence parameters in a family of multivariate distributions the proposed estimator obtained as a solution of a pseudo likelihood equation is shown to be consistent asymptotically normal and fully efficient at independence a natural estimator of its asymptotic variance is proved to be consistent comparisons are made with alternative semiparametric estimators in the special case of clayton s model for association in bivariate data

near tip field for diffraction on square lattice by crack the displacement field near a tip of a finite crack due to the diffraction of a wave on a square lattice is studied the finite section method in the theory of toeplitz operators on ell 2 is invoked as the semi infinite crack diffraction problem is shown equivalent to the inversion of a toeplitz operator a truncation of which appears in the finite crack diffraction problem for the same incident wave the existence and uniqueness of the solution in ell 2 for the semi infinite crack problem is established by an application of the well known krein conditions continuum limit of the semi infinite crack diffraction problem is established in a discrete sobolev space a graphical illustration of convergence in the relevant sobolev norm is also included a low frequency asymptotic approximation of the normalized shear force in vertical bonds ahead of the crack tip recovers the classical crack tip singularity displacement of particles in the vicinity of the crack tip a closed form expression of wh

copula based time series with filtered nonstationarity abstract economic and financial time series data can exhibit nonstationary and nonlinear patterns simultaneously this paper studies copula based time series models that capture both patterns we introduce a procedure where nonstationarity is removed via a filtration and then the nonlinear temporal dependence in the filtered data is captured via a flexible markov copula we propose two estimators of the copula dependence parameters the parametric two step copula estimator where the marginal distribution of the filtered series is estimated parametrically and the semiparametric two step copula estimator where the marginal distribution is estimated via a rescaled empirical distribution of the filtered series we show that the limiting distribution of the parametric copula estimator depends on the nonstationary filtration and the parametric marginal distribution estimation and may be non normal surprisingly the limiting distribution of the semiparametric copula estimator using the filtered data is shown to be the same as that without nonstationary filtration which is normal and free of marginal distribution specification the simple and robust properties of the semiparametric copula estimators extend to models with misspecified copulas and facilitate statistical inferences such as hypothesis testing and model selection tests on semiparametric copula based dynamic models in the presence of nonstationarity monte carlo studies and real data applications are presented

the global convergence of the polak ribiere polyak conjugate gradient algorithm under inexact line search for nonconvex functions abstract powell 1984 and dai 2003 constructed respectively a counterexample to show that the polak ribiere polyak prp conjugate gradient algorithm fails to globally converge for nonconvex functions even when the exact line search technique is used which implies similar failure of the weak wolfe powell wwp inexact line search technique does another inexact line search technique exist that can ensure global convergence for nonconvex functions this paper gives a positive answer to this question and proposes a modified wwp mwwp inexact line search technique an algorithm is presented that uses the mwwp inexact line search technique the next point x k 1 generated by the prp formula is accepted if a positive condition holds and otherwise x k 1 is defined by a technique for projection onto a parabolic surface the proposed prp conjugate gradient algorithm is shown to possess global convergence under inexact line search for nonconvex functions numerical performance of the proposed algorithm is shown to be competitive with those of similar algorithms

from molecular models to continuum mechanics we present here a limiting process allowing us to write some continuum mechanics models as a natural asymptotic of molecular models the approach is based on the hypothesis that the macroscopic displacement is equal to the microscopic one we carry out the corresponding calculations in the case of two body energies including higher order terms and also in the case of thomas fermi type models

a new class of nonlinear conjugate gradient coefficients with global convergence properties abstract nonlinear conjugate gradient cg methods have played an important role in solving large scale unconstrained optimization their wide application in many fields is due to their low memory requirements and global convergence properties numerous studies and modifications have been conducted recently to improve this method in this paper a new class of conjugate gradient coefficients k that possess global convergence properties is presented the global convergence result is established using exact line searches numerical result shows that the proposed formula is superior and more efficient when compared to other cg coefficients

convergence properties of algorithms for nonlinear optimization this paper reviews some of the most successful methods for unconstrained constrained and nondifferentiable optimization calculations particular attention is given to the contribution that theoretical analysis has made to the development of algorithms it seems that practical considerations provide the main new ideas and that subsequent theoretical studies give improvements to algorithms coherence to the subject and better understanding

the convergence properties of some new conjugate gradient methods abstract in this paper a new conjugate gradient formula k is given to compute the search directions for unconstrained optimization problems general convergence results for the proposed formula with some line searches such as the exact line search the wolfe powell line search and the grippo lucidi line search are discussed under the above line searches and some assumptions the global convergence properties of the given methods are discussed the given formula k 0 and has the similar form with k prp preliminary numerical results show that the proposed methods are efficient

stopping criteria for linesearch methods without derivatives in this paper acceptability criteria for the linesearch stepsize are introduced which require only function values simple algorithm models based on these criteria are presented some modifications of criteria based on the knowledge of the directional derivative are also illustrated

convergence of liu storey conjugate gradient method abstract the conjugate gradient method is a useful and powerful approach for solving large scale minimization problems liu and storey developed a conjugate gradient method which has good numerical performance but no global convergence result under traditional line searches such as armijo wolfe and goldstein line searches in this paper a convergent version of liu storey conjugate gradient method ls in short is proposed for minimizing functions that have lipschitz continuous partial derivatives by estimating the lipschitz constant of the derivative of objective functions we can find an adequate step size at each iteration so as to guarantee the global convergence and improve the efficiency of ls method in practical computation

global convergence of a modified fletcher reeves conjugate gradient method with armijo type line search in this paper we are concerned with the conjugate gradient methods for solving unconstrained optimization problems it is well known that the direction generated by a conjugate gradient method may not be a descent direction of the objective function in this paper we take a little modification to the fletcher reeves fr method such that the direction generated by the modified method provides a descent direction for the objective function this property depends neither on the line search used nor on the convexity of the objective function moreover the modified method reduces to the standard fr method if line search is exact under mild conditions we prove that the modified method with armijo type line search is globally convergent even if the objective function is nonconvex we also present some numerical results to show the efficiency of the proposed method

maximization by parts in likelihood inference this article presents and examines a new algorithm for solving a score equation for the maximum likelihood estimate in certain problems of practical interest the method circumvents the need to compute second order derivatives of the full likelihood function it exploits the structure of certain models that yield a natural decomposition of a very complicated likelihood function in this decomposition the first part is a log likelihood from a simply analyzed model and the second part is used to update estimates from the first part convergence properties of this iterative fixed point algorithm are examined and asymptotics are derived for estimators obtained using only a finite number of iterations illustrative examples considered in the article include multivariate gaussian copula models nonnormal random effects models generalized linear mixed models and state space models properties of the algorithm and of estimators are evaluated in simulation studies on a bivariate copula model and a nonnormal

new properties of a nonlinear conjugate gradient method this paper provides several new properties of the nonlinear conjugate gradient method in 5 firstly the method is proved to have a certain self adjusting property that is independent of the line search and the function convexity secondly under mild assumptions on the objective function the method is shown to be globally convergent with a variety of line searches thirdly we find that instead of the negative gradient direction the search direction defined by the nonlinear conjugate gradient method in 5 can be used to restart any optimization method while guaranteeing the global convergence of the method some numerical results are also presented

globally convergent conjugate gradient algorithms the paper examines a method of implementing an angle test for determining when to restart conjugate gradient methods in a steepest descent direction the test is based on guaranteeing that the cosine of the angle between the search direction and the negative gradient is within a constant multiple of the cosine of the angle between the fletcher reeves search direction and the negative gradient this guarantees convergence for the fletcher reeves method is known to converge numerical results indicate little if anything is lost in efficiency and indicate gains may well be possible for large problems

multivariate spectral gradient method for unconstrained optimization multivariate spectral gradient method is proposed for solving unconstrained optimization problems combined with some quasi newton property multivariate spectral gradient method allows an individual adaptive stepsize along each coordinate direction which guarantees that the method is finitely convergent for positive definite quadratics especially it converges no more than two steps for positive definite quadratics with diagonal hessian and quadratically for objective functions with positive definite diagonal hessian moreover based on a nonmonotone line search global convergence is established for multivariate spectral gradient algorithm at last numerical results are reported which show that this method is promising and deserves further discussing

generalized method of moments specification testing abstract this paper analyzes the asymptotic power properties of specification tests which are based on a finite set of moment conditions it shows that any such test may fail against general misspecification that causes estimator inconsistency the mutual asymptotic equivalence of maximal degree of freedom tests is shown and the form of optimal tests against specific forms of misspecification is derived applications to testing for exogeneity of a set of instrumental variables are presented

covariance matrix estimation and the power of the overidentifying restrictions test class of heteroscedasticity and autocorrelation consistent covariance hacc matrix estimators that are now routinely used by practitioners in the calculation of the two step gmm estimator in applications it is customary to assume the model is correctly specified during the estimation and only to assess the specification after the second step using a statistic such as the overidentifying restrictions test one important consequence of this methodology is that the hacc estimator is calculated under the assumption that the model is correctly specified in this paper we examine the implications of model misspecification for the hacc covariance matrix estimator and the overidentifying restrictions test it is shown that the hacc estimator is asymptotically equivalent to the sum of two matrices one of these matrices is nonsingular and o 1 the other is of rank one and ob where b is the tt bandwidth used in the hacc estimator it is shown that this structure implies the inverse of the hacc estimator converges to a singular matrix as b with t and this t limiting matrix has rank q y 1 where q is the dimension of the moment condition it is shown that this limiting behavior translates into an overidentifying restrictions test that is ot rb in contrast it is shown that the overidentifying restrictions test is consistent pt and ot if the covariance matrix estimator is consistent under both null and alterna p tive this can be achieved by constructing the hacc estimator using the sample moment in mean deviation form 3

a prp based residual method for large scale monotone nonlinear equations this paper presents a derivative free residual method for solving large scale monotone nonlinear equations which may be nonsmooth the method is constructed by replacing the gradients of the unmodified polak ribiere polyak prp nonlinear conjugate gradient method with the residuals and combining the hyperplane projection technique under suitable conditions we show that the proposed method converges globally in the sense that the whole iterative sequence converges to a solution of the problem some numerical results are reported to show its efficiency

a new descent algorithm with curve search rule in the paper we present a new descent algorithm with curve search rule for unconstrained minimization problems at each iteration the next iterative point is determined by means of a curve search rule it is particular that the search direction and the step size is determined simultaneously at each iteration of the new algorithm the algorithm similarly to conjugate gradient methods avoids the computation and storage of some matrices associated with the hessian of objective functions unlike ode methods it is not required to solve ordinary differential equations at each iteration although the convergence rate of our algorithm is not as fast as that of newton like methods it is suitable to solve large scale minimization problems

convergence of line search methods for unconstrained optimization line search methods are traditional and successful methods for solving unconstrained optimization problems its convergence has attracted more attention in recent years in this paper we analyze the general results on convergence of line search methods with seven line search rules it is clarified that the search direction plays a main role in these methods and that step size guarantees the global convergence in some cases it is also proved that many line search methods have same convergence property these convergence results can enable us to design powerful effective and stable algorithms in practice finally a class of special line search methods is investigated

a globally convergent version of the polak ribiere conjugate gradient method in this paper we propose a new line search algorithm that ensures global convergence of the polak ribiere conjugate gradient method for the unconstrained minimization of nonconvex differentiable functions in particular we show that with this line search every limit point produced by the polak ribiere iteration is a stationary point of the objective function moreover we define adaptive rules for the choice of the parameters in a way that the first stationary point along a search direction can be eventually accepted when the algorithm is converging to a minimum point with positive definite hessian matrix under strong convexity assumptions the known global convergence results can be reobtained as a special case from a computational point of view we may expect that an algorithm incorporating the step size acceptance rules proposed here will retain the same good features of the polak ribiere method while avoiding pathological situations

quadratic cost flow and the conjugate gradient method by introducing quadratic penalty terms a convex non separable quadratic network program can be reduced to an unconstrained optimization problem whose objective function is a piecewise quadratic and continuously differentiable function a conjugate gradient method is applied to the reduced problem and its convergence is proved the computation exploits the special network data structures originated from the network simplex method this algorithmic framework allows direct extension to multicommodity cost flows some preliminary computational results are presented

two new conjugate gradient methods based on modified secant equations following the approach proposed by dai and liao we introduce two nonlinear conjugate gradient methods for unconstrained optimization problems one of our proposed methods is based on a modified version of the secant equation proposed by zhang deng and chen and zhang and xu and the other is based on the modified bfgs update proposed by yuan an interesting feature of our methods is their account of both the gradient and function values under proper conditions we show that one of the proposed methods is globally convergent for general functions and that the other is globally convergent for uniformly convex functions to enhance the performance of the line search procedure we also propose a new approach for computing the initial steplength to be used for initiating the procedure we provide a comparison of implementations of our methods with the efficient conjugate gradient methods proposed by dai and liao and hestenes and stiefel numerical test results show the efficiency of our proposed methods

a new conjugate gradient method with guaranteed descent and an efficient line search a new nonlinear conjugate gradient method and an associated implementation based on an inexact line search are proposed and analyzed with exact line search our method reduces to a nonlinear version of the hestenes stiefel conjugate gradient scheme for any inexact line search our scheme satisfies the descent condition bf g k sf t bf d k le frac 7 8 bf g k 2 moreover a global convergence result is established when the line search fulfills the wolfe conditions a new line search scheme is developed that is efficient and highly accurate efficiency is achieved by exploiting properties of linear interpolants in a neighborhood of a local minimizer high accuracy is achieved by using a convergence criterion which we call the approximate wolfe conditions obtained by replacing the sufficient decrease criterion in the wolfe conditions with an approximation that can be evaluated with greater precision in a neighborhood of a local minimum than the usual sufficient decrease criterion numerical comparisons are given with both l bfgs and conjugate gradient methods using the unconstrained optimization problems in the cute library

a note about wyl s conjugate gradient method and its applications this paper reviews the development of different versions of nonlinear conjugate gradient methods shortly the special attentions were given to the wyl method which was proposed by zengxin wei et al the convergence properties of some conjugate gradient methods applied mathematics and computation 183 2006 1341 1350 and its applications

a practical pr conjugate gradient method only using gradient abstract in this paper we consider the polak ribiere or polak ribiere plus conjugate gradient method for solving optimality condition of an unconstrained minimization problem we give two new steplength rules only using gradient and under gradient lipschitz assumption prove this method s global convergence correspondingly then we develop a practical polak ribiere plus method whose steplength is located by one inequality only using gradient and report promising numerical results on high accuracy solution for some standard test problems when compared to the state of art methods in this research direction importantly our work provides a new idea of devising a practical version of the celebrated polak ribiere or polak ribiere plus method

the barzilai and borwein gradient method for the large scale unconstrained minimization problem the barzilai and borwein gradient method for the solution of large scale unconstrained minimization problems is considered this method requires few storage locations and very inexpensive computations furthermore it does not guarantee descent in the objective function and no line search is required recently the global convergence for the convex quadratic case has been established however for the nonquadratic case the method needs to be incorporated in a globalization scheme in this work a nonmonotone line search strategy that guarantees global convergence is combined with the barzilai and borwein method this strategy is based on the nonmonotone line search technique proposed by grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 numerical results to compare the behavior of this method with recent implementations of the conjugate gradient method are presented these results indicate that the global barzilai and borwein method may allow some significant reduction in the number of line searches and also in the number of gradient evaluations

low complexity rls algorithms using dichotomous coordinate descent iterations in this paper we derive low complexity recursive least squares rls adaptive filtering algorithms we express the rls problem in terms of auxiliary normal equations with respect to increments of the filter weights and apply this approach to the exponentially weighted and sliding window cases to derive new rls techniques for solving the auxiliary equations line search methods are used we first consider conjugate gradient iterations with a complexity of operations per sample being the number of the filter weights to reduce the complexity and make the algorithms more suitable for finite precision implementation we propose a new dichotomous coordinate descent dcd algorithm and apply it to the auxiliary equations this results in a transversal rls adaptive filter with complexity as low as multiplications per sample which is only slightly higher than the complexity of the least mean squares lms algorithm multiplications simulations are used to compare the performance of the proposed algorithms against the classical rls and known advanced adaptive algorithms fixed point fpga implementation of the proposed dcd based rls algorithm is also discussed and results of such implementation are presented

global convergence of the polak ribiere polyak conjugate gradient method with an armijo type inexact line search for nonconvex unconstrained optimization problems we propose two algorithms for nonconvex unconstrained optimization problems that employ polak ribiere polyak conjugate gradient formula and new inexact line search techniques we show that the new algorithms converge globally if the function to be minimized has lipschitz continuous gradients preliminary numerical results show that the proposed methods for particularly chosen line search conditions are very promising

two modified hs type conjugate gradient methods for unconstrained optimization problems abstract based on the modified secant equation we propose two new hs type conjugate gradient formulas their forms are similar to the original hs conjugate gradient formula and inherit all nice properties of the hs method by utilizing the technique of the three term hs method in zhang et al 2007 15 without the requirement of truncation and convexity of the objective function we show that one with wolfe line search and the other with armijo line search are globally convergent moreover under some mild conditions the linear convergence rate of the two modified methods is established the numerical results show that the proposed methods are efficient

testing when a parameter is on the boundary of the maintained hypothesis this paper considers testing problems where several of the standard regularity conditions fail to hold we consider the case where i parameter vectors in the null hypothesis may lie on the boundary of the maintained hypothesis and ii there may be a nuisance parameter that appears under the alternative hypothesis but not under the null the paper establishes the asymptotic null and local alternative distributions of quasi likelihood ratio rescaled quasi likelihood ratio wald and score tests in this case the results apply to tests based on a wide variety of extremum estimators and apply to a wide variety of models examples treated in the paper are i tests of the null hypothesis of no conditional heteroskedasticity in a garch 1 1 regression model and ii tests of the null hypothesis that some random coefficients have variances equal to zero in a random coefficients regression model with possibly correlated random coefficients

further studies on the wei yao liu nonlinear conjugate gradient method the wei yao liu wyl method proposed by wei et al z wei s yao l liu the convergence properties of some new conjugate gradient methods appl math comput 183 2006 1341 1350 has been proved to converge globally for uniformly convex functions with exact line search and nonconvex functions with the strong wolfe line search if the parameter email protected 0 1 4 in this paper we further study the global convergence properties of the wyl method we show that this method still converges globally for nonconvex optimization when the strong wolfe line search with the parameter s 1 4 or some armijo type line searches are used

an inexact and nonmonotone proximal method for smooth unconstrained minimization an implementable proximal point algorithm is established for the smooth nonconvex unconstrained minimization problem at each iteration the algorithm minimizes approximately a general quadratic by a truncated strategy with step length control the main contributions are i a framework for updating the proximal parameter ii inexact criteria for approximately solving the subproblems iii a nonmonotone criterion for accepting the iterate the global convergence analysis is presented together with numerical results that validate and put into perspective the proposed approach

a method of simulated moments for estimation of discrete response models without numerical integration this paper proposes a simple modification of a conventional method of moments estimator for a discrete response model replacing response probabilities that require numerical integration with estimators obtained by monte carlo simulation this method of simulated moments msm does not require precise estimates of these probabilities for consistency and asymptotic normality relying instead on the law of large numbers operating across observations to control simulation error and hence can use simulations of practical size the method is useful for models such as high dimensional multinomial probit mnp where computation has restricted applications

a class of gradient unconstrained minimization algorithms with adaptive stepsize in this paper the development convergence theory and numerical testing of a class of gradient unconstrained minimization algorithms with adaptive stepsize are presented the proposed class comprises four algorithms the first two incorporate techniques for the adaptation of a common stepsize for all coordinate directions and the other two allow an individual adaptive stepsize along each coordinate direction all the algorithms are computationally efficient and possess interesting convergence properties utilizing estimates of the lipschitz constant that are obtained without additional function or gradient evaluations the algorithms have been implemented and tested on some well known test cases as well as on real life artificial neural network applications and the results have been very satisfactory

convergence of nonmonotone line search method in this paper we develop a new nonmonotone line search for general line search method and establish some global convergence theorems the new nonmonotone line search is a novel form of the nonmonotone armijo line search and allows one to choose a larger step size at each iteration which is available in constructing new line search methods and possibly reduces the function evaluations at each iteration moreover we analyze the convergence rate of some special line search methods with the new line search preliminary numerical results show that some line search methods with the new nonmonotone line search are available and efficient in practical computation

a new super memory gradient method with curve search rule in this paper we propose a new super memory gradient method with curve search rule for unconstrained optimization problems the method uses previous multi step iterative information and some curve search rules to generate new iterative points at each iteration this makes the new method converge stably and be more suitable for solving large scale optimization problems than other similar methods we analyze the global convergence and convergence rate under some mild conditions numerical experiments show that some new algorithms are available and effective in practical computation

a sufficient descent nonlinear conjugate gradient method for solving m tensor equations abstract tensor equations is a kind of important tensor optimization problems with higher order nonlinear equations which are widely used in engineering and economics this paper is concerned with solving m tensor equations we transform m tensor equations to nonlinear unconstrained optimization problems then a sufficient descent nonlinear conjugate gradient method with inexact line search is proposed for solving the transformed unconstrained optimization problem under only one mild assumption the global convergence of the proposed method is proved finally to show the effectiveness of the proposed nonlinear conjugate gradient method we compare it with three term conjugate gradient method and newton method the numerical results show that the proposed nonlinear conjugate gradient method is potentially efficient

the large sample behaviour of the generalized method of moments estimator in misspecified models this paper presents the limiting distribution theory for the gmm estimator when the estimation is based on a population moment condition which is subject to non local or fixed misspecification it is shown that if the parameter vector is overidentified then the weighting matrix plays a far more fundamental role than it does in the corresponding analysis for correctly specified models specifically the rate of convergence of the estimator depends on the rate of convergence of the weighting matrix to its probability limit the analysis is presented for four particular choices of weighting matrix which are commonly used in practice in each case the limiting distribution theory is different and also different from the limiting distribution in a correctly specified model statistics are proposed which allow the researcher to test hypotheses about the parameters in misspecified models

convergence of descent method without line search line search method is sometimes a useful and efficient technique for solving unconstrained optimization problems especially for solving small and middle scale problems however a line search procedure is necessary at each iteration leading to a significant amount of computation in order to reduce the evaluations of objective functions and gradients the line search procedure should be avoided in algorithm design in this paper we propose a new descent method without line search for unconstrained optimization problems the algorithm is required to estimate some parameters at each iteration we analyze theoretically the global convergence of the new algorithm under mild conditions these theoretical conclusions can help us in designing new efficient methods for optimization problems

a conjugate gradient method with descent direction for unconstrained optimization a modified conjugate gradient method is presented for solving unconstrained optimization problems which possesses the following properties i the sufficient descent property is satisfied without any line search ii the search direction will be in a trust region automatically iii the zoutendijk condition holds for the wolfe powell line search technique iv this method inherits an important property of the well known polak ribiere polyak prp method the tendency to turn towards the steepest descent direction if a small step is generated away from the solution preventing a sequence of tiny steps from happening the global convergence and the linearly convergent rate of the given method are established numerical results show that this method is interesting

a new class of spectral conjugate gradient methods based on a modified secant equation for unconstrained optimization conjugate gradient methods have played a special role for solving large scale optimization problems due to the simplicity of their iteration convergence properties and their low memory requirements in this work we propose a new class of spectral conjugate gradient methods which ensures sufficient descent independent of the accuracy of the line search moreover an attractive property of our proposed methods is that they achieve a high order accuracy in approximating the second order curvature information of the objective function by utilizing the modified secant condition proposed by babaie kafaki et al s babaie kafaki r ghanbari n mahdavi amiri two new conjugate gradient methods based on modified secant equations journal of computational and applied mathematics 234 2010 1374 1386 further a global convergence result for general functions is established provided that the line search satisfies the wolfe conditions our numerical experiments indicate that our proposed methods are preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness

a new family of conjugate gradient methods in this paper we develop a new class of conjugate gradient methods for unconstrained optimization problems a new nonmonotone line search technique is proposed to guarantee the global convergence of these conjugate gradient methods under some mild conditions in particular polak ribiere polyak and liu storey conjugate gradient methods are special cases of the new class of conjugate gradient methods by estimating the local lipschitz constant of the derivative of objective functions we can find an adequate step size and substantially decrease the function evaluations at each iteration numerical results show that these new conjugate gradient methods are effective in minimizing large scale non convex non quadratic functions

coupling finite element and reliability analysis through proper generalized decomposition model reduction summary the fem is the main tool used for structural analysis when the design of the mechanical system involves uncertain parameters a coupling of the fem with reliability analysis algorithms allows to compute the failure probability of the system however this coupling leads to successive finite element analysis of parametric models involving high computational effort over the past years model reduction techniques have been developed in order to reduce the computational requirements in the numerical simulation of complex models the objective of this work is to propose an efficient methodology to compute the failure probability for a multi material elastic structure where the young moduli are considered as uncertain variables a proper generalized decomposition algorithm is developed to compute the solution of parametric multi material model this parametrized solution is used in conjunction with a first order reliability method to compute the failure probability of the structure applications to multilayered structures in two dimensional plane elasticity are presented copyright 2013 john wiley sons ltd

statistical inference procedures for bivariate archimedean copulas abstract a bivariate distribution function h x y with marginals f x and g y is said to be generated by an archimedean copula if it can be expressed in the form h x y 1 f x g y for some convex decreasing function defined on 0 1 in such a way that 1 0 many well known systems of bivariate distributions belong to this class including those of gumbel ali mikhail haq thelot clayton frank and hougaard frailty models also fall under that general prescription this article examines the problem of selecting an archimedean copula providing a suitable representation of the dependence structure between two variates x and y in the light of a random sample x 1 y 1 x n y n the key to the estimation procedure is a one dimensional empirical distribution function that can be constructed whether the uniform representation of x and y is archimedean or not and independently of their marginals this semiparametric estimator based on a decomposition of kendall s tau statistic

a nonlinear conjugate gradient method with a strong global convergence property conjugate gradient methods are widely used for unconstrained optimization especially large scale problems the strong wolfe conditions are usually used in the analyses and implementations of conjugate gradient methods this paper presents a new version of the conjugate gradient method which converges globally provided the line search satisfies the standard wolfe conditions the conditions on the objective function are also weak being similar to those required by the zoutendijk condition

estimation and model selection of semiparametric copula based multivariate dynamic models under copula misspecification we introduce a new class of semiparametric copula based multivariate dynamic scomdy models which specify the conditional mean and the conditional variance of a multivariate time series parametrically but specify the multivariate distribution of the standardized innovation semiparametrically as a parametric copula evaluated at nonparametric marginal distributions we first study large sample properties of the estimators of scomdy model parameters under a misspecified parametric copula then propose pseudo likelihood ratio plr tests for model selection between two scomdy models with possibly misspecified copulas and finally develop plr tests for model selection between more than two scomdy models the limiting null distributions of the plr tests do not depend on the estimation of conditional mean and conditional variance parameters hence are very easy to simulate empirical applications to three and higher dimensional daily exchange rate series indicate that a scomdy model with a tail dependent copula is generally preferred

on association in a copula with time transformations we consider estimation of the cross ratio in clayton s 1978 copula in which covariates are incorporated into the marginal distributions via semiparametric accelerated life regression models generalisations of oakes 1982 1986 concordance estimating equations yield a closed form for the association parameter under right censoring joint inferences about covariate effects in the marginal models and the cross ratio are obtained with u statistics martingales and a resampling technique for nonsmooth estimating equations simulating under an exponential model we find that our procedure may provide a more precise estimate of association than other methods for proportional hazards models a goodness of fit test for the assumed copula is presented and used in an analysis of a diabetic retinopathy dataset

an assessment of nonmonotone linesearch techniques for unconstrained optimization the purpose of this paper is to discuss the potential of nonmonotone techniques for enforcing convergence of unconstrained minimization algorithms from starting points distant from the solution linesearch based algorithms are considered for both small and large problems and extensive numerical experiments show that this potential is sometimes considerable a new variant is introduced in order to limit some of the identified drawbacks of the existing techniques this variant is again numerically tested and appears to be competitive finally the impact of preconditioning on the considered methods is examined

semiparametric dynamic max copula model for multivariate time series summary the paper presents a novel non linear framework for the construction of flexible multivariate dependence structure i e copulas from existing copulas based on a straightforward pairwise max rule the newly constructed max copula has a closed form and has strong interpretability compared with the classical linear symmetric mixture copula the max copula can be viewed as a non linear asymmetric framework it is capable of modelling asymmetric dependence and joint tail behaviour while also offering good performance in non extremal behaviour modelling max copulas that are based on single factor and block factor models are developed to offer parsimonious modelling for structured dependence especially in high dimensional applications combined with semiparametric time series models the max copula can be used to develop flexible and accurate models for multivariate time series a new semiparametric composite maximum likelihood method is proposed for parameter estimation where the consistency and asymptotic normality of estimators are established the flexibility of the max copula and the accuracy of the proposed estimation procedure are illustrated through extensive numerical experiments real data applications in value at risk estimation and portfolio optimization for financial risk management demonstrate the max copula s promising ability to capture accurately joint movements of high dimensional multivariate stock returns under both normal and crisis regimes of the financial market

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

simulated method of moments estimation for copula based multivariate models this article considers the estimation of the parameters of a copula via a simulated method of moments mm type approach this approach is attractive when the likelihood of the copula model is not known in closed form or when the researcher has a set of dependence measures or other functionals of the copula that are of particular interest the proposed approach naturally also nests mm and generalized method of moments estimators drawing on results for simulation based estimation and on recent work in empirical copula process theory we show the consistency and asymptotic normality of the proposed estimator and obtain a simple test of overidentifying restrictions as a specification test the results apply to both iid and time series data we analyze the finite sample behavior of these estimators in an extensive simulation study we apply the model to a group of seven financial stock returns and find evidence of statistically significant tail dependence and mild evidence that the dependence between thes

global convergence of a two parameter family of conjugate gradient methods without line search we study the global convergence of a two parameter family of conjugate gradient methods in which the line search procedure is replaced by a fixed formula of stepsize this character is of significance if the line search is expensive in a particular application in addition to the convergence results we present computational results for various conjugate gradient methods without line search including those discussed by sun and zhang ann oper res 103 2001 161 173

memory gradient method with goldstein line search in this paper we present a multi step memory gradient method with goldstein line search for unconstrained optimization problems and prove its global convergence under some mild conditions we also prove the linear convergence rate of the new method when the objective function is uniformly convex numerical results show that the new algorithm is suitable to solve large scale optimization problems and is more stable than other similar methods in practical computation

a new class of nonlinear conjugate gradient coefficients with exact and inexact line searches conjugate gradient cg methods have played an important role in solving large scale unconstrained optimization in this paper we propose a new family of cg coefficients k that possess sufficient descent conditions and global convergence properties this new k is an extension of the already proven k rmil from rivaie et al 19 a new class of nonlinear conjugate gradient coefficient with global convergence properties appl math comp 218 2012 11323 11332 global convergence result is established using both exact and inexact line searches numerical results show that the performance of the new proposed formula is quite similar to k rmil and suited to both line searches importantly the performance of this k is more efficient and superior than the other well known k

a barzilai and borwein scaling conjugate gradient method for unconstrained optimization problems in this paper we combine the conjugate gradient method with the barzilai and borwein gradient method and propose a barzilai and borwein scaling conjugate gradient method for nonlinear unconstrained optimization problems the new method does not require to compute and store matrices associated with hessian of the objective functions and has an advantage of less computational efforts moreover the descent direction property and the global convergence are established when the line search fulfills the wolfe conditions the limited numerical experiments and comparisons show that the proposed algorithm is potentially efficient

diffraction of waves on square lattice by semi infinite crack in this paper a discrete analogue of sommerfeld half plane diffraction is investigated the two dimensional problem of diffraction on a square lattice of a plane transverse wave by a semi infinite crack is solved the discrete wiener hopf method has been used to obtain the exact solution of the discrete helmholtz equation with input data prescribed on the crack boundary sites due to a time harmonic incident wave it is established that there exists a unique saddle point for the diffraction integral and its properties are characterized an asymptotic approximation of the solution in the far field is provided and for some values of the frequency it is compared with the numerical solution of the diffraction problem using a finite grid a low frequency approximation of the solution in integral form recovers the classical continuum solution at sufficiently large frequency in the pass band the effects due to discreteness and anisotropy appear the analysis is relevant to a 5 point discretization bas

globally convergent modified perry s conjugate gradient method abstract conjugate gradient methods are probably the most famous iterative methods for solving large scale optimization problems in scientific and engineering computation characterized by the simplicity of their iteration and their low memory requirements in this paper we propose a new conjugate gradient method which is based on the mbfgs secant condition by modifying perry s method our proposed method ensures sufficient descent independent of the accuracy of the line search and it is globally convergent under some assumptions numerical experiments are also presented

a nonmonotone line search technique and its application to unconstrained optimization a new nonmonotone line search algorithm is proposed and analyzed in our scheme we require that an average of the successive function values decreases while the traditional nonmonotone approach of grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 requires that a maximum of recent function values decreases we prove global convergence for nonconvex smooth functions and r linear convergence for strongly convex functions for the l bfgs method and the unconstrained optimization problems in the cute library the new nonmonotone line search algorithm used fewer function and gradient evaluations on average than either the monotone or the traditional nonmonotone scheme

a family of distributions for modelling non elliptically symmetric multivariate data summary the paucity of distributions that can be used as an aid to modelling the structure of multivariate data is a recognized limitation we present a relatively simple family of distributions for use in describing data which are not elliptically symmetric properties of the family are discussed and it is shown that multivariate pareto burr and logistic distributions are special cases in addition simulation methods for applications in monte carlo robustness studies are outlined and finally the fit to a data set on uranium exploration is compared to that obtained using the usual bivariate normal distribution

testing inequality constraints in linear econometric models abstract this paper develops three asymptotically equivalent tests for examining the validity of imposing linear inequality restrictions on the parameters of linear econometric models first we consider the model y x e where e is n 0 and the hypothesis test h r r versus k rk later we generalize this testing framework to the linear simultaneous equations model we show that the joint asymptotic distribution of these test statistics and the test statistics from the hypothesis test h r r versus k r r is a weighted sum of two sets of independent 2 distributions we also derive a useful duality relation between the multivariate inequality constraints test developed here and the multivariate one sided hypothesis test in small samples these three test statistics satisfy inequalities similar to those derived by berndt and savin 1977 for the case of equality constraints the paper also contains an illustrative application of this testing technique

diffraction of waves on triangular lattice by a semi infinite rigid constraint and crack abstract an exact solution is provided for a discrete analog of each of the two sommerfeld diffraction problems using a triangular lattice model with nearest neighbor interactions deforming in the anti plane direction the discrete helmholtz equation with time harmonic data prescribed on semi infinite row s of lattice sites is solved using the discrete wiener hopf method an asymptotic expression of the exact solution given in the form of a contour integral has been obtained in far field using the standard approximation of the diffraction integrals based on the method of stationary phase in case of the rigid constraint diffraction the displacement of particles on the semi infinite row complementing the constrained lattice sites as well as on the adjacent row is presented in closed form as a discrete convolution for the crack diffraction problem the length of both types of slant bonds on the semi infinite row complementing the crack as well as the crack opening displacement is given in a similar form but in terms of the fourier coefficients of the associated wiener hopf kernel which are not available in closed form the displacement field associated with the scattered waves at sites far from the defect tip as well as few sites near the tip is compared graphically with that of a numerical solution on a finite grid both discrete sommerfeld problems are naturally relevant to their continuous counterparts involving the traditional helmholtz equation that model the diffraction of electromagnetic and acoustic waves by a semi infinite screen based on 7 point discretization on a triangular grid