a variant of trust region methods for unconstrained optimization in traditional trust region methods one in practice always employs quadratic model or conic model as the local approximation of the objective function and there are lots of theoretical results and ripe algorithms in this paper we develop a practical trust region algorithm with a linear model for unconstrained optimization problems in particular we combine a special weighted norm with the linear model so that the subproblem contains the information of hessian matrix of the objective function which successfully overcome the drawbacks of linear model and we further complete the trust region methods with three main types of models namely linear model quadratic model and conic model we show that the new method preserves the strong global convergence moreover under the linear model it unveils independently that the line search algorithms can be viewed as a special case of trust region methods numerical results indicate that the new method is effective and practical

an augmented lagrangian interior point method using directions of negative curvature we describe an efficient implementation of an interior point algorithm for non convex problems that uses directions of negative curvature these directions should ensure convergence to second order kkt points and improve the computational efficiency of the procedure some relevant aspects of the implementation are the strategy to combine a direction of negative curvature and a modified newton direction and the conditions to ensure feasibility of the iterates with respect to the simple bounds the use of multivariate barrier and penalty parameters is also discussed as well as the update rules for these parameters we analyze the convergence of the procedure both the linesearch and the update rule for the barrier parameter behave appropriately as the main goal of the paper is the practical usage of negative curvature a set of numerical results on small test problems is presented based on these results the relevance of using directions of negative curvature is discussed

convergence of the iterates of descent methods for analytic cost functions in the early eighties lojasiewicz in seminari di geometria 1982 1983 universita di bologna istituto di geometria dipartimento di matematica 1984 pp 115 117 proved that a bounded solution of a gradient flow for an analytic cost function converges to a well defined limit point in this paper we show that the iterates of numerical descent algorithms for an analytic cost function share this convergence property if they satisfy certain natural descent conditions the results obtained are applicable to a broad class of optimization schemes and strengthen classical weak convergence results for descent methods to strong limit point convergence for a large class of cost functions of practical interest the result does not require that the cost has isolated critical points and requires no assumptions on the convexity of the cost nor any nondegeneracy conditions on the hessian of the cost at critical points

tensor methods for nonlinear equations a new class of methods for solving systems of nonlinear equations called tensor methods is introduced tensor methods are general purpose methods intended especially for problems where the jacobian matrix at the solution is singular or ill conditioned they base each iteration on a quadratic model of the nonlinear function the standard linear model augmented by a simple second order term the second order term is selected so that the model interpolates function values from several previous iterations as well as the current function value and jacobian the tensor method requires no more function and derivative information per iteration and hardly more storage or arithmetic per iteration than a standard method based on newton s method in extensive computational tests a tensor algorithm is significantly more efficient than a similar algorithm based on the standard linear model both on standard nonsingular test problems and on problems where the jacobian at the solution is singular

dogleg paths and trust region methods with back tracking technique for unconstrained optimization in this paper we improve approximate trust region methods via a class of dogleg paths for unconstrained optimization the dogleg paths include both definite and indefinite ones a hybrid strategy using both trust region and line search techniques is adopted which switches to back tracking steps when a trial step produced by the trust region subproblem is unacceptable we show that the algorithm preserves the strong convergence properties of trust region methods numerical results are presented and discussed

exploiting hessian matrix and trust region algorithm in hyperparameters estimation of gaussian process gaussian process gp regression is a bayesian non parametric regression model showing good performance in various applications however it is quite rare to see research results on log likelihood maximization algorithms instead of the commonly used conjugate gradient method the hessian matrix is first derived simplified in this paper and the trust region optimization method is then presented to estimate gp hyperparameters numerical experiments verify the theoretical analysis showing the advantages of using hessian matrix and trust region algorithms in the gp context the trust region optimization method is a robust alternative to conjugate gradient method also in view of future researches on approximate and or parallel gp implementation

combining and scaling descent and negative curvature directions the aim of this paper is the study of different approaches to combine and scale in an efficient manner descent information for the solution of unconstrained optimization problems we consider the situation in which different directions are available in a given iteration and we wish to analyze how to combine these directions in order to provide a method more efficient and robust than the standard newton approach in particular we will focus on the scaling process that should be carried out before combining the directions we derive some theoretical results regarding the conditions necessary to ensure the convergence of combination procedures following schemes similar to our proposals finally we conduct some computational experiments to compare these proposals with a modified newton s method and other procedures in the literature for the combination of information

application of the heston stochastic volatility model for borsa istanbul using impression matrix norm we study the behavior of solutions for stochastic differential equations such as the heston stochastic volatility model we examine the numerical solutions using euler maruyama milstein and stochastic runge kutta methods to investigate whether there is a role of the methods for different volatility cases or not related to the impact of cumulative errors on this application we perform simulations for different stock market conditions by using the large data set from borsa istanbul 100 bist 100 between 04 01 2007 and 31 12 2012 we use volatilities in terms of extreme values at the overlapping case when we examine initial and long term volatilities for the application of the heston model we also apply unit volatility based on extreme values to approximate volatilities in our analysis we examine the advantages and limitations of the model moreover we introduce 3 dimensional matrix norms furthermore we define market impression matrix norm as an application to the 3 dimensional matrix norms we can benefit from it to quantify market impression approximately by means of the numerical solutions for the stochastic differential equations finally we analyze the simulation results for various parameters

on the convergence of interior reflective newton methods for nonlinear minimization subject to bounds we consider a new algorithm an interior reflective newton approach for the problem of minimizing a smooth nonlinear function of many variables subject to upper and or lower bounds on some of the variables this approach generatesstrictly feasible iterates by using a new affine scaling transformation and following piecewise linear paths reflection paths the interior reflective approach does not require identification of an activity set in this paper we establish that the interior reflective newton approach is globally and quadratically convergent moreover we develop a specific example of interior reflective newton methods which can be used for large scale and sparse problems

trust region methods for nonlinear elliptic equations with radial basis functions we consider the numerical solution of nonlinear elliptic boundary value problems with kansas method we derive analytic formulas for the jacobian and hessian of the resulting nonlinear collocation system and exploit them within the framework of the trust region algorithm this ansatz is tested on semilinear quasilinear and fully nonlinear elliptic pdes including plateaus problem heleshaw flow and the mongeampre equation with excellent results the new approach distinctly outperforms previous ones based on linearization or finite difference jacobians

numerical solutions of stochastic differential equations implementation and stability issues stochastic differential equations sdes arise from physical systems where the parameters describing the system can only be estimated or are subject to noise there has been much work done recently on developing numerical methods for solving sdes this paper will focus on stability issues and variable stepsize implementation techniques for numerically solving sdes effectively

statistical analysis of path losses for sectorized wireless networks in modern mobile communication networks such as 3g and 4g networks sectorized antennas have been widely used to divide each cell into multiple sectors in order to improve coverage spectrum efficiency and quality of service large scale path loss from a transmitting antenna to a receiving antenna should include 1 propagation attenuation that depends on transmission distance 2 shadowing that depends on surrounding environment and 3 antenna loss that depends on a sectorized antenna pattern and transmission angle an in depth analysis of statistical characteristics of large scale path losses involving with these three factors is crucial for the design operation evaluation and optimization of modern sectorized wireless networks in this paper a sectorized antenna pattern is for the first time considered in the derivation of a closed form expression of a probability density function pdf of large scale path losses specifically we first discover that the normalized pdf of propagation attenuation plus shadowing which can be approximated by the gaussian mixture model gmm with all system parameters is fully determined by our newly defined metric 10 ln 10 beta sigma s namely the attenuation exponent beta to standard deviation of shadowing sigma s ratio asr the convolution of gmm and antenna loss statistics is elaborately transformed to a series of differential equations a closed form pdf of large scale path losses with sectorized antenna pattern can be obtained by solving these differential equations to reduce the computational complexity we further prove that the exciting sources of these differential equations can be tightly approximated by weighted gaussian functions and thus the final solutions i e pdf of path losses can be derived in the form of gaussian and dawson functions our analytical results are verified by extensive numerical computation and monte carlo simulation results e g the impact of asr on the shape of pdf of propagation attenuation plus shadowing compared with traditional gaussian fitting approach our newly derived pdf of large scale path losses with sectorized antenna patterns is at least two orders of magnitude more accurate in terms of kullback leibler divergence under typical propagation attenuation and shadowing conditions

two step algorithms for nonlinear optimization with structured applications in this paper we propose extensions to trust region algorithms in which the classical step is augmented with a second step that we insist yields a decrease in the value of the objective function the classical convergence theory for trust region algorithms is adapted to this class of two step algorithms the algorithms can be applied to any problem with variable s whose contribution to the objective function is a known functional form in the nonlinear programming package lancelot they have been applied to update slack variables and variables introduced to solve minimax problems leading to enhanced optimization efficiency extensive numerical results are presented to show the effectiveness of these techniques

newton s method with a model trust region modification a modified newton method for unconstrained minimization is presented and analyzed the modification is based upon the model trust region approach this report contains a thorough analysis of the locally constrained quadratic minimizations that arise as subproblems in the modified newton iteration several promising alternatives are presented for solving these subproblems in ways that overcome certain theoretical difficulties exposed by this analysis very strong convergence results are presented concerning the minimization algorithm in particular the explicit use of second order information is justified by demonstrating that the iterates converge to a point which satisfies the second order necessary conditions for minimization with the exception of very pathological cases this occurs whenever the algorithm is applied to problems with continuous second partial derivatives

on local solutions of the celis dennis tapia subproblem we discuss the distribution of the local solutions of the celis dennis tapia cdt subproblem which appears in some trust region algorithms for nonlinear optimization we also give some examples to show the differences between the cdt subproblem and the single ball constraint subproblem these results show that the complexity of the cdt subproblem does not depend on the complexity of the structure of the dual plane thus they provide the possibility to search for the global minimizer in the dual plane

similarity transformation parameters recovery based on radon transform application in image registration and object recognition the radon transform since its introduction in the beginning of the last century has been studied deeply and applied by researchers in a great number of applications especially in the biomedical imaging fields by using the radon transform properties the issue is to recover the transformation parameters regarding the rotation scaling and translation by handling only the image projections assuming no access to the spatial domain of the image this paper proposes an algorithm using an extended version of the radon transform to recover such parameters relating to two unknown images directly from their projection data especially our approach deals with the problem of the estimation accuracy of the rotation angle and its finding in one step instead of two steps as it is reported in the literature this method may be applied in image registration as well in object recognition the results are for the first time exploited in object recognition where comparison with powerful descriptors shows the outstanding performance of the proposed paradigm moreover the influence of additive noise on registration and recognition experiments is discussed and shows the efficiency of the method to reduce the effect of the noise highlightswe define 2 based radon transform rt and motivate its use we propose an algorithm to recover rst transforms using only rt the algorithm is tested on images in real world application e g ct mri etc this rst parameter recovery method is newly applied in object recognition a new confusion matrix is designed to identify also a mislabeled class element

nonconvex optimization using negative curvature within a modified linesearch this paper describes a new algorithm for the solution of nonconvex unconstrained optimization problems with the property of converging to points satisfying second order necessary optimality conditions the algorithm is based on a procedure which from two descent directions a newton type direction and a direction of negative curvature selects in each iteration the linesearch model best adapted to the properties of these directions the paper also presents results of numerical experiments that illustrate its practical efficiency

unconstrained optimization of real functions in complex variables nonlinear optimization problems in complex variables are frequently encountered in applied mathematics and engineering applications such as control theory signal processing and electrical engineering optimization of these problems often requires a first or second order approximation of the objective function to generate a new step or descent direction however such methods cannot be applied to real functions of complex variables because they are necessarily nonanalytic in their argument i e the taylor series expansion in their argument alone does not exist to overcome this problem the objective function is usually redefined as a function of the real and imaginary parts of its complex argument so that standard optimization methods can be applied however this approach may needlessly disguise any inherent structure present in the derivatives of such complex problems although little known it is possible to construct an expansion of the objective function in its original complex variables by noti

a class of indefinite dogleg path methods for unconstrained minimization in this paper we propose a convenient curvilinear search method to solve trust region problems arising from unconstrained optimization problems the curvilinear paths we set forth are dogleg paths generated mainly by employing bunch parlett factorization for general symmetric matrices that may be indefinite this method is easy to implement and globally convergent it is proved that the method satisfies the first and second order stationary point convergence properties and that the convergence rate is quadratic under commonly used conditions on functions numerical experiments are conducted to compare this method with some existing methods

model based subspace projection beamforming for deep interference nulling this paper considers the problem of adaptive array processing for interference canceling to drive very deep nulls in difficult signal environments in many practical scenarios the achievable null depth is limited by covariance matrix estimation error leading to poor identification of the interference subspace we address the particularly troublesome cases of low interference to noise ratio inr relatively rapid interference motion and correlated noise across the receiving array a polynomial based model is incorporated in the proposed algorithm to track changes in the array covariance matrix over time mitigate interference subspace estimation errors and improve canceler performance the application of phased array feeds for radio astronomical telescopes is used to illustrate the problem and proposed solution here even weak residual interference after cancellation may obscure a signal of interest so very deep beampattern nulls are required performance for conventional subspace projection sp is compared with polynomial augmented sp using simulated and real experimental data showing null depth improvement of 6 to 30 db

a subspace implementation of quasi newton trust region methods for unconstrained optimization this paper studies subspace properties of trust region methods for unconstrained optimization assuming the approximate hessian is updated by quasi newton formulae and the initial hessian approximation is appropriately chosen it is shown that the trial step obtained by solving the trust region subproblem is in the subspace spanned by all the gradient vectors computed thus the trial step can be defined by minimizing the quasi newton quadratic model in the subspace based on this observation some subspace trust region algorithms are proposed and numerical results are also reported

numerical solution of bivariate and polyanalytic polynomial systems finding the real solutions of a bivariate polynomial system is a central problem in robotics computer modeling and graphics computational geometry and numerical optimization we propose an efficient and numerically robust algorithm for solving bivariate and polyanalytic polynomial systems using a single generalized eigenvalue decomposition in contrast to existing eigen based solvers the proposed algorithm does not depend on grobner bases or normal sets nor does it require computing eigenvectors or solving additional eigenproblems to recover the solution the method transforms bivariate systems into polyanalytic systems and then uses resultants in a novel way to project the variables onto the real plane associated with the two variables solutions are returned counting multiplicity and their accuracy is maximized by means of numerical balancing and newton raphson refinement numerical experiments show that the proposed algorithm consistently recovers a higher percentage of solutions and is at the sa

a conic trust region method and its convergence properties in this paper we consider a conic trust region method for unconstrained optimization problems and analyze its convergence properties we propose a convenient curvilinear search method to approximately solve the arising conic trust region subproblem note that this approximate method preserves the strong convergence properties of the exact solution methods and is easy to implement both the linear and superlinear convergence of the method are established under commonly used conditions numerical experiments are conducted to show the efficiency of the new method

spectral gradient flow and equilibrium configurations of point vortices we formulate the problem of finding equilibrium configurations of n point vortices in the plane in terms of a gradient flow on the smallest singular value of a skew symmetric matrix m whose nullspa

constrained optimization for liquid crystal equilibria this paper compares the performance of penalty and lagrange multiplier approaches for the necessary unit length constraint in the computation of liquid crystal equilibrium configurations building on previous work in siam j sci comput 37 2015 pp s157 s176 siam j numer anal 53 2015 pp 2226 2254 the penalty method is derived and well posedness of the linearizations within the nonlinear iteration is discussed in addition the paper considers the effects of tailored trust region methods in the context of finite element discretizations and nested iteration for both formulations such methods are aimed at increasing the efficiency and robustness of each algorithm s nonlinear iterations three representative elastic equilibrium problems are considered to examine each method s performance the first two configurations have analytical expressions for their exact solutions and therefore convergence to the true solution is considered the third problem considers complicated boundary conditio

an energy error based method for the resolution of the cauchy problem in 3d linear elasticity a new method is described for the problem of expanding known displacement fields at the boundary of a solid together with the surface tractions on it towards the solid interior up to the inaccessible part of the boundary the solid is supposedly linearly elastic with known elastic moduli but not necessarily homogeneous nor isotropic the problem is the cauchy problem for the lame operator a new form of this cauchy problem suited for applications associated with surface tangential fields measurements is also stated and studied the method is based on the splitting of the elastic fields into two separate solutions of wellposed problems the gap between these fields is subsequently minimised with respect to the unknown boundary data in order to produce the desired expanded elastic fields the gap used here is an energy error associated with the elastic energy of the system various 3d applications are given including non linear boundary conditions on the unreachable boundary

roots of bivariate polynomial systems via determinantal representations we give two determinantal representations for a bivariate polynomial they may be used to compute the zeros of a system of two of these polynomials via the eigenvalues of a two parameter eigenvalue problem the first determinantal representation is suitable for polynomials with scalar or matrix coefficients and consists of matrices with asymptotic order n 2 4 where n is the degree of the polynomial the second representation is useful for scalar polynomials and has asymptotic order n 2 6 the resulting method to compute the roots of a system of two bivariate polynomials is very competitive with some existing methods for polynomials up to degree 10 as well as for polynomials with a small number of terms

approximate solution of the trust region problem by minimization over two dimensional subspaces the trust region problem minimization of a quadratic function subject to a spherical trust region constraint occurs in many optimization algorithms in a previous paper the authors introduced an inexpensive approximate solution technique for this problem that involves the solution of a two dimensional trust region problem they showed that using this approximation in an unconstrained optimization algorithm leads to the same theoretical global and local convergence properties as are obtained using the exact solution to the trust region problem this paper reports computational results showing that the two dimensional minimization approach gives nearly optimal reductions in then dimension quadratic model over a wide range of test cases we also show that there is very little difference in efficiency and reliability between using the approximate or exact trust region step in solving standard test problems for unconstrained optimization these results may encourage the application of similar approximate trust region techniques in other contexts

optimization based algorithms for tensor decompositions canonical polyadic decomposition decomposition in rank l_r l_r 1 terms and a new generalization the canonical polyadic and rank l r l r 1 block term decomposition cpd and btd respectively are two closely related tensor decompositions the cpd and recently btd are important tools in psychometrics chemometrics neuroscience and signal processing we present a decomposition that generalizes these two and develop algorithms for its computation among these algorithms are alternating least squares schemes several general unconstrained optimization techniques and matrix free nonlinear least squares methods in the latter we exploit the structure of the jacobian s gramian to reduce computational and memory cost combined with an effective preconditioner numerical experiments confirm that these methods are among the most efficient and robust currently available for computing the cpd rank l r l r 1 btd and their generalized decomposition

combining search directions using gradient flows the efficient combination of directions is a significant problem in line search methods that either use negative curvature or wish to include additional information such as the gradient or different approximations to the newton direction

a subspace interior and conjugate gradient method for large scale bound constrained minimization problems a subspace adaptation of the coleman li trust region and interior method is proposed for solving large scale bound constrained minimization problems this method can be implemented with either sparse cholesky factorization or conjugate gradient computation under reasonable conditions the convergence properties of this subspace trust region method are as strong as those of its full space version computational performance on various large test problems is reported advantages of our approach are demonstrated our experience indicates that our proposed method represents an efficient way to solve large bound constrained minimization problems

a unified approach to global convergence of trust region methods for nonsmooth optimization this paper investigates the global convergence of trust region tr methods for solving nonsmooth minimization problems for a class of nonsmooth objective functions called regular functions conditions are found on the tr local models that imply three fundamental convergence properties these conditions are shown to be satisfied by appropriate forms of fletcher s tr method for solving constrained optimization problems powell and yuan s tr method for solving nonlinear fitting problems zhang kim and lasdon s successive linear programming method for solving constrained problems duff nocedal and reid s tr method for solving systems of nonlinear equations and el hallabi and tapia s tr method for solving systems of nonlinear equations thus our results can be viewed as a unified convergence theory for tr methods for nonsmooth problems

superefficiency in blind source separation blind source separation is the problem of extracting independent signals from their mixtures without knowing the mixing coefficients nor the probability distributions of source signals and may be applied to eeg and meg imaging of the brain it is already known that certain algorithms work well for the extraction of independent components the present paper is concerned with superefficiency of these based on the statistical and dynamical analysis in a statistical estimation using t examples the covariance of any two extracted independent signals converges to 0 of the order of 1 t on line dynamics shows that the covariance is of the order of spl eta when the learning rate spl eta is fixed to a small constant in contrast with the above general properties a surprising superefficiency holds in blind source separation under certain conditions where superefficiency implies that covariance decreases in the order of 1 t sup 2 or of spl eta sup 2 the paper uses the natural gradient learning algorithm and method of estimating functions to obtain superefficient procedures for both batch estimation and on line learning a standardized estimating function is introduced to this end superefficiency does not imply that the error variances of the extracted signals decrease in the order of 1 t sup 2 or spl eta sup 2 but implies that their covariances and independencies do