an affine scaling interior trust region method for lc1 minimization subject to bounds on variables in this literature we extend the classical affine scaling interior trust region algorithm for smooth bounded constrained nonlinear programming to the nonsmooth case where the objective function is only locally lipschitzian we propose and analyze a new affine scaling trust region method in association with nonmonotonic interior backtracking line search technique for solving the linear equality constrained lc 1 optimization subject to bounds on variables at each iteration the trust region subproblem is defined by minimizing a quadratic function which is required both first and second order information of the objective function and a quadratic affine scaling matrix model subject only to an ellipsoidal constraint in a null space of the affine scaling linear equality constraints the second order derivative of the objective function is explicitly required to be lipschitzian each iterate switches to a very general setting of computing trial backtracking steps generated by the general trust region subproblem and then the line search technique projects the trial backtracking steps onto the strictly feasible set of the bounded constraints on variables the global convergence and fast local convergence rate of the proposed algorithm are established under some reasonable conditions where twice smoothness of the objective function is not required a nonmonotonic criterion should bring about speeding up the convergence progress in some ill conditioned cases applications of the algorithm to various nonsmooth optimization problems are discussed

robust mse equalizer design for mimo communication systems in the presence of model uncertainties this paper considers a robust mean square error mse equalizer design problem for multiple input multiple output mimo communication systems with imperfect channel and noise information at the receiver when the channel state information csi and the noise covariance are known exactly at the receiver a minimum mean square error mmse equalizer can be employed to estimate the transmitted signal however in actual systems it is necessary to take into account channel and noise estimation errors we consider here a worst case equalizer design problem where the goal is to find the equalizer minimizing the equalization mse for the least favorable channel model within a neighborhood of the estimated model the neighborhood is formed by placing a bound on the kullback leibler kl divergence between the actual and estimated channel models lagrangian optimization is used to convert this min max problem into a convex min min problem over a convex domain which is solved by interchanging the minimization order the robust mse equalizer and associated least favorable channel model can then be obtained by solving numerically a scalar convex minimization problem simulation results are presented to demonstrate the mse and bit error rate ber performance of robust equalizers when applied to the least favorable channel model

limited memory bfgs systems with diagonal updates abstract in this paper we investigate a formula to solve systems of the form b i x y where b is a limited memory bfgs quasi newton matrix and is a positive constant these types of systems arise naturally in large scale optimization such as trust region methods as well as doubly augmented lagrangian methods we show that provided a simple condition holds on b 0 and the system b i x y can be solved via a recursion formula that requires only vector inner products this formula has complexity m 2 n where m is the number of l bfgs updates and n m is the dimension of x

low temperature phase diagrams of quantum lattice systems i stability for quantum perturbations of classical systems with finitely many ground states starting from classical lattice systems ind 2 dimensions with a regular zerotemperature phase diagram involving a finite number of periodic ground states we prove that adding a small quantum perturbation and or increasing the temperature produce only smooth deformations of their phase diagrams the quantum perturbations can involve bosons or fermions and can be of infinite range but decaying exponentially fast with the size of the bonds for fermions the interactions must be given by monomials of even degree in creation and annihilation operators our methods can be applied to some anyonic systems as well our analysis is based on an extension of pirogov sinai theory to contour expansions ind 1 dimensions obtained by iteration of the duhamel formula

a new matrix free algorithm for the large scale trust region subproblem we present a new method for the large scale trust region subproblem the method is matrix free in the sense that only matrix vector products are required we recast the trust region subproblem as a parameterized eigenvalue problem and compute an optimal value for the parameter we then find the solution of the trust region subproblem from the eigenvectors associated with two of the smallest eigenvalues of the parameterized eigenvalue problem corresponding to the optimal parameter the new algorithm uses a different interpolating scheme than existing methods and introduces a unified iteration that naturally includes the so called hard case we show that the new iteration is well defined and convergent at a superlinear rate we present computational results to illustrate convergence properties and robustness of the method

effective hamiltonians and phase diagrams for tight binding models we present rigorous results for several variants of the hubbard model in the strong coupling regime we establish a mathematically controlled perturbation expansion which shows how previously proposed effective interactions are in fact leading order terms of well defined volume independent unitarily equivalent interactions in addition in the very asymmetric falicov kimball regime we are able to apply recently developed phase diagram technology quantum pirogov sinai theory to conclude that the zero temperature phase diagrams obtained for the leading classical part remain valid except for thin excluded regions and small deformations for the full fledged quantum interaction at zero or low temperature moreover the phase diagram is stable against addition of arbitrary but sufficiently small further quantum terms that do not break the ground state symmetries this generalizes and unifies a number of previous results on the subject in particular published results on the zero temperature phase diagram of the falikov kimball model with and without magnetic flux are extended to small temperatures and or small ionic hopping we give explicit expressions for the first few orders in the hopping amplitude of equivalent interactions and we describe the resulting phase diagram our approach yields algorithms to compute equivalent interactions to arbitrarily high order in the hopping amplitude

backward step control for global newton type methods we present and analyze a new damping approach called backward step control for the globalization of the convergence of newton type methods for the numerical solution of nonlinear root finding problems we provide and discuss reasonable assumptions that imply convergence of backward step control on the basis of generalized newton paths in conjunction with a backward analysis argument in particular convergence to a specific solution and a priori estimates on the residual reduction can be shown furthermore we can guarantee a transition to full steps in the vicinity of a solution which implies fast local convergence we present two algorithmic realizations of backward step control and apply the method to more than one hundred examples including comparisons with different globalization approaches for the minimization of the rosenbrock function and to large scale unconstrained optimization problems from the cutest benchmark library using backward step control for an inexact newton method based on minres

optimality condition and complexity analysis for linearly constrained optimization without differentiability on the boundary in this paper we consider the minimization of a continuous function that is potentially not differentiable or not twice differentiable on the boundary of the feasible region by exploiting an interior point technique we present first and second order optimality conditions for this problem that reduces to classical ones when the derivative on the boundary is available for this type of problems existing necessary conditions often rely on the notion of subdifferential or become non trivially weaker than the kkt condition in the twice differentiable counterpart problems in contrast this paper presents a new set of first and second order necessary conditions that are derived without the use of subdifferential and reduce to exactly the kkt condition when twice differentiability holds as a result these conditions are stronger than some existing ones considered for the discussed minimization problem when only non negativity constraints are present to solve for these optimality conditions in the special but important case of linearly constrained problems we present two novel interior point trust region algorithms and show that their worst case computational efficiency in achieving the potentially stronger optimality conditions match the best known complexity bounds since this work considers a more general problem than those in the literature our results also indicate that best known existing complexity bounds are actually held for a wider class of nonlinear programming problems this new development is significant since optimality conditions play a fundamental role in computational optimization and more and more nonlinear and nonconvex problems need to be solved in practice

on affine scaling algorithms for nonconvex quadratic programming we investigate the use of interior algorithms especially the affine scaling algorithm to solve nonconvex indefinite or negative definite quadratic programming qp problems although the nonconvex qp with a polytope constraint is a hard problem we show that the problem with an ellipsoidal constraint is easy when the hard qp is solved by successively solving the easy qp the sequence of points monotonically converge to a feasible point satisfying both the first and the second order optimality conditions

a subspace minimization method for the trust region step we consider methods for large scale unconstrained minimization based on finding an approximate minimizer of a quadratic function subject to a two norm trust region constraint the steihaug toint method uses the conjugate gradient method to minimize the quadratic over a sequence of expanding subspaces until the iterates either converge to an interior point or cross the constraint boundary however if the conjugate gradient method is used with a preconditioner the steihaug toint method requires that the trust region norm be defined in terms of the preconditioning matrix if a different preconditioner is used for each subproblem the shape of the trust region can change substantially from one subproblem to the next which invalidates many of the assumptions on which standard methods for adjusting the trust region radius are based in this paper we propose a method that allows the trust region norm to be defined independently of the preconditioner the method solves the inequality constrained trust region subproblem over a sequence of evolving low dimensional subspaces each subspace includes an accelerator direction defined by a regularized newton method for satisfying the optimality conditions of a primal dual interior method a crucial property of this direction is that it can be computed by applying the preconditioned conjugate gradient method to a positive definite system in both the primal and dual variables of the trust region subproblem numerical experiments on problems from the cuter test collection indicate that the method can require significantly fewer function evaluations than other methods in addition experiments with general purpose preconditioners show that it is possible to significantly reduce the number of matrix vector products relative to those required without preconditioning

strong duality in nonconvex quadratic optimization with two quadratic constraints we consider the problem of minimizing an indefinite quadratic function subject to two quadratic inequality constraints when the problem is defined over the complex plane we show that strong duality holds and obtain necessary and sufficient optimality conditions we then develop a connection between the image of the real and complex spaces under a quadratic mapping which together with the results in the complex case lead to a condition that ensures strong duality in the real setting preliminary numerical simulations suggest that for random instances of the extended trust region subproblem the sufficient condition is satisfied with a high probability furthermore we show that the sufficient condition is always satisfied in two classes of nonconvex quadratic problems finally we discuss an application of our results to robust least squares problems

an approximation scheme for distributionally robust nonlinear optimization we consider distributionally robust optimization problems drops with nonlinear and nonconcave dependence on uncertain parameters the drop can be written as a nonsmooth nonlinear program with a

robustness of trajectories with finite time extent the problem of estimating perturbation bounds of finite trajectories is considered the trajectory is assumed to be generated by a linear system with uncertainty characterized in terms of integral quadratic constraints it is shown that such perturbation bounds can be obtained as the solution to a nonconvex quadratic optimization problem which can be addressed using lagrange relaxation the result can be used in robustness analysis of hybrid systems and switched dynamical systems

on the convergence theory of trust region based algorithms for equality constrained optimization in a recent paper dennis el alem and maciel proved global convergence to a stationary point for a general trust region based algorithm for equality constrained optimization this general algorithm is based on appropriate choices of trust region subproblems and seems particularly suitable for large problems this paper shows global convergence to a point satisfying the second order necessary optimality conditions for the same general trust region based algorithm the results given here can be seen as a generalization of the convergence results for trust regions methods for unconstrained optimization obtained by more and sorensen the behavior of the trust radius and the local rate of convergence are analyzed some interesting facts concerning the trust region subproblem for the linearized constraints the quasi normal component of the step and the hard case are presented it is shown how these results can be applied to a class of discretized optimal control problems

there are no c1 stable intersections of regular cantor sets we prove that there are no stable intersections of regular cantor sets in the c 1 topology given any pair k k of regular cantor sets we can find arbitrarily close to it in the c 1 topology pairs left tilde k tilde k right of regular cantor sets with tilde k cap tilde k emptyset

solving large nonlinear systems of equations by an adaptive condensation process we present an algorithm which efficiently solves large nonlinear systems of the form au f u u in mathbb r n whenever an iterative solver a 1 for the symmetric positive definite matrixa is available andf u is symmetric such problems arise from the discretization of nonlinear elliptic partial differential equations by means of an adaptive decomposition process we split the original system into a low dimensional system to be treated by any sophisticated solver and a remaining high dimensional system which can easily be solved by fixed point iteration specifically we choose a newton type trust region algorithm for the treatment of the small system we show global convergence under natural assumptions on the nonlinearity the convergence results typical for trust region algorithms carry over to the full iteration process the only large systems to be solved are linear ones with the fixed matrixa thus existing software for positive definite sparse linear systems can be used

a trust region strategy for minimization on arbitrary domains we present a trust region method for minimizing a general differentiable function restricted to an arbitrary closed set we prove a global convergence theorem the trust region method defines difficult subproblems that are solvable in some particular cases we analyze in detail the case where the domain is a euclidean ball for this case we present numerical experiments where we consider different hessian approximations

optimization of unconstrained functions with sparse hessian matrices newton type methods newton type methods for unconstrained optimization problems have been very successful when coupled with a modified cholesky factorization to take into account the possible lack of positive definiteness in the hessian matrix in this paper we discuss the application of these method to large problems that have a sparse hessian matrix whose sparsity is known a priori

on the convergence of a new trust region algorithm in this paper we present a new trust region algorithm for general nonlinear constrained optimization problems the algorithm is based on the l infty exact penalty function under very mild conditions global convergence results for the algorithm are given local convergence properties are also studied it is shown that the penalty parameter generated by the algorithm will be eventually not less than the l 1 norm of the lagrange multipliers at the accumulation point it is proved that the method is equivalent to the sequential quadratic programming method for all large k hence superlinearly convergent results of the sqp method can be applied numerical results are also reported

characterization of the smoothness and curvature of a marginal function for a trust region problem this paper studies the smoothness and curvature of a marginal function for a trust region problem in this problem a quadratic function is minimized over an ellipsoid the marginal function considered is obtained by perturbing the trust radius i e by changing the size of the ellipsoidal constraint the values of the marginal function and of its first and second derivatives are explicitly calculated in all possible scenarios a complete study of the smoothness and curvature of this marginal function is given the main motivation for this work arises from an application in statistics

object pose from 2 d to 3 d point and line correspondences in this paper we present a method for optimally estimating the rotation and translation between a camera and a 3 d object from point and or line correspondences first we devise an error function and second we show how to minimize this error function the quadratic nature of this function is made possible by representing rotation and translation with a dual number quaternion we provide a detailed account of the computational aspects of a trust region optimization method this method compares favourably with newton s method which has extensively been used to solve the problem at hand with faugeras toscani s linear method faugeras and toscani 1986 for calibrating a camera and with the levenberg marquardt non linear optimization method finally we present some experimental results which demonstrate the robustness of our method with respect to image noise and matching errors

a modified nearly exact method for solving low rank trust region subproblem in this paper we first discuss how the nearly exact ne method proposed by more and sorensen 14 for solving trust region tr subproblems can be modified to solve large scale low rank tr subproblems efficiently our modified algorithm completely avoids computation of cholesky factorizations by instead relying primarily on the sherman morrison woodbury formula for computing inverses of diagonal plus low rank type matrices we also implement a specific version of the modified log barrier mlb algorithm proposed by polyak 17 where the generated log barrier subproblems are solved by a trust region method the corresponding direction finding tr subproblems are of the low rank type and are then solved by our modified ne method we finally discuss the computational results of our implementation of the mlb method and its comparison with a version of lancelot 5 based on a collection extracted from cuter 12 of nonlinear programming problems with simple bound constraints

deriving collinear scaling algorithms as extension of quasi newton methods and the local convergence of dfp and bfgs related collinear scaling algorithms this paper is concerned with collinear scaling algorithms for unconstrained minimization where the underlying local approximants are forced to interpolate the objective function value and gradient at only the two most recent iterates by suitably modifying the procedure of sorensen 1980 for deriving such algorithms we show that two members of the algorithm class derived related to the dfp and bfgs methods respectively are locally and q superlinearly convergent this local analysis as well as the results they yield exhibit the same sort of duality exhibited by those of broyden dennis and more 1973 and dennis and more 1974 for the dfp and bfgs methods the results in this paper also imply the local and q superlinear convergence of collinear scaling algorithms of sorensen 1982 pp 154 156 related to the dfp and bfgs methods

optimality conditions for trust region subproblems involving a conic model the problem of minimizing a conic function subject to an ellipsoidal constraint is discussed in this paper the feasible set of this problem is divided into three different cases necessary and sufficient optimality conditions are obtained for different cases where there is a characterization of optimality for the basic case these conditions provide a sound basis for the further development of efficient methods for trust region subproblems of a conic model

quantum lattice models at intermediate temperature we analyze the free energy and construct the gibbs kms states for a class of quantum lattice systems at low temperature and when the interactions are almost diagonal in a suitable basis the models we study may have continuous symmetries our results however apply to intermediate temperatures where discrete symmetries are broken but continuous symmetries are not our results are based on quantum pirogov sinai theory and a combination of high and low temperature expansions

computing a celis dennis tapia trust region step for equality constrained optimization we study an approach for minimizing a convex quadratic function subject to two quadratic constraints this problem stems from computing a trust region step for an sqp algorithm proposed by celis dennis and tapia 1985 for equality constrained optimization our approach is to reformulate the problem into a univariate nonlinear equation 0 where the function is continuous at least piecewise differentiable and monotone well established methods then can be readily applied we also consider an extension of our approach to a class of non convex quadratic functions and show that our approach is applicable to reduced hessian sqp algorithms numerical results are presented indicating that our algorithm is reliable robust and has the potential to be used as a building block to construct trust region algorithms for small sized problems in constrained optimization

globally solving the trust region subproblem using simple first order methods we consider the trust region subproblem which is given by a minimization of a quadratic not necessarily convex function over the euclidean ball based on the well known second order necessary and

on the complexity of approximating a kkt point of quadratic programming we present a potential reduction algorithm to approximate a karush kuhn tucker kkt point of general quadratic programming qp we show that the algorithm is a fully polynomial time approximation scheme and its running time dependency on accuracy 0 1 is o l log l log log l compared to the previously best known result o l 2 furthermore the limit of the kkt point satisfies the second order necessary optimality condition of being a local minimizer 1998 the mathematical programming society inc published by elsevier science b v

phase diagram of the two dimensional t t falicov kimball model the ground state phase diagram of the two dimensional falicov kimball model with nearest neighbour and next nearest neighbour hoppings has been studied in the perturbative regime where hoppings are small compared with the on site coulomb interaction the phase diagram at fourth order exhibits a richer structure than the one of the ordinary falicov kimball model

analysis of a quadratic programming decomposition algorithm we analyze a decomposition algorithm for minimizing a quadratic objective function separable in mathbf x 1 and mathbf x 2 subject to the constraint that mathbf x 1 and mathbf x 2 are orthogonal vectors on the unit sphere our algorithm consists of a local step where we minimize the objective function in either variable separately while enforcing the constraints followed by a global step where we minimize over a subspace generated by solutions to the local subproblems we establish a local convergence result when the global minimizers are nondegenerate our analysis employs necessary and sufficient conditions and continuity properties for a global optimum of a quadratic objective function subject to a sphere constraint and a linear constraint the analysis is connected with a new domain decomposition algorithm for electronic structure calculations

a class of collinear scaling algorithms for bound constrained optimization derivation and computational results a family of algorithms for the approximate solution of the bound constrained minimization problem is described these algorithms employ the standard barrier method with the inner iteration based on trust region methods local models are conic functions rather than the usual quadratic functions and are required to match first and second derivatives of the barrier function at the current iterate the various members of the family are distinguished by the choice of a vector valued parameter which is the zero vector in the degenerate case that quadratic local models are used computational results are used to compare the efficiency of various members of the family on a selection of test functions

quasi newton trust region algorithm for non smooth least squares problems1this work was supported by cnpq of brazil and the national natural science foundation of china 1 in this paper we present a quasi newton trust region algorithm for non smooth least squares problems which possess the smooth plus non smooth decomposition this method uses a smooth subproblem with second order information to approximate the locally lipschitzian function which is important to increase the convergence rate of the algorithm and deal with the large residual non smooth least squares problems the global and superlinear convergence of this algorithm are established

rss based sensor localization in the presence of unknown channel parameters this correspondence studies the received signal strength based localization problem when the transmit power or path loss exponent is unknown the corresponding maximum likelihood estimator mle poses a difficult nonconvex optimization problem to avoid the difficulty in solving the mle we use suitable approximations and formulate the localization problem as a general trust region subproblem which can be solved exactly under mild conditions simulation results show a promising performance for the proposed methods which also have reasonable complexities compared to existing approaches

separating multipaths by global optimization of a multidimensional matched filter a transmitted signal can arrive at a receiver via several refracted fermat paths if the paths are independent in the fresnel sense then the received signal can be modeled as the sum of amplitude scaled and time shifted copies of a predetermined replica plus white noise we present an algorithm that uses the replica to determine the time shifts and amplitudes for each path it is referred to as an n dimensional matched filter algorithm by analogy with the well known matched filter algorithm the cross correlation between the received signal and the replica oscillates near the center frequency of the transmitted signal this causes the n dimensional matched filter output to have many local maxima that are not globally optimal the time shifts and amplitude scalings for the fermat paths are determined by maximizing the output of the n dimensional matched filter the algorithm is more robust and efficient than others currently available simulated realizations of received signals were generated with multipath and noise characteristics similar to an ocean acoustic transmission case these realizations were then separated into arrival times and corresponding amplitudes by the algorithm the results of these tests and the general limitations of the algorithm are discussed

newton s method with a model trust region modification a modified newton method for unconstrained minimization is presented and analyzed the modification is based upon the model trust region approach this report contains a thorough analysis of the locally constrained quadratic minimizations that arise as subproblems in the modified newton iteration several promising alternatives are presented for solving these subproblems in ways that overcome certain theoretical difficulties exposed by this analysis very strong convergence results are presented concerning the minimization algorithm in particular the explicit use of second order information is justified by demonstrating that the iterates converge to a point which satisfies the second order necessary conditions for minimization with the exception of very pathological cases this occurs whenever the algorithm is applied to problems with continuous second partial derivatives

stability of linear quadratic minimization over euclidean balls stability properties of the problem of minimizing a nonconvex linear quadratic function over a euclidean ball known as the trust region subproblem are studied in this paper we investigate in detail the case where the linear part of the objective function is perturbed and obtain necessary and sufficient conditions for the upper lower semicontinuity of the karush kuhn tucker kkt point set map and the global solution map explicit formulas for computing the directional derivative and the frechet derivative of the optimal value function stability of the karush kuhn tucker point set under the perturbation of the quadratic form is also studied

minimization of a large scale quadratic function subject to a spherical constraint an important problem in linear algebra and optimization is the trust region subproblem minimize a quadratic function subject to an ellipsoidal or spherical constraint this basic problem has several important large scale applications including seismic inversion and forcing convergence in optimization methods existing methods to solve the trust region subproblem require matrix factorizations which are not feasible in the large scale setting this paper presents an algorithm for solving the large scale trust region subproblem that requires a fixed size limited storage proportional to the order of the quadratic and that relies only on matrix vector products the algorithm recasts the trust region subproblem in terms of a parameterized eigenvalue problem and adjusts the parameter with a superlinearly convergent iteration to find the optimal solution from the eigenvector of the parameterized problem only the smallest eigenvalue and corresponding eigenvector of the parameterized problem needs to be computed the implicitly restarted lanczos method is well suited to this subproblem

a geometric characterization of strong duality in nonconvex quadratic programming with linear and nonconvex quadratic constraints we first establish a relaxed version of dines theorem associated to quadratic minimization problems with finitely many linear equality and a single nonconvex quadratic inequality constraints the case of unbounded optimal valued is also discussed then we characterize geometrically the strong duality and some relationships with the conditions employed in finsler theorem are established furthermore necessary and sufficient optimality conditions with or without the slater assumption are derived our results can be used to situations where none of the results appearing elsewhere are applicable in addition a revisited theorem due to frank and wolfe along with that due to eaves is established for asymptotically linear sets

trust region methods for structural optimization using exact second order sensitivity the performance of multiplier algorithms for structural optimization has been significantly improved by using trust regions the trust regions are constructed using analytical second order sensitivity and within this region the augmented lagrangian is minimized subject to bounds evaluation of first and second derivatives of by the adjoint method does not require derivations of individual implicit constraint functions which makes the method economical eight test problems are considered and a vast improvement over previously used multiplier algorithms has been noted also the algorithm is robust with respect to scaling input parameters and starting designs

a class of indefinite dogleg path methods for unconstrained minimization in this paper we propose a convenient curvilinear search method to solve trust region problems arising from unconstrained optimization problems the curvilinear paths we set forth are dogleg paths generated mainly by employing bunch parlett factorization for general symmetric matrices that may be indefinite this method is easy to implement and globally convergent it is proved that the method satisfies the first and second order stationary point convergence properties and that the convergence rate is quadratic under commonly used conditions on functions numerical experiments are conducted to compare this method with some existing methods

a trust region method for norm constrained problems a trust region method for the solution of nonlinear optimization problems with norm constraints is analyzed such optimization problems arise in parameter identification and nonlinear eigenvalue problems the characteristics of these applications motivate the formulation and analysis of the trust region method the algorithms studied here allow for inexact gradient information and the use of subspace methods for the approximate solution of subproblems characterizations and the descent properties of trust region steps are given criteria for the existence of successful iterations under inexact gradient information and under the use of subspace methods are established and global convergence of the method is proven

a class of collinear scaling algorithms for bound constrained optimization convergence theorems a family of algorithms for approximate solution of the bound constrained minimization problem was introduced in k a ariyawansa w l tabor a class of collinear scaling algorithms for bound constrained optimization derivation and computational results technical report 2003 1 department of mathematics washington state university pullman wa 2003 submitted for publication available at http www math wsu edu math trs 2003 1 pdf these algorithms employ the standard barrier method with the inner iteration based on trust region methods local models are conic functions rather than the usual quadratic functions and are required to match first and second derivatives of the barrier function at the current iterate the various members of the family are distinguished by the choice of a vector valued parameter which is the zero vector in the degenerate case that quadratic local models are used this paper presents a convergence analysis of the family of algorithms presented in k a ariyawansa w l tabor a class of collinear scaling algorithms for bound constrained optimization derivation and computational results technical report 2003 1 department of mathematics washington state university pullman wa 2003 submitted for publication available at http www math wsu edu math trs 2003 1 pdf specifically convergence properties similar to those of barrier methods using quadratic local models are established

tractable bayesian variable selection beyond normality abstractbayesian variable selection often assumes normality but the effects of model misspecification are not sufficiently understood there are sound reasons behind this assumption particularly for large p ease of interpretation analytical and computational convenience more flexible frameworks exist including semi or non parametric models often at the cost of some tractability we propose a simple extension that allows for skewness and thicker than normal tails but preserves tractability it leads to easy interpretation and a log concave likelihood that facilitates optimization and integration we characterize asymptotically parameter estimation and bayes factor rates under certain model misspecification under suitable conditions misspecified bayes factors induce sparsity at the same rates than under the correct model however the rates to detect signal change by an exponential factor often reducing sensitivity these deficiencies can be ameliorated by inferring the error distribution a simp

a subspace implementation of quasi newton trust region methods for unconstrained optimization this paper studies subspace properties of trust region methods for unconstrained optimization assuming the approximate hessian is updated by quasi newton formulae and the initial hessian approximation is appropriately chosen it is shown that the trial step obtained by solving the trust region subproblem is in the subspace spanned by all the gradient vectors computed thus the trial step can be defined by minimizing the quasi newton quadratic model in the subspace based on this observation some subspace trust region algorithms are proposed and numerical results are also reported

a semidefinite framework for trust region subproblems with applications to large scale minimization primal dual pairs of semidefinite programs provide a general framework for the theory and algorithms for the trust region subproblem trs this latter problem consists in minimizing a general quadratic function subject to a convex quadratic constraint and therefore it is a generalization of the minimum eigenvalue problem the importance of trs is due to the fact that it provides the step in trust region minimization algorithms the semidefinite framework is studied as an interesting instance of semidefinite programming as well as a tool for viewing known algorithms and deriving new algorithms for trs in particular a dual simplex type method is studied that solves trs as a parametric eigenvalue problem this method uses the lanczos algorithm for the smallest eigenvalue as a black box therefore the essential cost of the algorithm is the matrix vector multiplication and thus sparsity can be exploited a primal simplex type method provides steps for the so called hard case extensive numerical tests for large sparse problems are discussed these tests show that the cost of the algorithm is 1 n times the cost of finding a minimum eigenvalue using the lanczos algorithm where 0 n 1 is a fraction which decreases as the dimension increases

duality and sensitivity in nonconvex quadratic optimization over an ellipsoid abstract in this paper a duality framework is discussed for the problem of optimizing a nonconvex quadratic function over an ellipsoid additional insight is obtained from the observation that this nonconvex problem is in a sense equivalent to a convex problem of the same type from which known necessary and sufficient conditions for optimality readily follow based on the duality results some existing solution procedures are interpreted as in fact solving the dual the duality relations are also shown to provide a natural framework for sensitivity analysis

on some properties of quadratic programs with a convex quadratic constraint in this paper we consider the problem of minimizing a possibly nonconvex quadratic function with a quadratic constraint we point out some new properties of the problem in particular in the first part of the paper we show that i given a kkt point that is not a global minimizer it is easy to find a better feasible point ii strict complementarity holds at the local nonglobal minimizer in the second part of this paper we show that the original constrained problem is equivalent to the unconstrained minimization of a piecewise quartic merit function using the unconstrained formulation we give in the nonconvex case a new second order necessary condition for global minimizers in the third part of this paper algorithmic applications of the preceding results are briefly outlined and some preliminary numerical experiments are reported

a global convergence theory for general trust region based algorithms for equality constrained optimization this work presents a global convergence theory for a broad class of trust region algorithms for the smooth nonlinear programming problem with equality constraints the main result generalizes powell s 1975 result for unconstrained trust region algorithms the trial step is characterized by very mild conditions on its normal and tangential components the normal component need not be computed accurately the theory requires a quasi normal component to satisfy a fraction of cauchy decrease condition on the quadratic model of the linearized constraints the tangential component then must satisfy a fraction of cauchy decrease condition on a quadratic model of the lagrangian function in the translated tangent space of the constraints determined by the quasi normal component estimates of the lagrange multipliers and the hessians are assumed only to be bounded the other main characteristic of this class of algorithms is that the step is evaluated by using the augmented lagrangian as a merit function with the penalty parameter updated using the el alem scheme the properties of the step and the way that the penalty parameter is chosen are sufficient to establish global convergence as an example an algorithm is presented that can be viewed as a generalization of the steihaug toint dogleg algorithm for the unconstrained case it is based on a quadratic programming algorithm that uses a step in a quasi normal direction to the tangent space of the constraints and then takes feasible conjugate reduced gradient steps to solve the reduced quadratic program this algorithm should cope quite well with large problems for which effective preconditioners are known

tikhonov regularization with a solution constraint many numerical methods for the solution of linear ill posed problems apply tikhonov regularization this paper presents a modification of a numerical method proposed by golub and von matt for quadratically constrained least squares problems and applies it to tikhonov regularization of large scale linear discrete ill posed problems the method is based on partial lanczos bidiagonalization and gauss quadrature computed examples illustrating its performance are presented

estimations of the free energy for the hubbard model a series of upper bounds and two lower bounds for the partition function of the hubbard model have been derived these bounds are expressible by certain properties of the falicov kimball model the upper bounds have been derived mainly by the use of the golden thompson inequality and its generalizations and the holder inequality the lower bounds are based on the bogoliubov peierls inequality the numerical values of bounds have also been calculated for small systems

variational analysis of an extended eigenvalue problem abstract for a symmetric matrix b rn n and a vector a rn the maximal extended eigenvalue a max x r n s t b i x a x t x 1 is known to arise in optimality conditions for the mathematical programming problemp a given by max x t b x 2 a t x x t x 1 as well as in extended rayleigh ritz type results pertaining to the one parameter family of nonsymmetric border perturbations ofb given by a t a b a t a t nonsmooth analysis is employed in order to describe the function with special emphasis on its sensitivity near the origin further connections with p a are drawn

an interior affine scaling projective algorithm for nonlinear equality and linear inequality constrained optimization in this paper we propose a new nonmonotonic interior point backtracking strategy to modify the reduced projective affine scaling trust region algorithm for solving optimization subject to nonlinear equality and linear inequality constraints the general full trust region subproblem for solving the nonlinear equality and linear inequality constrained optimization is decomposed to a pair of trust region subproblems in horizontal and vertical subspaces of linearize equality constraints and extended affine scaling equality constraints the horizontal subproblem in the proposed algorithm is defined by minimizing a quadratic projective reduced hessian function subject only to an ellipsoidal trust region constraint in a null subspace of the tangential space while the vertical subproblem is also defined by the least squares subproblem subject only to an ellipsoidal trust region constraint by introducing the fletcher s penalty function as the merit function trust region strategy with interior point backtracking technique will switch to strictly feasible interior point step generated by a component direction of the two trust region subproblems the global convergence of the proposed algorithm while maintaining fast local convergence rate of the proposed algorithm are established under some reasonable conditions a nonmonotonic criterion should bring about speeding up the convergence progress in some high nonlinear function conditioned cases

segregation in the asymmetric hubbard model we study the asymmetric hubbard model where hoppings of electrons depend on their spin for strong interactions and sufficiently asymmetric hoppings it is proved that the ground state displays phase separation away from half filling this extends a recent result obtained with freericks and lieb for the falicov kimball model it is based on estimates for the sum of lowest eigenvalues of the discrete laplacian in arbitrary domains

unitary dressing transformations and exponential decay below threshold for quantum spin systems parts i and ii this is the continuation of a series of articles concerning a class of quantum spin systems with hamiltonian operators of the form h lambda sum limits x in lambda s x sum limits yo subset lambda lambda left yo right c 1 t yo where is a graph is a small parameter andsx has a gap 1 for all x e i in the singular seti the gap ofsx can be arbitrarily small part iii is devoted to the proof of a preliminary result while in part iv we consider the case in whichi is a subset of finite density of this completes the first iteration step of the deterministic part of the proof of localization in the ground state of the random field quantumxy model

line integral alternating minimization algorithm for dual energy x ray ct image reconstruction we propose a new algorithm called line integral alternating minimization liam for dual energy x ray ct image reconstruction instead of obtaining component images by minimizing the discrepancy between the data and the mean estimates liam allows for a tunable discrepancy between the basis material projections and the basis sinograms a parameter is introduced that controls the size of this discrepancy and with this parameter the new algorithm can continuously go from a two step approach to the joint estimation approach liam alternates between iteratively updating the line integrals of the component images and reconstruction of the component images using an image iterative deblurring algorithm an edge preserving penalty function can be incorporated in the iterative deblurring step to decrease the roughness in component images images from both simulated and experimentally acquired sinograms from a clinical scanner were reconstructed by liam while varying the regularization parameters to identify good choices the results from the dual energy alternating minimization algorithm applied to the same data were used for comparison using a small fraction of the computation time of dual energy alternating minimization liam achieves better accuracy of the component images in the presence of poisson noise for simulated data reconstruction and achieves the same level of accuracy for real data reconstruction

an approach to identification for robust control given measured data generated by a discrete time linear system we propose a model consisting of a linear time invariant system affected by norm bounded perturbation under mild assumptions the plants belonging to the resulting uncertain family form a convex set the approach depends on two key parameters an a priori given bound of the perturbation and the input used to generate the data it turns out that the size of the uncertain family can be reduced by intersecting the model families obtained by making use of different inputs the model validation problem in this identification scheme is analyzed for a given energy level the invalidation problem yields the family of those models which can never be invalidated for any possible input of fixed energy and any possible perturbation this leads to the intersection of all uncertain families a consequence of the invalidation problem is that for finite length measurements not all models can be invalidated using fixed energy inputs

an affine scaling projective reduced hessian algorithm for minimum optimization with nonlinear equality and linear inequality constraints abstract in this paper we propose a nonmonotonic interior point backtracking strategy to modify the reduced projective affine scaling trust region algorithm for solving minimum optimization subject to both nonlinear equality and linear inequality constraints the general full trust region subproblem for solving the minimum optimization is decomposed to a pair of trust region subproblems in horizontal and vertical subspaces of linearize equality constraints and extended affine scaling equality constraints by qr decomposition of an affine scaling matrix and an orthonormal basis on the null subspace the horizontal subproblem in the proposed algorithm is defined by minimizing a quadratic projective reduced hessian function subject only to an ellipsoidal trust region constraint while the vertical subproblem is also defined by the least squares subproblem subject only to an ellipsoidal trust region constraint combining trust region strategy with line search technique will switch to strictly feasible interior point step generated by a component direction of the two trust region subproblems by adopting the l1 penalty function as the merit function the global convergence and fast local convergence rate of the proposed algorithm are established under some reasonable conditions the second order correction step and a nonmonotonic criterion are used to overcome maratos effect and speed up the convergence progress in some ill conditioned cases respectively

a mathematical biography of danny c sorensen abstract on the occasion of his 65th birthday we briefly recount dan sorensen s profound contributions to optimization numerical linear algebra and model order reduction for dynamical systems

a trust region approach to the regularization of large scale discrete forms of ill posed problems we consider large scale least squares problems where the coefficient matrix comes from the discretization of an operator in an ill posed problem and the right hand side contains noise special techniques known as regularization methods are needed to treat these problems in order to control the effect of the noise on the solution we pose the regularization problem as a quadratically constrained least squares problem this formulation is equivalent to tikhonov regularization and we note that it is also a special case of the trust region subproblem from optimization we analyze the trust region subproblem in the regularization case and we consider the nontrivial extensions of a recently developed method for general large scale subproblems that will allow us to handle this case the method relies on matrix vector products only has low and fixed storage requirements and can handle the singularities arising in ill posed problems we present numerical results on test problems on an inverse interpolation problem with field data and on a model seismic inversion problem with field data

approximate solution of the trust region problem by minimization over two dimensional subspaces the trust region problem minimization of a quadratic function subject to a spherical trust region constraint occurs in many optimization algorithms in a previous paper the authors introduced an inexpensive approximate solution technique for this problem that involves the solution of a two dimensional trust region problem they showed that using this approximation in an unconstrained optimization algorithm leads to the same theoretical global and local convergence properties as are obtained using the exact solution to the trust region problem this paper reports computational results showing that the two dimensional minimization approach gives nearly optimal reductions in then dimension quadratic model over a wide range of test cases we also show that there is very little difference in efficiency and reliability between using the approximate or exact trust region step in solving standard test problems for unconstrained optimization these results may encourage the application of similar approximate trust region techniques in other contexts

a new affine scaling interior point algorithm for nonlinear optimization subject to linear equality and inequality constraints in this paper we propose a new interior affine scaling region algorithm with nonmonotonic interior point backtracking technique for nonlinear optimization subject to linear equality and inequality constraints the trust region subproblem in the proposed algorithm is defined by minimizing a quadratic function subject only to an affine scaling ellipsoidal constraint in a null subspace of the extended equality constraints using both trust region strategy and line search technique the affine scaling trust region subproblem at each iteration generates backtracking interior step to obtain a new accepted step the global convergence and fast local convergence rate of the proposed algorithm are established under some reasonable conditions a nonmonotonic criterion should bring about speeding up the convergence progress in some ill conditioned cases

convergence to second order stationary points in inequality constrained optimization we propose a new algorithm for the nonlinear inequality constrained minimization problem and prove that it generates a sequence converging to points satisfying the kkt second order necessary conditions for optimality the algorithm is a line search algorithm using directions of negative curvature and it can be viewed as a nontrivial extension of corresponding known techniques from unconstrained to constrained problems the main tools employed in the definition and in the analysis of the algorithm are a differentiable exact penalty function and results from the theory of lc1 functions

degenerate behavior in non hyperbolic semigroup actions on the interval fast growth of periodic points and universal dynamics we consider semigroup actions on the unit interval generated by strictly increasing c r maps we assume that one of the generators has a pair of fixed points one attracting and one repelling and a heteroclinic orbit that connects the repeller and attractor we also assume that the other generators form a robust blender which can bring the points from a small neighborhood of the attractor to an arbitrarily small neighborhood of the repeller this is a model setting for partially hyperbolic systems with one central direction we show that under additional conditions on frac f f and the schwarzian derivative the above semigroups exhibit c r generically for any r ge 3 arbitrarily fast growth of the number of periodic points as a function of the period we also show that a c r generic semigroup from the class under consideration supports an ultimately complicated behavior called universal dynamics

on the global convergence of trust region algorithms using inexact gradient information trust region algorithms are an important class of methods that can be used to solve unconstrained optimization problems strong global convergence results are demonstrated for a class of methods where the gradient values are approximated rather than computed exactly provided they obey a simple relative error condition no requirement is made that gradients be recomputed to successively greater accuracy after unsuccessful iterations

influence of anisotropic next nearest neighbour hopping on diagonal charge striped phases we consider the model of a strongly correlated system of electrons on a two dimensional square lattice described by an extended falicov kimball hamiltonian where the stability of some axial and diagonal striped phases was proved introducing a next nearest neighbour hopping small enough not to destroy the striped structure we examine rigorously how the presence of the next nearest neighbour hopping anisotropy reduces the 2 rotation degeneracy of the diagonal striped phase the effect appears to be similar to that in the case of anisotropy of the nearest neighbour hopping diagonal stripes are oriented in the direction of the weaker next nearest neighbour hopping

rigidity for c 1 actions on the interval arising from hyperbolicity i solvable groups we consider abelian by cyclic groups for which the cyclic factor acts by hyperbolic automorphisms on the abelian subgroup we show that if such a group acts faithfully by c 1 diffeomorphisms of the closed interval with no global fixed point at the interior then the action is topologically conjugate to that of an affine group moreover in case of non abelian image we show a rigidity result concerning the multipliers of the homotheties despite the fact that the conjugacy is not necessarily smooth some consequences for non solvable groups are proposed in particular we give new proofs examples yielding the existence of finitely generated locally indicable groups with no faithful action by c 1 diffeomorphisms of the interval

on piecewise quadratic newton and trust region problems some recent algorithms for nonsmooth optimization require solutions to certain piecewise quadratic programming subproblems two types of subproblems are considered in this paper the first type seeks the minimization of a continuously differentiable and strictly convex piecewise quadratic function subject to linear equality constraints we prove that a nonsmooth version of newton s method is globally and finitely convergent in this case the second type involves the minimization of a possibly nonconvex and nondifferentiable piecewise quadratic function over a euclidean ball characterizations of the global minimizer are studied under various conditions the results extend a classical result on the trust region problem

an affine scaling reduced preconditional conjugate gradient path method for linear constrained optimization abstract this paper presents an affine scaling reduced preconditional conjugate gradient path approach in association with nonmonotonic interior backtracking line search technique for the linear constrained optimization employing the affine scaling preconditional conjugate gradient to form the curvilinear path and using interior backtracking line search technique each iterate switches to trial step of strict interior feasibility the nonmonotone criterion is used to speed up the convergence progress in the contours of objective function with large curvature theoretical analysis are given which prove that the proposed algorithm is globally convergent and has a local superlinear convergence rate under some reasonable conditions

sur la dynamique unidimensionnelle en regularite intermediaire using probabilistic methods we prove new rigidity results for groups and pseudo groups of diffeomorphisms of one dimensional manifolds with intermediate regularity class i e between c 1 and c 2 in particular we show some generalizations of denjoy theorem and the classical kopell lemma for abelian groups these techniques are also applied to the study of codimension 1 foliations for instance we obtain several generalized versions of sacksteder theorem in class c 1 we conclude with some remarks about the stationary measure

a second order algorithm for continuous time nonlinear optimal control problems a second order algorithm is presented for the solution of continuous time nonlinear optimal control problems the algorithm is an adaptation of the trust region modifications of newton s method and solves at each iteration a linear quadratic control problem with an additional constraint under some assumptions the proposed algorithm is shown to possess a global convergence property a numerical example is presented to illustrate the method

an interior point method for solving box constrained underdetermined nonlinear systems a method introduced recently by bellavia macconi and morini for solving square nonlinear systems with bounds is modified and extended to cope with the underdetermined case the algorithm produces a sequence of interior iterates and is based on globalization techniques due to coleman and li global and local convergence results are proved and numerical experiments are presented

algorithms for the solution of quadratic knapsack problems we consider the problem of minimizing a quadratic function with a knapsack constraint quadratic knapsack problems have numerous applications including the least distance problem quadratic programming defined on the convex hull of a set of points and the maximum clique problem we propose and analyze three algorithms for solving quadratic knapsack problems two algorithms are based on recently developed interior point methods the first solves convex quadratic problems and the second computes a stationary point for the indefinite case for both computational results on a variety of test problems are presented the third algorithm based on simplicial partitioning and convex underestimating functions can be used to compute the global optimum of indefinite quadratic knapsack problems

nonmonotonic back tracking trust region interior point algorithm for linear constrained optimization in this paper we modify the trust region interior point algorithm proposed by bonnans and pola in siam j optim 7 3 1997 717 for linear constrained optimization a mixed strategy using both trust region and line search techniques is adopted which switches to back tracking steps when a trial step produced by the trust region subproblem may be unacceptable the global convergence and local convergence rate of the improved algorithm are established under some reasonable conditions a nonmonotonic criterion is used to speed up the convergence progress in some ill conditioned cases the results of numerical experiments are reported to show the effectiveness of the proposed algorithm

global convergence of trust region interior point algorithms for infinite dimensional nonconvex minimization subject to pointwise bounds a class of interior point trust region algorithms for infinite dimensional nonlinear optimization subject to pointwise bounds in l p banach spaces 2 le p le infty is formulated and analyzed the problem formulation is motivated by optimal control problems with l p controls and pointwise control constraints the interior point trust region algorithms are generalizations of those recently introduced by coleman and li siam j optim 6 1996 pp 418 445 for finite dimensional problems many of the generalizations derived in this paper are also important in the finite dimensional context all first and second order global convergence results known for trust region methods in the finite dimensional setting are extended to the infinite dimensional framework of this paper

alternative theorems for quadratic inequality systems and global quadratic optimization we establish alternative theorems for quadratic inequality systems consequently we obtain lagrange multiplier characterizations of global optimality for classes of nonconvex quadratic optimization problems we present a generalization of dine s theorem to a system of two homogeneous quadratic functions with a regular cone the class of regular cones are cones k for which k cup k is a subspace as a consequence we establish a generalization of the powerful s lemma which paves the way to obtain a complete characterization of global optimality for a general quadratic optimization model problem involving a system of equality constraints in addition to a single quadratic inequality constraint we then present an alternative theorem for a system of three nonhomogeneous inequalities by way of establishing the convexity of the joint range of three homogeneous quadratic functions using a regular cone this yields lagrange multiplier characterizations of global optimality for classes of trust region type problems with two inequality constraints finally we establish an alternative theorem for systems involving an arbitrary finite number of quadratic inequalities involving z matrices which are matrices with nonpositive off diagonal elements and present necessary and sufficient conditions for global optimality for classes of nonconvex inequality constrained quadratic optimization problems