the lagrangian relaxation method for solving integer programming problems this article originally appeared in management science january 1981 volume 27 number 1 pp 1 18 published by the institute of management sciences one of the most computationally useful ideas of the 1970s is the observation that many hard integer programming problems can be viewed as easy problems complicated by a relatively small set of side constraints dualizing the side constraints produces a lagrangian problem that is easy to solve and whose optimal value is a lower bound for minimization problems on the optimal value of the original problem the lagrangian problem can thus be used in place of a linear programming relaxation to provide bounds in a branch and bound algorithm this approach has led to dramatically improved algorithms for a number of important problems in the areas of routing location scheduling assignment and set covering this paper is a review of lagrangian relaxation based on what has been learned in the last decade

entropy like proximal methods in convex programming we study an extension of the proximal method for convex programming where the quadratic regularization kernel is substituted by a class of convex statistical distances called divergences which are typically entropy like in form after establishing several basic properties of these quasi distances we present a convergence analysis of the resulting entropy like proximal algorithm applying this algorithm to the dual of a convex program we recover a wide class of nonquadratic multiplier methods and prove their convergence

primal dual nonlinear rescaling method with dynamic scaling parameter update in this paper we developed a general primal dual nonlinear rescaling method with dynamic scaling parameter update pdnrd for convex optimization we proved the global convergence established 1 5 q superlinear rate of convergence under the standard second order optimality conditions the pdnrd was numerically implemented and tested on a number of nonlinear problems from cops and cute sets we present numerical results which strongly corroborate the theory

lipschitz continuity of solutions of linear inequalities programs and complementarity problems it is shown that solutions of linear inequalities linear programs and certain linear complementarity problems e g those with p matrices or z matrices but not semidefinite matrices are lipschitz continuous with respect to changes in the right hand side data of the problem solutions of linear programs are not lipschitz continuous with respect to the coefficients of the objective function the lipschitz constant given here is a generalization of the role played by the norm of the inverse of a nonsingular matrix in bounding the perturbation of the solution of a system of equations in terms of a right hand side perturbation

separation of nonconvex sets with general augmenting functions in this paper we consider two geometric optimization problems that are dual to each other and characterize conditions under which the optimal values of the two problems are equal this characterization relies on establishing separation results for nonconvex sets using general concave surfaces defined in terms of convex augmenting functions we prove separation results for augmenting functions that are bounded from below unbounded augmenting functions and asymptotic augmenting functions

improving the accuracy of computed eigenvalues and eigenvectors this paper describes a computational method for improving the accuracy of a given eigenvalue and its associated eigenvector the method is analogous to iterative improvement for the solution of linear systems an iterative algorithm using working precision arithmetic is applied to increase the accuracy of the eigenpair the only extended precision computation is the residual calculation the method is related to inverse iteration and to newton s method applied to the eigenvalue problem

understanding the qr algorithm the qr algorithm is currently the most popular method for finding all eigenvalues of a full matrix while qr is now well understood by specialists in eigenvalue computations this understanding is not being conveyed effectively to the mathematical public many accounts present wilkinson s 1965 convergence proof others establish some of the connections between the qr algorithm the power method and inverse iteration usually much emphasis is rightly placed on the refinements such as shifts of origin which are required to make the algorithm competitive but practically all accounts fail to explain the basic meaning of qr iterations as a consequence the qr algorithm is widely thought to be difficult to understand the purpose of this paper is to try to convince the reader that the opposite is true in fact the qr algorithm is neither more nor less than a clever implementation of simultaneous iteration which is itself a natural easily understood extension of the power method this point

nonlinear proximal point algorithms using bregman functions with applications to convex programming a bregman function is a strictly convex differentiable function that induces a well behaved distance measure or d function on euclidean space this paper shows that for every bregman function there exists a nonlinear version of the proximal point algorithm and presents an accompanying convergence theory applying this generalization of the proximal point algorithm to convex programming one obtains the d function proximal minimization algorithm of censor and zenios and a wide variety of new multiplier methods these multiplier methods are different from those studied by kort and bertsekas and include nonquadratic variations on the proximal method of multipliers

convergence of gradient descent for low rank matrix approximation this paper provides a proof of global convergence of gradient search for low rank matrix approximation such approximations have recently been of interest for large scale problems as well as for dictionary learning for sparse signal representations and matrix completion the proof is based on the interpretation of the problem as an optimization on the grassmann manifold and fubiny study distance on this space

an exact two dimensional non guillotine cutting tree search procedure we consider the two dimensional cutting problem of cutting a number of rectangular pieces from a single large rectangle so as to maximize the value of the pieces cut we develop a lagrangean relaxation of a zero one integer programming formulation of the problem and use it as a bound in a tree search procedure subgradient optimization is used to optimize the bound derived from the lagrangean relaxation problem reduction tests derived from both the original problem and the lagrangean relaxation are given incorporating the bound and the reduction tests into a tree search procedure enables moderately sized problems to be solved

perturbation analysis of a condition number for convex inequality systems and global error bounds for analytic systems in this paper several types of perturbations on a convex inequality system are considered and conditions are obtained for the system to be well conditioned under these types of perturbations where the well conditionedness of a convex inequality system is defined in terms of the uniform boundedness of condition numbers under a set of perturbations it is shown that certain types of perturbations can be used to characterize the well conditionedness of a convex inequality system in which either the system has a bounded solution set and satisfies the slater condition or an associated convex inequality system which defines the recession cone of the solution set for the system satisfies the slater condition finally sufficient conditions are given for the existence of a global error bound for an analytic system it is shown that such a global error bound always holds for any inequality system defined by finitely many convex analytic functions when the zero vector is in the relative interior of the domain of an associated convex conjugate function

bases for some reciprocity algebras i for a complex vector space v let p v be the algebra of polynomial functions on v in this paper we construct bases for the algebra of all gl n c x gl m1 c x gl m2 c x x gl mr c highest weight vectors in p c n c m where m m 1 m r and mj n for all 1 j r and the algebra of gl n c x gl k c x gl 1 c highest weight vectors in p c n c k c n c l

a re scaled twin augmented lagrangian algorithm for saddle point seeking abstract in a ouorou a primal dual algorithm for monotropic programming and its application to network optimization computational optimization and application 15 2002 125 143 a block wise gauss seidel method has been developed for monotropic programming problems using two different quadratic augmented lagrangian functions defined for the primal and the dual problems in this paper we extend the concept by introducing a nonlinear re scaling principle obtained recently by polyak r polyak nonlinear rescaling vs smoothing technique in constrained optimization mathematical programming 92 2002 197 235

modified barrier functions theory and methods the nonlinear rescaling principle employs monotone and sufficiently smooth functions to transform the constraints and or the objective function into an equivalent problem the classical lagrangian which has important properties on the primal and the dual spaces the application of the nonlinear rescaling principle to constrained optimization problems leads to a class of modified barrier functions mbf s and mbf methods mbfm s being classical lagrangians cl s for an equivalent problem the mbf s combine the best properties of the cl s and classical barrier functions cbf s but at the same time are free of their most essential deficiencies due to the excellent mbf properties new characteristics of the dual pair convex programming problems have been found and the duality theory for nonconvex constrained optimization has been developed the mbfm have up to a superlinear rate of convergence and are to the classical barrier functions cbf s method as the multipliers method for augmented lagrangians is to the classical penalty function method based on the dual theory associated with mbf the method for the simultaneous solution of the dual pair convex programming problems with up to quadratic rates of convergence have been developed the application of the mbf to linear lp and quadratic qp programming leads to a new type of multipliers methods which have a much better rate of convergence under lower computational complexity at each step as compared to the cbf methods the numerical realization of the mbfm leads to the newton modified barrier method nmbm the excellent mbf properties allow us to discover that for any nondegenerate constrained optimization problem there exists a hot start from which the nmbm has a better rate of convergence a better complexity bound and is more stable than the interior point methods which are based on the classical barrier functions

interior gradient and epsilon subgradient descent methods for constrained convex minimization we extend epsilon subgradient descent methods for unconstrained nonsmooth convex minimization to constrained problems over polyhedral sets in particular over r p this is done by replacing the usual squared quadratic regularization term used in subgradient schemes by the logarithmic quadratic distancelike function recently introduced by the authors we then obtain interior e subgradient descent methods which allow us to provide a natural extension of bundle methods and polyak s subgradient projection methods for nonsmooth convex minimization furthermore similar extensions are considered for smooth constrained minimization to produce interior gradient descent methods global convergence as well as an improved global efficiency estimate are established for these methods within a unifying principle and minimal assumptions on the problem s data

local convergence analysis of fastica and related algorithms the fastica algorithm is one of the most prominent methods to solve the problem of linear independent component analysis ica although there have been several attempts to prove local convergence properties of fastica rigorous analysis is still missing in the community the major difficulty of analysis is because of the well known sign flipping phenomenon of fastica which causes the discontinuity of the corresponding fastica map on the unit sphere in this paper by using the concept of principal fiber bundles fastica is proven to be locally quadratically convergent to a correct separation higher order local convergence properties of fastica are also investigated in the framework of a scalar shift strategy moreover as a parallelized version of fastica the so called qr fastica algorithm which employs the qr decomposition gram schmidt orthonormalization process instead of the polar decomposition is shown to share similar local convergence properties with the original fastica

the geometry of weighted low rank approximations the low rank approximation problem is to approximate optimally with respect to some norm a matrix by one of the same dimension but smaller rank it is known that under the frobenius norm the best low rank approximation can be found by using the singular value decomposition svd although this is no longer true under weighted norms in general it is demonstrated here that the weighted low rank approximation problem can be solved by finding the subspace that minimizes a particular cost function a number of advantages of this parameterization over the traditional parameterization are elucidated finding the minimizing subspace is equivalent to minimizing a cost function on the grassmann manifold a general framework for constructing optimization algorithms on manifolds is presented and it is shown that existing algorithms in the literature are special cases of this framework within this framework two novel algorithms a steepest descent algorithm and a newton like algorithm are derived for solving the weighted low rank approximation problem they are compared with other algorithms for low rank approximation as well as with other algorithms for minimizing a cost function on a grassmann manifold

convergence of proximal like algorithms we analyze proximal methods based on entropy like distances for the minimization of convex functions subject to nonnegativity constraints we prove global convergence results for the methods with approximate minimization steps and an ergodic convergence result for the case of finding a zero of a maximal monotone operator we also consider linearly constrained convex problems and establish a quadratic convergence rate result for linear programs our analysis allows us to simplify and extend the available convergence results for these methods

on the largest principal angle between random subspaces formulas are derived for the probability density function and the probability distribution function of the largest canonical angle between two p dirnensional subspaces of r n chosen from the uniform distribution on the grassmann manifold which is the unique distribution invariant by orthogonal transformations of r n the formulas involve the gamma function and the hypergeometric function of a matrix argument c 2005 elsevier inc all rights reserved

a practical general approximation criterion for methods of multipliers based on bregman distances this paper demonstrates that for generalized methods of multipliers for convex programming based on bregman distance kernels including the classical quadratic method of multipliers the minimization of the augmented lagrangian can be truncated using a simple generally implementable stopping criterion based only on the norms of the primal iterate and the gradient or a subgradient of the augmented lagrangian at that iterate previous results in this and related areas have required conditions that are much harder to verify such as e optimality with respect to the augmented lagrangian or strong conditions on the convex program to be solved here only existence of a kkt pair is required and the convergence properties of the exact form of the method are preserved the key new element in the analysis is the use of a full conjugate duality framework as opposed to mainly examining the action of the method on the standard dual function of the convex program an existence result for the iterates stronger than those possible for the exact form of the algorithm is also included

lagrangian regularization approach to constrained optimization problems abstract this paper proposes a new regularization approach referred to as the lagrangian regularization approach which aims to circumvent the nondifferentiability of a positively homogeneous function r m in a conceptual unconstrained reformulation for constrained optimization problems with the appropriate choices of regularizing function we obtain a family of smooth functions that include as special cases the existing penalty and barrier functions as such our approach can be used as an instrumental tool to resolve the nondifferentiability of r m as well as a unified way to construct penalty functions for convex programming cases we present its global convergence analysis

augmented lagrangian applied to convex quadratic problems an algorithm based on the augmented lagrangian method is proposed to solve convex quadratic programming problem the quadratic penalty is considered here hence the augmented lagrangian function is quadratic when applied to quadratic programming problem for this penalty we show that if the lagrangian function associated with the original problem is strict convex or convex then the hessian matrix of augmented lagrangian function is positive definite or positive semi definite numerical experiments are presented illustrating the performance of the algorithm for the cute test set

algorithms for the analysis of 3d finite strain contact problems from the constraint imposition aspects in 3d to friction regularization various ideas are exposed in this paper a variation of the rockafellar lagrangian is proposed which results in continuous second order derivatives if lagrange multiplier estimates are greater or equal than one this fact allows the adoption of a full second order i e lagrange newton method avoiding sequential unconstrained minimization techniques an algorithm for global and local contact detection is presented which is developed for dealing with large step sizes typical of implicit methods a modified constraint definition to deal with non smooth situations is presented aspects of friction implementation including a regularization scheme which ensures stepwise objectivity are detailed finally several illustrative examples are carried out with success

global convergence properties of some iterative methods for linear complementarity problems the subject of this work is a class of iterative methods for solving the linear complementarily problem lcp these methods are based on a reformulation of the lcp consisting of a usually differentiable system of nonlinear equations to which newton s method is applied thus the algorithms are locally q quadratically convergent furthermore global convergence results for these methods are proved for lcps associated with r 0 nondegenerate and p matrices finally some promising numerical results are reported for both constructed and randomly generated problems

a method for the cutting stock problem with different qualities abstract an operations research project at a swedish company is reported the project deals with the problem of allocating wood products with different quality requirements to be cut from wooden boards which in turn consists of a number of quality zones a major difficulty is the existence of distinct time limits since the cross cutting of the boards is done in a real time process in this paper we derive a mathematical model which includes a set packing problem that accurately describes the real world problem this problem is then solved using a combination of lagrangean relaxation and subgradient optimization numerical results for practical problems are reported the results indicate a valid model and satisfactory efficiency of the proposed solution procedure

some convergence properties of the modified log barrier method for linear programming on each iteration of the modified log barrier method for linear programming a vector of variables is calculated by the unconstrained minimization of a fixed positive multiple of the objective function minus a linear combination of the logarithms of the constraint residuals plus positive constants thus in contrast to interior point methods all the logarithms are finite whenever the vector of variables is feasible the parameters of the calculation have the property that the coefficients of the linear combination of logarithms can be viewed as estimates of lagrange multipliers of karush kuhn tucker conditions of the given problem initially these coefficients have any positive values and then their adjustment is derived from the zero gradient that occurs at the end of each unconstrained minimization calculation these techniques provide a simple and interesting algorithm for linear programming that was proposed by polyak we study its convergence properties finding that the sequence of calculated vari

log sigmoid nonlinear lagrange method for nonlinear optimization problems over second order cones this paper analyzes the rate of local convergence of the log sigmoid nonlinear lagrange method for nonconvex nonlinear second order cone programming under the componentwise strict complementarity condition the constraint nondegeneracy condition and the second order sufficient condition we show that the sequence of iteration points generated by the proposed method locally converges to a local solution when the penalty parameter is less than a threshold and the error bound of solution is proportional to the penalty parameter finally we report numerical results to show the efficiency of the method

decomposing combinatorial auctions and set packing problems combinatorial auctions allow bidders to bid on bundles of items rather than just on single items the winner determination problem in combinatorial auctions is the problem of determining the allocation of items to bidders such that the sum of the accepted bid prices is maximized this problem is equivalent to the well known maximum weight set packing problem even though these problems are np hard in general they can be solved in polynomial time on instances whose associated item graphs have bounded treewidth called structured item graphs however the tractability of determining whether for a given problem instance a structured item graph of fixed treewidth exists and if so computing one efficiently was an open problem in this article we solve this problem by proving that deciding the existence of structured item graphs is computationally intractable even for treewidth 3 motivated by this unfavorable complexity result we investigate other structural restrictions and we show that the notion of hypertree decomposition a well studied measure of hypergraph cyclicity turns out to be most useful here indeed we show that the winner determination problem is solvable in polynomial time on instances whose dual auction hypergraphs have bounded hypertree width our solution method is based on encoding winner determination via a constraint satisfaction optimization problem and on exhibiting an algorithm to solve this latter problem efficiently for such structurally restricted instances the class of tractable instances identified by our approach while being efficiently recognizable properly contains the class of instances having a structured item graph moreover on the larger class our method solves winner determination with the same asymptotic complexity as the best algorithm proposed in the literature for the subclass of structured item graphs hypertree decompositions can equally profitably be applied to the maximum weight independent set problem which is the dual problem of maximum weight set packing

a nonlinear lagrangian based on fischer burmeister ncp function this paper proposes a nonlinear lagrangian based on fischer burmeister ncp function for solving nonlinear programming problems with inequality constraints the convergence theorem shows that the sequence of points generated by this nonlinear lagrange algorithm is locally convergent when the penalty parameter is less than a threshold under a set of suitable conditions on problem functions and the error bound of solution depending on the penalty parameter is also established moreover the paper develops the dual approach associated with the proposed nonlinear lagrangian in which the related duality theorem is demonstrated furthermore it is shown that the condition number of the nonlinear lagrangian hessian at the optimal solution is proportional to the controlling penalty parameter numerical results for solving several nonlinear programming problems are reported showing that the new nonlinear lagrangian is superior over other known nonlinear lagrangians for solving some nonlinear programming problems

nonlinear rescaling vs smoothing technique in convex optimization we introduce an alternative to the smoothing technique approach for constrained optimization as it turns out for any given smoothing function there exists a modification with particular properties we use the modification for nonlinear rescaling nr the constraints of a given constrained optimization problem into an equivalent set of constraints the constraints transformation is scaled by a vector of positive parameters the lagrangian for the equivalent problems is to the correspondent smoothing penalty functions as augmented lagrangian to the classical penalty function or mbfs to the barrier functions moreover the lagrangians for the equivalent problems combine the best properties of quadratic and nonquadratic augmented lagrangians and at the same time are free from their main drawbacks sequential unconstrained minimization of the lagrangian for the equivalent problem in primal space followed by both lagrange multipliers and scaling parameters update leads to a new class of nr multipliers methods which are equivalent to the interior quadratic prox methods for the dual problem we proved convergence and estimate the rate of convergence of the nr multipliers method under very mild assumptions on the input data we also estimate the rate of convergence under various assumptions on the input data in particular under the standard second order optimality conditions the nr method converges with q linear rate without unbounded increase of the scaling parameters which correspond to the active constraints we also established global quadratic convergence of the nr methods for linear programming with unique dual solution we provide numerical results which strongly support the theory

a grassmann rayleigh quotient iteration for computing invariant subspaces the classical rayleigh quotient iteration rqi allows one to compute a one dimensional invariant subspace of a symmetric matrix a here we propose a generalization of the rqi which computes a p dimensional invariant subspace of a cubic convergence is preserved and the cost per iteration is low compared to other methods proposed in the literature

the dynamics of rayleigh quotient iteration this paper analyzes the rayleigh quotient iteration algorithm for symmetric matrices dynamical systems techniques are employed to characterize the sets of points for which the algorithm will converge to an eigenvector it is shown that these sets have full measure and that their geometric nature is related to the spacing of the eigenvalues

convergence rate analysis of nonquadratic proximal methods for convex and linear programming the divergence proximal method is an extension of the proximal minimization algorithm where the usual quadratic proximal term is substituted by a class of convex statistical distances called divergences in this paper we study the convergence rate of this nonquadratic proximal method for convex and particularly linear programming we identify a class of divergences for which superlinear convergence is attained both for optimization problems with strongly convex objectives at the optimum and linear programming problems when the regularization parameters tend to zero we prove also that with regularization parameters bounded away from zero convergence is at least linear for a wider class of divergences when the method is applied to the same kinds of problems we further analyze the associated class of augmented lagrangian methods for convex programming with nonquadratic penalty terms and prove convergence of the dual generated by these methods for linear programming problems under a weak nondegeneracy assumption

convergence analysis of a proximal like minimization algorithm using bregman functions an alternative convergence proof of a proximal like minimization algorithm using bregman functions recently proposed by censor and zenios is presented the analysis allows the establishment of a global convergence rate of the algorithm expressed in terms of function values

an interior proximal algorithm and the exponential multiplier method for semidefinite programming we introduce an interior proximal algorithm for semidefinite optimization problems and establish its convergence properties we also study the corresponding dual algorithm leading to an exponential multiplier method for semidefinite programs potential applications and extensions are also discussed

a framework for generalising the newton method and other iterative methods from euclidean space to manifolds the newton iteration is a popular method for minimising a cost function on euclidean space various generalisations to cost functions defined on manifolds appear in the literature in each case the convergence rate of the generalised newton iteration needed establishing from first principles the present paper presents a framework for generalising iterative methods from euclidean space to manifolds that ensures local convergence rates are preserved it applies to any memoryless iterative method computing a coordinate independent property of a function such as a zero or a local minimum all possible newton methods on manifolds are believed to come under this framework changes of coordinates and not any riemannian structure are shown to play a natural role in lifting the newton method to a manifold the framework also gives new insight into the design of newton methods in general

a generalization of the proximal point algorithm the problem considered in this paper is to find a solution to the generalized equation 0 in t x y where t is a maximal monotone operator on the product h 1 times h 2 of two hilbert spaces h 1 and h 2 we give a generalization of the proximal map and the proximal point algorithm in which the proposed iterative procedure is based on just one variable applying to convex programming problems instead of adding a quadratic term for all variables as in the proximal point algorithm a quadratic term for a subset of variables is added this paper proves that under a mild assumption our algorithm has the same convergence properties as the regular proximal point algorithm

the global convergence of augmented lagrangian methods based on ncp function in constrained nonconvex optimization abstract in this paper we present the global convergence properties of the primal dual method using a class of augmented lagrangian functions based on ncp function for inequality constrained nonconvex optimization problems we construct four modified augmented lagrangian methods based on different algorithmic strategies we show that the convergence to a kkt point or a degenerate point of the original problem can be ensured without requiring the boundedness condition of the multiplier sequence

block newton method and block rayleigh quotient iteration for computing invariant subspaces of general complex matrices abstract we consider the block newton method and a modification of it the block rayleigh quotient iteration for approximating a simple p dimensional invariant subspace x im x and the corresponding eigenvalues collected in the projection l x h x 1 x h a x of an arbitrary complex matrix a c n n both methods generate a sequence of bases u k such that the subspaces im u k approximate the target subspace x the block newton method is newton s method applied to the extended system a x x l 0 w h x i with respect to x l where w is a normalization matrix we give a direct convergence analysis which in contrast to the approach by beyn kless thummler 2001 explicitly exploits the product form of the second derivative and therefore leads to new and essentially sharper error recursions for each of the sequences u k and k where u k and k are the k th x and l iterates resp then following losche schwetlick timmermann 1998 where the case of real symmetric a is considered we modify the block newton method as follows in step k we set w u k and use only the x update from the newton method to improve u k and then with the improved basis perform a rayleigh schur process and take the orthonormal basis and the generalized rayleigh quotient matrix obtained from it as new iterates so the method can be considered as a block rayleigh quotient iteration we show that for a sufficiently good initial approximation u 0 the method converges in that the sequence sin k with k im u k x converges q quadratically to zero provided that the invariant subspace x is simple i e that the eigenvalues corresponding to x which are the eigenvalues of the projection l x h x 1 x h a x of a onto x are separated from those corresponding to x which are given by a l the convergence results are directly proven without exploiting standard newton convergence results by using the angles between the corresponding subspaces instead of working with the norm differences of the bases as done by beyn kless thummler 2001 as a byproduct the method is shown to converge cubically if a a h as in the case p 1

grasp for set packing problems abstract the principles of the greedy randomized adaptative search procedure grasp metaheuristic are instantiated for the set packing problem we investigated several construction phases and evaluated improvements based on advanced strategies these improvements include a self tuning procedure using reactive grasp an intensification procedure using path relinking and a procedure involving the diversification of the selection using a learning process two sets of various numerical instances were used to perform the computational experiments the first set contains randomly generated instances while the second includes instances relating to real problems in railway planning no metaheuristic has previously been applied to this combinatorial problem consequently we have discussed grasp s performances both in relation to lower upper bounds and to the results obtained with cplex when such results are available our analysis based on the average performances observed shows the impact of the suggested strategies and indicates the configuration that produces the best results

interior proximal and multiplier methods based on second order homogeneous kernels we study a class of interior proximal algorithms and nonquadratic multiplier methods for solving convex programs where the usual proximal quadratic term is replaced by an homogeneous functional of order two defined in terms of a convex function we prove under mild assumptions several new convergence results in both the primal and dual framework allowing also for approximate minimization in particular we introduce a new class of interior proximal methods which is globally convergent and allows for generating c multiplier methods with bounded hessians which exhibit strong convergence properties we also consider linearly constrained convex problems and establish global quadratic convergence rates results for linear programs we then study in detail a particular realization of these algorithms leading to a new class of logarithmic quadratic interior point algorithms which are shown to enjoy several attractive properties for solving constrained convex optimization problems

nonlinear rescaling and proximal like methods in convex optimization the nonlinear rescaling principle nrp consists of transforming the objective function and or the constraints of a given constrained optimization problem into another problem which is equivalent to the original one in the sense that their optimal set of solutions coincides a nonlinear transformation parameterized by a positive scalar parameter and based on a smooth sealing function is used to transform the constraints the methods based on nrp consist of sequential unconstrained minimization of the classical lagrangian for the equivalent problem followed by an explicit formula updating the lagrange multipliers we first show that the nrp leads naturally to proximal methods with an entropy like kernel which is defined by the conjugate of the scaling function and establish that the two methods are dually equivalent for convex constrained minimization problems we then study the convergence properties of the nonlinear rescaling algorithm and the corresponding entropy like proximal methods for convex constrained optimization problems special cases of the nonlinear rescaling algorithm are presented in particular a new class of exponential penalty modified barrier functions methods is introduced

two sided grassmann rayleigh quotient iteration the two sided rayleigh quotient iteration proposed by ostrowski computes a pair of corresponding left right eigenvectors of a matrix c we propose a grassmannian version of this iteration i e its iterates are pairs of p dimensional subspaces instead of one dimensional subspaces in the classical case the new iteration generically converges locally cubically to the pairs of left right p dimensional invariant subspaces of c moreover grassmannian versions of the rayleigh quotient iteration are given for the generalized hermitian eigenproblem the hamiltonian eigenproblem and the skew hamiltonian eigenproblem

asymptotic analysis for penalty and barrier methods in convex and linear programming we consider a wide class of penalty and barrier methods for convex programming which includes a number of specific functions proposed in the literature we provide a systematic way to generate penalty and barrier functions in this class and we analyze the existence of primal and dual optimal paths generated by these penalty methods as well as their convergence to the primal and dual optimal sets for linear programming we prove that these optimal paths converge to single points

message passing for graph structured linear programs proximal methods and rounding schemes the problem of computing a maximum a posteriori map configuration is a central computational challenge associated with markov random fields there has been some focus on tree based linear programming lp relaxations for the map problem this paper develops a family of super linearly convergent algorithms for solving these lps based on proximal minimization schemes using bregman divergences as with standard message passing on graphs the algorithms are distributed and exploit the underlying graphical structure and so scale well to large problems our algorithms have a double loop character with the outer loop corresponding to the proximal sequence and an inner loop of cyclic bregman projections used to compute each proximal update we establish convergence guarantees for our algorithms and illustrate their performance via some simulations we also develop two classes of rounding schemes deterministic and randomized for obtaining integral configurations from the lp solutions our deterministic rounding schemes use a re parameterization property of our algorithms so that when the lp solution is integral the map solution can be obtained even before the lp solver converges to the optimum we also propose graph structured randomized rounding schemes applicable to iterative lp solving algorithms in general we analyze the performance of and report simulations comparing these rounding schemes

the continuous time rayleigh quotient flow on the sphere a continuous time differential equation inspired by the rayleigh quotient iteration for a symmetric matrix a is studied the set of all continuous solutions termed the rayleigh quotient flow is shown to be a time scaled version of the newton flow for rayleigh quotient cost functional the scaling factor ensures that the rate of variation of the rayleigh quotient is constant and positive along solutions this interpretation leads to a precise phase portrait for the rayleigh quotient flow it is shown that a complete non degenerate solution of the rayleigh quotient flow visits each of the eigenvectors of a in ascending order

smoothing methods for nonsmooth nonconvex minimization we consider a class of smoothing methods for minimization problems where the feasible set is convex but the objective function is not convex not differentiable and perhaps not even locally lipschitz at the solutions such optimization problems arise from wide applications including image restoration signal reconstruction variable selection optimal control stochastic equilibrium and spherical approximations in this paper we focus on smoothing methods for solving such optimization problems which use the structure of the minimization problems and composition of smoothing functions for the plus function x many existing optimization algorithms and codes can be used in the inner iteration of the smoothing methods we present properties of the smoothing functions and the gradient consistency of subdifferential associated with a smoothing function moreover we describe how to update the smoothing parameter in the outer iteration of the smoothing methods to guarantee convergence of the smoothing methods to a stationary point of the original minimization problem

on dual convergence of the generalized proximal point method with bregman distances the use of generalized distances e g bregman distances instead of the euclidean one in the proximal point method for convex optimization allows for elimination of the inequality constraints from the subproblems in this paper we consider the proximal point method with bregman distances applied to linearly constrained convex optimization problems and study the behavior of the dual sequence obtained from the optimal multipliers of the linear constraints of each subproblem under rather general assumptions which cover most bregman distances of interest we obtain an ergodic convergence result namely that a sequence of weighted averages of the dual sequence converges to a specific point of the dual optimal set as an intermediate result we prove under the same assumptions that the dual central path generated by a large class of barriers including the generalized bregman distances converges to the same point

choosing the forcing terms in an inexact newton method an inexact newton method is a generalization of newton s method for solving f x 0 f mathbb r n to mathbb r n in which at the kth iteration the step s k from the current approximate solution x k is required to satisfy a condition f x k f x k s k leqslant eta k f x k for a forcing term eta k in 0 1 in typical applications the choice of the forcing terms is critical to the efficiency of the method and can affect robustness as well promising choices of the forcing terms are given their local convergence properties are analyzed and their practical performance is shown on a representative set of test problems

generalized spectral decomposition method for solving stochastic finite element equations invariant subspace problem and dedicated algorithms stochastic galerkin methods have become a significant tool for the resolution of stochastic partial differential equations spde however they suffer from prohibitive computational times and memory requirements when dealing with large scale applications and high stochastic dimensionality some alternative techniques based on the construction of suitable reduced deterministic or stochastic bases have been proposed in order to reduce these computational costs recently a new approach based on the concept of generalized spectral decomposition gsd has been introduced for the definition and the automatic construction of reduced bases in this paper the concept of gsd initially introduced for a class of linear elliptic spde is extended to a wider class of stochastic problems the proposed definition of the gsd leads to the resolution of an invariant subspace problem which is interpreted as an eigen like problem this interpretation allows the construction of efficient numerical algorithms for building optimal reduced bases which are associated with dominant generalized eigenspaces the proposed algorithms by separating the resolution of reduced stochastic and deterministic problems lead to drastic computational savings their efficiency is illustrated on several examples where they are compared to classical resolution techniques

on the convergence of the exponential multiplier method for convex programming in this paper we analyze the exponential method of multipliers for convex constrained minimization problems which operates like the usual augmented lagrangian method except that it uses an exponential penalty function in place of the usual quadratic we also analyze a dual counterpart the entropy minimization algorithm which operates like the proximal minimization algorithm except that it uses a logarithmic entropy proximal term in place of a quadratic we strengthen substantially the available convergence results for these methods and we derive the convergence rate of these methods when applied to linear programs

entropic proximal mappings with applications to nonlinear programming we introduce a family of new transforms based on imitating the proximal mapping of moreau and the associated moreau yosida proximal approximation of a function the transforms are constructed in terms of the a divergence functional a generalization of the relative entropy and of bregman s measure of distance an analogue of moreau s theorem associated with these entropy like distances is proved we show that the resulting entropic proximal maps share properties similar to the proximal mapping and provide a fairly general framework for constructing approximation and smoothing schemes for optimization problems applications of the results to the construction of generalized augmented lagrangians for nonlinear programs and the minimax problem are presented