stopping rules and backward error analysis for bound constrained optimization termination criteria for the iterative solution of bound constrained optimization problems are examined in the light of backward error analysis it is shown that the problem of determining a suitable perturbation on the problem s data corresponding to the definition of the backward error is analytically solvable under mild assumptions moreover a link between existing termination criteria and this solution is clarified indicating that some standard measures of criticality may be interpreted in the sense of backward error analysis the backward error problem is finally considered from the multicriteria optimization point of view and some numerical illustration is provided

new active set identification for general constrained optimization and minimax problems abstract the purpose of this paper is to discuss the problem of identifying the active constraints for general constrained nonlinear programming and constrained minimax problems at an isolated local solution facchinei fischer and kanzow 4 proposed an effective technique which can identify the active set in a neighborhood of an isolated local solution for nonlinear programming and han jian and li 5 improved and extended this to minimax problems in this work a new active constraint identification set is constructed not only is it tighter than the previous two identification sets but also it can be used effectively in penalty algorithms without strict complementarity and linear independence it is shown that the new identification technique can accurately identify the active constraints of nonlinear programming and constrained minimax problems some numerical examples illustrate the identification technique

some numerical experiments with variable storage quasi newton algorithms this paper describes some numerical experiments with variable storage quasi newton methods for the optimization of some large scale models coming from fluid mechanics and molecular biology in addition to assessing these kinds of methods in real life situations we compare an algorithm of a buckley with a proposal by j nocedal the latter seems generally superior provided that careful attention is given to some nontrivial implementation aspects which concern the general question of properly initializing a quasi newton matrix in this context we find it appropriate to use a diagonal matrix generated by an update of the identity matrix so as to fit the rayleigh ellipsoid of the local hessian in the direction of the change in the gradient also a variational derivation of some rank one and rank two updates in hilbert spaces is given

non monotone trust region algorithms for nonlinear optimization subject to convex constraints this paper presents two new trust region methods for solving nonlinear optimization problems over convex feasible domains these methods are distinguished by the fact that they do not enforce strict monotonicity of the objective function values at successive iterates the algorithms are proved to be convergent to critical points of the problem from any starting point extensive numerical experiments show that this approach is competitive with the lancelot package

modified subspace limited memory bfgs algorithm for large scale bound constrained optimization in this paper a subspace limited memory bfgs algorithm for solving large scale bound constrained optimization problems is developed it is modifications of the subspace limited memory quasi newton method proposed by ni and yuan q ni y x yuan a subspace limited memory quasi newton algorithm for large scale nonlinear bound constrained optimization math comput 66 1997 1509 1520 an important property of our proposed method is that more limited memory bfgs update is used under appropriate conditions the global convergence of the method is established the implementations of the method on cute test problems are presented which indicate the modifications are beneficial to the performance of the algorithm

pattern search methods for linearly constrained minimization we extend pattern search methods to linearly constrained minimization we develop a general class of feasible point pattern search algorithms and prove global convergence to a karush kuhn tucker point as in the case of unconstrained minimization pattern search methods for linearly constrained problems accomplish this without explicit recourse to the gradient or the directional derivative of the objective key to the analysis of the algorithms is the way in which the local search patterns conform to the geometry of the boundary of the feasible region

a projected newton method for minimization problems with nonlinear inequality constraints recently developed projected newton methods for minimization problems in polyhedrons and cartesian products of euclidean balls are extended here to general convex feasible sets defined by finitely many smooth nonlinear inequalities iterate sequences generated by this scheme are shown to be locally superlinearly convergent to nonsingular extremals bar u and more specifically to local minimizers bar u satisfying the standard second order kuhn tucker sufficient conditions moreover all such convergent iterate sequences eventually enter and remain within the smooth manifold defined by the active constraints at bar u implementation issues are considered for large scale specially structured nonlinear programs and in particular for multistage discrete time optimal control problems in the latter case overall per iteration computational costs will typically increase only linearly with the number of stages sample calculations are presented for nonlinear programs in a right circular cylinder in 3

connectives and quantifiers in fuzzy sets abstract we discuss some of the issues involved in the selection of appropriate operators for implementing the union and intersection of fuzzy subsets we discuss the extension principles used in extending operators to fuzzy subsets we provide a structure for aggregating linguistic variables in the theory of approximate reasoning we look at mean operators on fuzzy subsets as well as the inclusion of importances in aggregation we discuss non monotone set operators we introduce the concept of linguistic quantifiers and look at some applications of these structures we are particularly concerned with their application to linguistic summarizers syllogistic reasoning and multi criteria decision making

an expanded theoretical treatment of iteration dependent majorize minimize algorithms the majorize minimize mm optimization technique has received considerable attention in signal and image processing applications as well as in statistics literature at each iteration of an mm algorithm one constructs a tangent majorant function that majorizes the given cost function and is equal to it at the current iterate the next iterate is obtained by minimizing this tangent majorant function resulting in a sequence of iterates that reduces the cost function monotonically a well known special case of mm methods are expectation maximization algorithms in this paper we expand on previous analyses of mm due to fessler and hero that allowed the tangent majorants to be constructed in iteration dependent ways also this paper overcomes an error in one of those earlier analyses there are three main aspects in which our analysis builds upon previous work first our treatment relaxes many assumptions related to the structure of the cost function feasible set and tangent majorants for example the cost function can be nonconvex and the feasible set for the problem can be any convex set second we propose convergence conditions based on upper curvature bounds that can be easier to verify than more standard continuity conditions furthermore these conditions allow for considerable design freedom in the iteration dependent behavior of the algorithm finally we give an original characterization of the local region of convergence of mm algorithms based on connected e g convex tangent majorants for such algorithms cost function minimizers will locally attract the iterates over larger neighborhoods than typically is guaranteed with other methods this expanded treatment widens the scope of the mm algorithm designs that can be considered for signal and image processing applications allows us to verify the convergent behavior of previously published algorithms and gives a fuller understanding overall of how these algorithms behave

an active set truncated newton method for large scale bound constrained optimization an active set truncated newton method for large scale bound constrained optimization is proposed the active sets are guessed by an identification technique the search direction consists of two parts some of the components are simply defined the other components are determined by the truncated newton method the method based on a nonmonotone line search technique is shown to be globally convergent numerical experiments are presented using bound constrained problems in the cuter test problem library the numerical performance reveals that our method is effective and competitive with the famous algorithm tron

newton s method for large bound constrained optimization problems we analyze a trust region version of newton s method for bound constrained problems our approach relies on the geometry of the feasible set not on the particular representation in terms of constraints the convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints we also show that the convergence theory leads to an efficient implementation for large bound constrained problems

on branching numbers of normal manifolds kuhn and lowen l and robinson 2 have provided deep and elegant results that characterize the homeomorphic members of certain classes of piecewise linear pl mappings the purpose of this paper is to show that the class of mappings examined by robinson is contained in the class of mappings examined by kuhn and lowen let 312 be a finite piecewise linear pl manifold of r and p r r be pl with respect to 3n i e p is affine on each set in 9x the branching number of 9x l is the maximum number of sets in 317 that can contain any common face of codimension 2 in l theorem 5 31 it is shown that if 312 has a branching number less than or equal to 4 then p is a homeomorphism if and only if it is coherently oriented i e the determinants of p on each set in 312 have the same nonzero sign let c be a nonempty polyhedral convex set in real n dimensional space ir and a belong to the space of real n x n matrices f robinson 2 defines a finite pl manifold 32 of k 2 proposition 2 41 called the normal manifold of c and the normal map a i t r induced by a c which is pl with respect to e 2 proposition 2 51 the main result 2 theorem 4 31 shows that ac is homeomorphic if and only if it is coherently oriented see ralph 3 for an alternative proof and robinson 4 for a much simpler proof in the special case that a is symmetric it is pointed out in 2 section l that if xc could be shown to have a branching number less than or equal to 4 then 2 theorem 4 31 would follow from l theorem 5 31 indeed showing that normal manifolds have this branching number property is precisely our goal throughout the paper c and a will be as above let q be the euclidean projection onto c that is rcc maps each point of ir to its nearest point in c with respect to the euclidean norm the identity in ir x is i

optimality identifiability and sensitivity around a solution of an optimization problem an identifiable subset of the feasible region is one containing all nearby solutions after small perturbations to the problem a quest for only the most essential ingredients of sensitivity analysis leads us to consider identifiable sets that are minimal this new notion lays a broad and intuitive variational analytic foundation for optimality conditions sensitivity and active set methods

conditional subgradient optimization theory and applications abstract we generalize the subgradient optimization method for nondifferentiable convex programming to utilize conditional subgradients firstly we derive the new method and establish its convergence by generalizing convergence results for traditional subgradient optimization secondly we consider a particular choice of conditional subgradients obtained by projections which leads to an easily implementable modification of traditional subgradient optimization schemes to evaluate the subgradient projection method we consider its use in three applications uncapacitated facility location two person zero sum matrix games and multicommodity network flows computational experiments show that the subgradient projection method performs better than traditional subgradient optimization in some cases the difference is considerable these results suggest that our simply modification may improve subgradient optimization schemes significantly this finding is important as such schemes are very popular especially in the context of lagrangean relaxation

a maximum a posteriori probability expectation maximization algorithm for image reconstruction in emission tomography the expectation maximization method for maximum likelihood image reconstruction in emission tomography based on the poisson distribution of the statistically independent components of the image and measurement vectors is extended to a maximum aposteriori image reconstruction using a multivariate gaussian a priori probability distribution of the image vector the approach is equivalent to a penalized maximum likelihood estimation with a special choice of the penalty function the expectation maximization method is applied to find the a posteriori probability maximizer a simple iterative formula is derived for a penalty function that is a weighted sum of the squared deviations of image vector components from their a priori mean values the method is demonstrated to be superior to pure likelihood maximization in that the penalty function prevents the occurrence of irregular high amplitude patterns in the image with a large number of iterations the so called checkerboard effect or noise artifact

on the convergence of nonconvex minimization methods for image recovery nonconvex nonsmooth regularization method has been shown to be effective for restoring images with neat edges fast alternating minimization schemes have also been proposed and developed to solve the nonconvex nonsmooth minimization problem the main contribution of this paper is to show the convergence of these alternating minimization schemes based on the kurdyka ojasiewicz property in particular we show that the iterates generated by the alternating minimization scheme converges to a critical point of this nonconvex nonsmooth objective function we also extend the analysis to nonconvex nonsmooth regularization model with box constraints and obtain similar convergence results of the related minimization algorithm numerical examples are given to illustrate our convergence analysis

newton like methods with limited storage for the solution of elasto viscoplasticity problems quasi newton and secant newton methods have been successful in solving problems of elasto plasticity in this paper these methods are applied to elasto viscoplastic analyses in an algorithm coupling the global solution for displacement increments with local newton raphson stress iteration at each sampling point the familiar rank two dfp and bfgs methods are generalized to the broyden family of algorithms and this generalization is also used to produce a secant newton algorithm corresponding to hoshino s method consideration is given to the most effective strategy when only a limited number of quasi newton iterates may be stored overwriting re start or continuation in secant newton form results are presented for a tunnelling problem using both plane strain and axisymmetric formulations

on the identification of active constraints nondegeneracy conditions that guarantee that the optimal active constraints are identified in a finite number of iterations are studied results of this type have only been established for a few algorithms and then under restrictive hypothesis the main result is a characterization of those algorithms that identify the optimal constraints in a finite number of iterations this result is obtained with a nondegeneracy assumption which is equivalent in the standard nonlinear programming problem to the assumption that there is a set of strictly complementary lagrange multipliers as an important consequence of the authors results the way that this characterization applies to gradient projection and sequential quadratic programming algorithms is shown

an efficient algorithm for large scale nonlinear programming problems with simple bounds on the variables in this paper we present a new conjugate gradient algorithm for nonlinear programming problems with simple bounds on the variables the method is an extension of our conjugate gradient algorithm described in r pytlak ima j numer anal 14 1994 pp 443 460 the simple constraints on the variables are treated by a projection under mild assumptions the method is globally convergent the algorithm has the property whereby many constraints can leave or enter an active set of constraints at one iteration if the strict complementarity condition is satisfied the active set at a solution is identified in a finite number of iterations the algorithm has been tested on problems with more than 10 000 variables

weak sharp minima revisited part iii error bounds for differentiable convex inclusions the notion of weak sharp minima unifies a number of important ideas in optimization part i of this work provides the foundation for the theory of weak sharp minima in the infinite dimensional setting part ii discusses applications of these results to linear regularity and error bounds for nondifferentiable convex inequalities this work applies the results of part i to error bounds for differentiable convex inclusions a number of standard constraint qualifications for such inclusions are also examined

a limited memory quasi newton trust region method for box constrained optimization by means of wolfe conditions strategy we propose a quasi newton trust region method to solve box constrained optimization problems this method is an adequate combination of the compact limited memory bfgs and the trust region direction while the generated point satisfies the wolfe conditions and therefore maintains a positive definite approximation to the hessian of the objective function the global convergence and the quadratic convergence of this method are established under suitable conditions finally we compare our algorithms iwtral and ibwtral with an active set trust region algorithm astral xu and burke 2007 on the cutest box constrained test problems presented by gould et al 2015 numerical results show that the presented method is competitive and totally interesting for solving box constrained optimization

pattern search algorithms for bound constrained minimization we present a convergence theory for pattern search methods for solving bound constrained nonlinear programs the analysis relies on the abstract structure of pattern search methods and an understanding of how the pattern interacts with the bound constraints this analysis makes it possible to develop pattern search methods for bound constrained problems while only slightly restricting the flexibility present in pattern search methods for unconstrained problems we prove global convergence despite the fact that pattern search methods do not have explicit information concerning the gradient and its projection onto the feasible region and consequently are unable to enforce explicitly a notion of sufficient feasible decrease

conjunction and disjunction on 0 1 abstract in classical logic the proposition p and q is true if and only if p is true and q is true in multi valued logic to functionally obtain the truth value of p and q t norms are used but there are many situations in real life in which another functions are also used this paper contains two definitions of conjunction including large families of possible conjunction functions furthermore a new definition of local conjunction is given and symmetrically some definitions of disjunctions and of local disjunction in particular quasiarithmetic and weighted means are studied

conjugate gradient methods less dependent on conjugacy this paper reviews some recent conjugate gradient methods for nonlinear minimization and outlines some promising directions for further development

global convergence of a class of trust region algorithms for optimization with simple bounds this paper extends the known excellent global convergence properties of trust region algorithms for unconstrained optimization to the case where bounds on the variables are present weak conditions on the accuracy of the hessian approximations are considered it is also shown that when the strict complementarily condition holds the proposed algorithms reduce to an unconstrained calculation after finitely many iterations allowing a fast asymptotic rate of convergence

on diagonally preconditioning the 2 step bfgs method with accumulated steps for linearly constrained nonlinear programming abstract we present an algorithm for very large scale linearly constrained nonlinear programming lcnp based on a limited storage quasi newton method in large scale programming solving the reduced newton equation at each iteration can be expensive and may not be justified when far from a local solution besides the amount of storage required by the reduced hessian matrix and even the computing time for its quasi newton approximation may be prohibitive an alternative based on the reduced truncated newton methodology that has proved to be satisfactory for large scale problems is not recommended for very large scale problems since it requires an additional gradient evaluation and the solving of two systems of linear equations per each minor iteration we recommend a 2 step bfgs approximation of the inverse of the reduced hessian matrix that does not require to store any matrix since the product matrix vector is the vector to be approximated it uses the reduced gradient and information from two previous iterations and the so termed restart iteration a diagonal direct bfgs preconditioning is used

maximum likelihood reconstruction for emission tomography previous models for emission tomography et do not distinguish the physics of et from that of transmission tomography we give a more accurate general mathematical model for et where an unknown emission density x y z generates and is to be reconstructed from the number of counts n d in each of d detector units d within the model we give an algorithm for determining an estimate of which maximizes the probability p n of observing the actual detector count data n over all possible densities let independent poisson variables n b with unknown means b b 1 b represent the number of unobserved emissions in each of b boxes pixels partitioning an object containing an emitter suppose each emission in box b is detected in detector unit d with probability p b d d 1 d with p b d a one step transition matrix assumed known we observe the total number n n d of emissions in each detector unit d and want to estimate the unknown b b 1 b for each the observed data n has probability or likelihood p n the em algorithm of mathematical statistics starts with an initial estimate 0 and gives the following simple iterative procedure for obtaining a new estimate new from an old estimate old to obtain k k 1 2 new b old b dd 1 n d p b d old b p b d b 1 b

generalised fuzzy aggregation in estimating the risk of desertification of a burned forest this paper investigates two aspects of using fuzzy logic with a gis the use of non conventional aggregation operators for conjunctive and disjunctive reasoning and the use of weights when combining aggregates of different significance to the classification two methods of weighting are used weights as powers of the membership functions and the novel concept of membership functions that take values greater than 1 the two approaches are investigated in conjunction with 121 possible combinations of aggregation operators that are used to reason in the disjunctive and conjunctive level of the problem of assessing the risk of desertification after a forest fire

convergence properties of trust region methods for linear and convex constraints we develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem global convergence is established for general convex constraints while the local analysis is for linearly constrained problems the main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations as a consequence of the identification properties we develop rate of convergence results by assuming that the step is a truncated newton method our development is mainly geometrical this approach allows the development of a convergence theory without any linear independence assumptions

asymptotic convergence analysis of the forward backward splitting algorithm the asymptotic convergence of the forward backward splitting algorithm for solving equations of type 0 t z is analyzed where t is a multivalued maximal monotone operator in the n dimensional euclidean space when the problem has a nonempty solution set and t is split in the form t h with being maximal monotone and h being co coercive with modulus greater than convergence rates are shown under mild conditions to be linear superlinear or sublinear depending on how rapidly 1 and h 1 grow in the neighborhoods of certain specific points as a special case when both and h are polyhedral functions we get r linear convergence and 2 step q linear convergence without any further assumptions on the strict monotonicity on t or on the uniqueness of the solution as another special case when h 0 the splitting algorithm reduces to the proximal point algorithm and we get new results which complement r t rockafellar s and f j luque s earlier results on the proximal point algorithm

complete characterizations of local weak sharp minima with applications to semi infinite optimization and complementarity abstract in this paper we identify a favorable class of nonsmooth functions for which local weak sharp minima can be completely characterized in terms of normal cones and subdifferentials or tangent cones and subderivatives or their mixture in finite dimensional spaces the results obtained not only extend previous ones in the literature but also allow us to provide new types of criteria for local weak sharpness applications of the developed theory are given to semi infinite programming and to a new class of semi infinite complementarity problems

on finite convergence and constraint identification of subgradient projection methods this paper deals with a continuous time subgradient projection algorithm shown to generate trajectories that accumulate to the solution set under a strong convexity assumption we show that convergence is exponential in norm a sharpness condition yields convergence in finite time and the necessary lapse is estimated invoking a constraint qualification and a non degeneracy assumption we demonstrate that optimally active constraints are identified in finite time

structure from motion without correspondence from tomographic projections by bayesian inversion theory in conventional tomography the interior of an object is reconstructed from tomographic projections such as x ray or transmission electron microscope images all the current reconstruction methods assume that projection geometry of the imaging device is either known or solved in advance by using e g fiducial or nonfiducial feature points in the images in this paper we propose a novel approach where the imaging geometry is solved simultaneously with the volume reconstruction problem while no correspondence information is needed our approach is a direct application of bayesian inversion theory and produces the maximum likelihood or maximum a posteriori estimates for the motion parameters under the selected noise and prior distributions in this paper the method is implemented for a two dimensional model problem with one dimensional affine projection data the performance of the method is tested with simulated and measured x ray projection data

a primal dual method for large scale image reconstruction in emission tomography in emission tomography images can be reconstructed from a set of measured projections using a maximum likelihood ml criterion in this paper we present a primal dual algorithm for large scale three dimensional image reconstruction the primal dual method is specialized to the ml reconstruction problem the reconstruction problem is extremely large in several of our data sets the hessian of the objective function is the product of a 1 4 million by 63 million matrix and its scaled transpose as such we consider only approaches that are suitable for large scale parallel computation we apply a stabilization technique to the system of equations for computing the primal direction and demonstrate the need for stabilization when approximately solving the system using an early terminated conjugate gradient iteration we demonstrate that the primal dual method for this problem converges faster than the logarithmic barrier method and considerably faster than the expectation maximization algorithm the use of extrapolation in conjunction with the primal dual method further reduces the overall computation required to achieve convergence

nondegenerate solutions and related concepts in affine variational inequalities the notion of a strictly complementary solution for complementarity problems is extended to that of a nondegenerate solution of variational inequalities several equivalent formulations of nondegeneracy are given in the affine case an existence theorem for a nondegenerate solution is given in terms of several related concepts which are shown to be equivalent in this context these include a weak sharp minimum the minimum principle sufficiency and error bounds the gap function associated with the variational inequality plays a central role in this existence theorem

preconditioned conjugate gradient algorithms for nonconvex problems with box constraints the paper describes new conjugate gradient algorithms for large scale nonconvex problems with box constraints in order to speed up convergence the algorithms employ scaling matrices which transform the space of original variables into the space in which hessian matrices of the problem s functionals have more clustered eigenvalues this is done by applying limited memory bfgs updating matrices once the scaling matrix is calculated the next few conjugate gradient iterations are performed in the transformed space the box constraints are treated efficiently by the projection we also present a limited memory quasi newton method which is a special version of our general algorithm the presented algorithms have strong global convergence properties in particular they identify constraints active at a solution in a finite number of iterations we believe that they are competitive to the l bfgs b method and present some numerical results which support our claim

stochastic quasi newton molecular simulations we report a new and efficient factorized algorithm for the determination of the adaptive compound mobility matrix b in a stochastic quasi newton method s qn that does not require additional potential evaluations for one dimensional and two dimensional test systems we previously showed that s qn gives rise to efficient configurational space sampling with good thermodynamic consistency c d chau g j a sevink and j g e m fraaije j chem phys 128 244110 2008 potential applications of s qn are quite ambitious and include structure optimization analysis of correlations and automated extraction of cooperative modes however the potential can only be fully exploited if the computational and memory requirements of the original algorithm are significantly reduced in this paper we consider a factorized mobility matrix b jj t and focus on the nontrivial fundamentals of an efficient algorithm for updating the noise multiplier j the new algorithm requires o n 2 multiplications per time step instead of the o n 3 multiplications in the original scheme due to choleski decomposition in a recursive form the update scheme circumvents matrix storage and enables limited memory implementation in the spirit of the well known limited memory broyden fletcher goldfarb shanno l bfgs method allowing for a further reduction of the computational effort to o n we analyze in detail the performance of the factorized fsu and limited memory l fsu algorithms in terms of convergence and multiscale sampling for an elementary but relevant system that involves multiple time and length scales finally we use this analysis to formulate conditions for the simulation of the complex high dimensional potential energy landscapes of interest

a new active set algorithm for box constrained optimization an active set algorithm asa for box constrained optimization is developed the algorithm consists of a nonmonotone gradient projection step an unconstrained optimization step and a set of rules for branching between the two steps global convergence to a stationary point is established for a nondegenerate stationary point the algorithm eventually reduces to unconstrained optimization without restarts similarly for a degenerate stationary point where the strong second order sufficient optimality condition holds the algorithm eventually reduces to unconstrained optimization without restarts a specific implementation of the asa is given which exploits the recently developed cyclic barzilai borwein cbb algorithm for the gradient projection step and the recently developed conjugate gradient algorithm cg descent for unconstrained optimization numerical experiments are presented using box constrained problems in the cuter and minpack 2 test problem libraries

a robust sequential quadratic programming method the sequential quadratic programming method developed by wilson han and powell may fail if the quadratic programming subproblems become infeasible or if the associated sequence of search directions is unbounded this paper considers techniques which circumvent these difficulties by modifying the structure of the constraint region in the quadratic programming subproblems furthermore questions concerning the occurrence of an unbounded sequence of multipliers and problem feasibility are also addressed

the barzilai and borwein gradient method for the large scale unconstrained minimization problem the barzilai and borwein gradient method for the solution of large scale unconstrained minimization problems is considered this method requires few storage locations and very inexpensive computations furthermore it does not guarantee descent in the objective function and no line search is required recently the global convergence for the convex quadratic case has been established however for the nonquadratic case the method needs to be incorporated in a globalization scheme in this work a nonmonotone line search strategy that guarantees global convergence is combined with the barzilai and borwein method this strategy is based on the nonmonotone line search technique proposed by grippo lampariello and lucidi siam j numer anal 23 1986 pp 707 716 numerical results to compare the behavior of this method with recent implementations of the conjugate gradient method are presented these results indicate that the global barzilai and borwein method may allow some significant reduction in the number of line searches and also in the number of gradient evaluations

a globally convergent lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds we consider the global and local convergence properties of a class of lagrangian barrier methods for solving nonlinear programming problems in such methods simple bound constraints may be treated separately from more general constraints the objective and general constraint functions are combined in a lagrangian barrier function a sequence of such functions are approximately minimized within the domain defined by the simple bounds global convergence of the sequence of generated iterates to a first order stationary point for the original problem is established furthermore possible numerical difficulties associated with barrier function methods are avoided as it is shown that a potentially troublesome penalty parameter is bounded away from zero this paper is a companion to previous work of ours on augmented lagrangian methods

on the accurate identification of active set for constrained minimax problems abstract in this paper the problem of identifying the active constraints for constrained nonlinear programming and minimax problems at an isolated local solution is discussed the correct identification of active constraints can improve the local convergence behavior of algorithms and considerably simplify algorithms for inequality constrained problems so it is a useful adjunct to nonlinear optimization algorithms facchinei et al f facchinei a fischer c kanzow on the accurate identification of active constraints siam j optim 9 1998 14 32 introduced an effective technique which can identify the active set in a neighborhood of a solution for nonlinear programming in this paper we first improve this conclusion to be more suitable for infeasible algorithms such as the strongly sub feasible direction method and the penalty function method then we present the identification technique of active constraints for constrained minimax problems without strict complementarity and linear independence some numerical results illustrating the identification technique are reported

active set newton methods and partial smoothness diverse optimization algorithms correctly identify in finite time intrinsic constraints that must be active at optimality analogous behavior extends beyond optimization to systems involving part

modified fixed point equations and related iterative methods for variational inequalities abstract in this paper we study the equivalence characterizations of several modified fixed point equations to variational inequalities vi based on these equations we give some applications in constructing iterative methods for the solution of the vi especially we show global convergence the sublinear convergence and the finite termination of a new iterative algorithm under certain conditions

weak sharp minima on riemannian manifolds this is the first paper dealing with the study of weak sharp minima for constrained optimization problems on riemannian manifolds which are important in many applications we consider the notions of local weak sharp minima boundedly weak sharp minima and global weak sharp minima for such problems and establish their complete characterizations in the case of convex problems on finite dimensional riemannian manifolds and hadamard manifolds a number of the results obtained in this paper are also new for the case of conventional problems in finite dimensional euclidean spaces our methods involve appropriate tools of variational analysis and generalized differentiation on riemannian and hadamard manifolds developed and efficiently implemented in this paper

shape based regularization of electron tomographic reconstruction we introduce a tomographic reconstruction method implemented using a shape based regularization technique spatial models of known features in the structure being reconstructed are integrated into the reconstruction process as regularizers our regularization scheme is driven locally through shape information obtained from segmentation and compared with a known spatial model we demonstrated our method on tomography data from digital phantoms simulated data and experimental electron tomography et data of virus complexes our reconstruction showed reduced blurring and an improvement in the resolution of the reconstructed volume was also measured this method also produced improved demarcation of spike boundaries in viral membranes when compared with popular techniques like weighted back projection and the algebraic reconstruction technique improved et reconstructions will provide better structure elucidation and improved feature visualization which can aid in solving key biological issues our method can also be generalized to other tomographic modalities

an inequality constrained nonlinear kalman bucy smoother by interior point likelihood maximization kalman bucy smoothers are often used to estimate the state variables as a function of time in a system with stochastic dynamics and measurement noise this is accomplished using an algorithm for which the number of numerical operations grows linearly with the number of time points all of the randomness in the model is assumed to be gaussian including other available information for example a bound on one of the state variables is non trivial because it does not fit into the standard kalman bucy smoother algorithm in this paper we present an interior point method that maximizes the likelihood with respect to the sequence of state vectors satisfying inequality constraints the method obtains the same decomposition that is normally obtained for the unconstrained kalman bucy smoother hence the resulting number of operations grows linearly with the number of time points we present two algorithms the first is for the affine case and the second is for the nonlinear case neither algorithm requires the optimization to start at a feasible sequence of state vector values both the unconstrained affine and unconstrained nonlinear kalman bucy smoother are special cases of the class of problems that can be handled by these algorithms

globally convergent algorithms for maximum a posteriori transmission tomography this paper reviews and compares three maximum likelihood algorithms for transmission tomography one of these algorithms is the em algorithm one is based on a convexity argument devised by de pierro see ieee trans med imaging vol 12 p 328 333 1993 in the context of emission tomography and one is an ad hoc gradient algorithm the algorithms enjoy desirable local and global convergence properties and combine gracefully with bayesian smoothing priors preliminary numerical testing of the algorithms on simulated data suggest that the convex algorithm and the ad hoc gradient algorithm are computationally superior to the em algorithm this superiority stems from the larger number of exponentiations required by the em algorithm the convex and gradient algorithms are well adapted to parallel computing

generic minimizing behavior in semialgebraic optimization we present a theorem of sard type for semialgebraic set valued mappings whose graphs have dimension no larger than that of their range space the inverse of such a mapping admits a single valued analytic localization around any pair in the graph for a generic value parameter this simple result yields a transparent and unified treatment of generic properties of semialgebraic optimization problems typical semialgebraic problems have finitely many critical points around each of which they admit a unique active manifold analogue of an active set in nonlinear optimization moreover such critical points satisfy strict complementarity and second order sufficient conditions for optimality are indeed necessary

interior point methodology for 3 d pet reconstruction interior point methods have been successfully applied to a wide variety of linear and nonlinear programming applications this paper presents a class of algorithms based on path following interior point methodology for performing regularized maximum likelihood ml reconstructions on three dimensional 3 d emission tomography data the algorithms solve a sequence of subproblems that converge to the regularized maximum likelihood solution from the interior of the feasible region the nonnegative orthant the authors propose two methods a primal method which updates only the primal image variables and a primal dual method which simultaneously updates the primal variables and the lagrange multipliers a parallel implementation permits the interior point methods to scale to very large reconstruction problems termination is based on well defined convergence measures namely the karush kuhn tucker first order necessary conditions for optimality the authors demonstrate the rapid convergence of the path following interior point methods using both data from a small animal scanner and monte carlo simulated data the proposed methods can readily be applied to solve the regularized weighted least squares reconstruction problem

convergence properties of minimization algorithms for convex constraints using a structured trust region in this paper we present a class of trust region algorithms for minimization problems within convex feasible regions in which the structure of the problem is explicitly used in the definition of the trust region this development is intended to reflect the possibility that some parts of the problem may be more accurately modelled than others a common occurrence in large scale nonlinear applications after describing the structured trust region mechanism we prove global convergence for all algorithms in our class

new projection type methods for monotone lcp with finite termination in this paper we establish two new projection type methods for the solution of monotone linear complementarity problem lcp the methods are a combination of the extragradient method and the newton method in which the active set strategy is used and only one linear system of equations with lower dimension is solved at each iteration it is shown that under the assumption of monotonicity these two methods are globally and linearly convergent furthermore under a nondegeneracy condition they have a finite termination property at last the methods are extended to solving monotone affine variational inequality problem

a linearly convergent dual based gradient projection algorithm for quadratically constrained convex minimization this paper presents a new dual formulation for quadratically constrained convex programs the special structure of the derived dual problem allows us to apply the gradient projection algorithm to produce a simple explicit method involving only elementary vector matrix operations proven to converge at a linear rate

termination and equivalence results for conjugate gradient algorithms this paper considers the problem of minimizing a functionf x when linear equality constraints onx could be present and when an algorithm is to be used which may employ both conjugate gradient and quasi newton steps the conjugate gradient algorithm is examined when a preconditioner is used which may be only positive semi definite quite a general theorem is given about the equivalence between conjugate gradient steps and quasi newton steps which includes several previously published results as special cases some results are also given for the case where some line searches are inexact

some remarks on conjugate gradient methods without line search abstract the conjugate gradient method is widely used in unconstrained optimization especially in case of large scale problems however the line search in the conjugate gradient method is sometimes very difficult or prohibitively expensive in sun and zhang j sun j p zhang global convergence of conjugate gradient methods without line search annals of operations research 103 2001 161 173 it is shown that by taking a fixed steplength k defined by the formula k g k t d k d k t q k d k the conjugate gradient method is globally convergent for several popular choices of k without line search in the simplest case all qk could be identity matrices however it would not even guarantee the descent property in this paper we study some methods to select qk which are based on the amount of descent and are superior to taking qk i the unit matrix

a modified expectation maximization algorithm for penalized likelihood estimation in emission tomography the maximum likelihood ml expectation maximization em approach in emission tomography has been very popular in medical imaging for several years in spite of this no satisfactory convergent modifications have been proposed for the regularized approach here a modification of the em algorithm is presented the new method is a natural extension of the em for maximizing likelihood with concave priors convergence proofs are given

generalized em type reconstruction algorithms for emission tomography we provide a general form for many reconstruction estimators of emission tomography these estimators include shepp and vardi s maximum likelihood ml estimator the quadratic weighted least squares wls estimator anderson s wls estimator and liu and wang s multi objective estimator and others we derive a generic update rule by constructing a surrogate function this work is inspired by the ml em em expectation maximization where the latter naturally arises as a special case a regularization with a specific form can also be incorporated by de pierro s trick we provide a general and quite different convergence proof compared with the proofs of the ml em and de pierro theoretical analysis shows that the proposed algorithm monotonically decreases the cost function and automatically meets nonnegativity constraints we have introduced a mechanism to provide monotonic self constraining and convergent algorithms from which some interesting existing and new algorithms can be derived simulation results illustrate the behavior of these algorithms in term of image quality and resolution noise tradeoff

on identification of active constraints ii the nonconvex case in this paper the results of burke and more 5 on the identification of active constraints are extended to the nonconvex constrained nonlinear programming problem the approach is motivated by the geometric structure of a certain polyhedral convex linearization of the constraint region at each iteration as in burke and more 5 questions of constraint identification are couched in terms of the faces of these polyhedra the main result employs a nondegeneracy condition due to dunn 7 and the linear independence condition to obtain a characterization of those algorithms that identify the optimal active constraints in a finite number of iterations the role of the linear independence condition is carefully examined and it is argued that it is required within the context of a first order theory of constraint identification in conclusion the characterization theorem is applied to the wilson han powell sequential quadratic programming algorithm j optim theory appl 22 1977 pp 297 309 proceedin

associative monotonic operations in fuzzy set theory abstract the properties of binary operations in a real interval are considered and used in the discussion of generalized operations on fuzzy sets on fuzzy numbers and on fuzzy probabilistic sets

bayesian reconstructions from emission tomography data using a modified em algorithm a novel method of reconstruction from single photon emission computerized tomography data is proposed this method builds on the expectation maximization em approach to maximum likelihood reconstruction from emission tomography data but aims instead at maximum posterior probability estimation which takes account of prior belief about smoothness in the isotope concentration a novel modification to the em algorithm yields a practical method the method is illustrated by an application to data from brain scans

on the convergence of a new trust region algorithm in this paper we present a new trust region algorithm for general nonlinear constrained optimization problems the algorithm is based on the l infty exact penalty function under very mild conditions global convergence results for the algorithm are given local convergence properties are also studied it is shown that the penalty parameter generated by the algorithm will be eventually not less than the l 1 norm of the lagrange multipliers at the accumulation point it is proved that the method is equivalent to the sequential quadratic programming method for all large k hence superlinearly convergent results of the sqp method can be applied numerical results are also reported

a proximal method for composite minimization we consider minimization of functions that are compositions of convex or prox regular functions possibly extended valued with smooth vector functions a wide variety of important optimization problems fall into this framework we describe an algorithmic framework based on a subproblem constructed from a linearized approximation to the objective and a regularization term properties of local solutions of this subproblem underlie both a global convergence result and an identification property of the active manifold containing the solution of the original problem preliminary computational results on both convex and nonconvex examples are promising

a new proof of robinson s homeomorphism theorem for pl normal maps abstract we study a certain class of piecewise linear functions from r n to r n namely robinson s normal maps induced by linear mappings and polyhedral convex sets called pl normal maps robinson s homeomorphism theorem characterizes the pl normal maps which are homeomorphisms as those which have nonzero determinants of the same sign on all pieces of linearity this paper presents a new shorter proof of the result pl normal systems include many optimization and equilibrium problems they arise from variational inequalities or equivalently generalized equations specified by linear maps and polyhedral convex sets unique continuous solvability of these systems which is important in theory and computation is captured by the homeomorphism result

global convergence properties of conjugate gradient methods for optimization this paper explores the convergence of nonlinear conjugate gradient methods without restarts and with practical line searches the analysis covers two classes of methods that are globally convergent on smooth nonconvex functions some properties of the fletcher reeves method play an important role in the first family whereas the second family shares an important property with the polak ribiere method numerical experiments are presented

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

qn like variable storage conjugate gradients both conjugate gradient and quasi newton methods are quite successful at minimizing smooth nonlinear functions of several variables and each has its advantages in particular conjugate gradient methods require much less storage to implement than a quasi newton code and therefore find application when storage limitations occur they are however slower so there have recently been attempts to combine cg and qn algorithms so as to obtain an algorithm with good convergence properties and low storage requirements one such method is the code conmin due to shanno and phua it has proven quite successful but it has one limitation it has no middle ground in that it either operates as a quasi newton code using o n 2 storage locations or as a conjugate gradient code using 7n locations but it cannot take advantage of the not unusual situation where more than 7n locations are available but a quasi newton code requires an excessive amount of storage

on the convergence of an em type algorithm for penalized likelihood estimation in emission tomography recently we proposed an extension of the expectation maximization em algorithm that was able to handle regularization terms in a natural way although very general convergence proofs were not valid for many possibly useful regularizations we present here a simple convergence result that is valid assuming only continuous differentiability of the penalty term and can be also extended to other methods for penalized likelihood estimation in tomography

algorithms for bound constrained quadratic programming problems we present an algorithm which combines standard active set strategies with the gradient projection method for the solution of quadratic programming problems subject to bounds we show in particular that if the quadratic is bounded below on the feasible set then termination occurs at a stationary point in a finite number of iterations moreover if all stationary points are nondegenerate termination occurs at a local minimizer a numerical comparison of the algorithm based on the gradient projection algorithm with a standard active set strategy shows that on mildly degenerate problems the gradient projection algorithm requires considerable less iterations and time than the active set strategy on nondegenerate problems the number of iterations typically decreases by at least a factor of 10 for strongly degenerate problems the performance of the gradient projection algorithm deteriorates but it still performs better than the active set method

tangent projection equations and general variational inequalities in this paper we establish the equivalence between the general variational inequalities and tangent projection equations this equivalence is used to discuss the local convergence analysis of a wide class of iterative methods for solving the general variational inequalities we show that some existing methods can identify the optimal face after finitely many iterations under the degenerate assumption

the enhancement of the cell based gis analyses with fuzzy processing capabilities in order to store and process natural phenomena in geographic information systems gis it is necessary to model the real world to form computational representation since classical set theory is used in conventional gis softwares to model uncertain real world the natural variability in the environmental phenomena cannot be modeled appropriately because pervasive imprecision of the real world is unavoidably reduced to artificially precise spatial entities when the conventional crisp logic is used for modeling an alternative approach is the fuzzy set theory which provides a formal framework to represent and reason with uncertain information in addition linguistic variable concept in a fuzzy logic system is useful for communicating concepts and knowledge with human beings fuzzycell is a system designed and implemented to enhance commercial gis software namely arcmap r with fuzzy set theory fuzzycell allows users to a incorporate human knowledge and experience in the form of linguistically defined variables into gis based spatial analyses b handle imprecision in the decision making processes and c approximate complex ill defined problems in decision making processes and classification it provides eight membership functions inference methods methods for rule aggregation operators for set operations and methods for defuzzification the operation of fuzzycell is presented through case studies which demonstrate its application for classification and decision making processes this paper shows how fuzzy logic approach may contribute to a better representation and reasoning with imprecise concepts which are inherent characteristics of geographic data stored and processed in gis

a fast algorithm for sparse reconstruction based on shrinkage subspace optimization and continuation we propose a fast algorithm for solving the ell 1 regularized minimization problem min x in mathbb r n mu x 1 ax b 2 2 for recovering sparse solutions to an undetermined system of linear equations ax b the algorithm is divided into two stages that are performed repeatedly in the first stage a first order iterative shrinkage method yields an estimate of the subset of components of x likely to be nonzero in an optimal solution restricting the decision variables x to this subset and fixing their signs at their current values reduces the ell 1 norm x 1 to a linear function of x the resulting subspace problem which involves the minimization of a smaller and smooth quadratic function is solved in the second phase our code fpc as embeds this basic two stage algorithm in a continuation homotopy approach by assigning a decreasing sequence of values to mu this code exhibits state of the art performance in terms of both its speed and its ability to recover sparse signals

accelerated penalized weighted least squares and maximum likelihood algorithms for reconstructing transmission images from pet transmission data we present penalized weighted least squares pwls and penalized maximum likelihood pml methods for reconstructing transmission images from positron emission tomography transmission data first we view the problem of minimizing the weighted least squares wls and maximum likelihood objective functions as a sequence of nonnegative least squares minimization problems this viewpoint follows from using certain quadratic functions as surrogate functions for the wls and maximum likelihood objective functions second we construct surrogate functions for a class of penalty functions that yield closed form expressions for the iterates of the pwls and pml algorithms due to the slow convergence of the pwls and pml algorithms accelerated versions of them are developed that are theoretically guaranteed to monotonically decrease their respective objective functions in experiments using real phantom data the pml images produced the most accurate attenuation correction factors on the other hand the pwls images produced images with the highest levels of contrast for low count data

a unified approach to statistical tomography using coordinate descent optimization over the past years there has been considerable interest in statistically optimal reconstruction of cross sectional images from tomographic data in particular a variety of such algorithms have been proposed for maximum a posteriori map reconstruction from emission tomographic data while map estimation requires the solution of an optimization problem most existing reconstruction algorithms take an indirect approach based on the expectation maximization em algorithm we propose a new approach to statistically optimal image reconstruction based on direct optimization of the map criterion the key to this direct optimization approach is greedy pixel wise computations known as iterative coordinate decent icd we propose a novel method for computing the icd updates which we call icd newton raphson we show that icd newton raphson requires approximately the same amount of computation per iteration as em based approaches but the new method converges much more rapidly in our experiments typically five to ten iterations other advantages of the icd newton raphson method are that it is easily applied to map estimation of transmission tomograms and typical convex constraints such as positivity are easily incorporated

generic optimality conditions for semialgebraic convex programs we consider linear optimization over a nonempty convex semialgebraic feasible region f semidefinite programming is an example if f is compact then for almost every linear objective there is a unique optimal solution lying on a unique active manifold around which f is partly smooth and the second order sufficient conditions hold perturbing the objective results in smooth variation of the optimal solution the active manifold consists locally of these perturbed optimal solutions it is independent of the representation of f and is eventually identified by a variety of iterative algorithms such as proximal and projected gradient schemes these results extend to unbounded sets f

on the relation between the isra and the em algorithm for positron emission tomography the image space reconstruction algorithm isra was proposed as a modification of the expectation maximization em algorithm based on physical considerations for application in volume emission computered tomography as a consequence of this modification isra searches for least squares solutions instead of maximizing poisson likelihoods as the em algorithm it is shown that both algorithms may be obtained from a common mathematical framework this fact is used to extend isra for penalized likelihood estimates

a new subspace limited memory bfgs algorithm for large scale bound constrained optimization an active set limited memory bfgs algorithm for large scale bound constrained optimization is introduced the active sets are based on guessing technique to be identified at each iteration the search direction in free subspace is determined by limited memory bfgs l bfgs algorithm which provides an efficient means for attacking large scale optimization problems the implementations of the method on cute test problems are described

some developments in general variational inequalities general variational inequalities provide us with a unified natural novel and simple framework to study a wide class of equilibrium problems arising in pure and applied sciences in this paper we present a number of new and known numerical techniques for solving general variational inequalities using various techniques including projection wiener hopf equations updating the solution auxiliary principle inertial proximal penalty function dynamical system and well posedness we also consider the local and global uniqueness of the solution and sensitivity analysis of the general variational inequalities as well as the finite convergence of the projection type algorithms our proofs of convergence are very simple as compared with other methods our results present a significant improvement of previously known methods for solving variational inequalities and related optimization problems since the general variational inequalities include quasi variational inequalities and quasi implicit complementarity problems as special cases results presented here continue to hold for these problems several open problems have been suggested for further research in these areas

projected newton methods for optimization problems with simple constraints we consider the problem min f x x geqq 0 and propose algorithms of the form x k 1 x k alpha k d k nabla f x k where cdot denotes projection on the positive orthant alpha k is a stepsize chosen by an armijo like rule and d k is a positive definite symmetric matrix which is partly diagonal we show that d k can be calculated simply on the basis of second derivatives of f so that the resulting newton like algorithm has a typically superlinear rate of convergence with other choices of d k convergence at a typically linear rate is obtained the algorithms are almost as simple as their unconstrained counterparts they are well suited for problems of large dimension such as those arising in optimal control while being competitive with existing methods for low dimensional problems the effectiveness of the newton like algorithm is demonstrated via computational examples involving as many as 10 000 variables extensions to general linearly constrained pr

on the accurate identification of active constraints we consider nonlinear programs with inequality constraints and we focus on the problem of identifying those constraints which will be active at an isolated local solution the correct identification of active constraints is important from both a theoretical and a practical point of view such an identification removes the combinatorial aspect of the problem and locally reduces the inequality constrained minimization problem to an equality constrained problem which can be more easily dealt with we present a new technique which identifies active constraints in a neighborhood of a solution and which requires neither complementary slackness nor uniqueness of the multipliers we also present extensions to variational inequalities and numerical examples illustrating the identification technique

active set identification for linearly constrained minimization without explicit derivatives we consider active set identification for linearly constrained optimization problems in the absence of explicit information about the derivative of the objective function we begin by presenting some general results on active set identification that are not tied to any particular algorithm these general results are sufficiently strong that given a sequence of iterates converging to a karush kuhn tucker point it is possible to identify binding constraints for which there are nonzero multipliers we then focus on generating set search methods a class of derivative free direct search methods we discuss why these general results which are posed in terms of the direction of steepest descent apply to generating set search even though these methods do not have explicit recourse to derivatives nevertheless there is a clearly identifiable subsequence of iterations at which we can reliably estimate the set of constraints that are binding at a solution we discuss how active set estimation can be used to accelerate generating set search methods and illustrate the appreciable improvement that can result using several examples from the cuter test suite we also introduce two algorithmic refinements for generating set search methods the first expands the subsequence of iterations at which we can make inferences about stationarity the second is a more flexible step acceptance criterion