semi infinite programming theory methods and applications starting from a number of motivating and abundant applications in 2 including control of robots eigenvalue computations mechanical stress of materials and statistical design the authors describe a class of optimization problems which are referred to as semi infinite because their constraints bound functions of a finite number of variables on a whole region in 3 5 first and second order optimality conditions are derived for general nonlinear problems as well as a procedure for reducing the problem locally to one with only finitely many constraints another main effort for achieving simplification is through duality in 6 there algebraic properties of finite linear programming are brought to bear on duality theory in semi infinite programming section 7 treats numerical methods based on either discretization or local reduction with the emphasis on the design of superlinearly convergent sqp type methods taking this differentiable point of view this paper can be considered to be complementar

abnormal equality constrained optimization problems sensitivity theory for the equality constrained optimization problem we consider the case when the customary regularity of constraints can be violated under the assumptions substantially weaker than those previously used in the literature we develop a reasonably complete local sensitivity theory for this class of problems including upper and lower bounds for the rate of change of the optimal value function subject to parametric perturbations as well as the estimates and the description of asymptotic behavior of solutions of the perturbed problems

multiaccess quantum channels shared communication channels are subject to multiple access interference transmitter and receiver design techniques that explicitly deal with this interference have been shown to improve substantially the performance of communication systems over radio frequency and other classical channels quantum multiple access communication channels on the other hand have received comparatively little attention in this paper an input output model for multiple access quantum channels relevant to optical communications is proposed the model accounts for multiaccess interference signal attenuation and random noise and can be used in the analysis and design of communication systems using a result from optimization a perturbation method is developed to find the minimum achievable error probability in small interference channels it is shown that the quantum measurement that minimizes the error probability in a no interference channel is robust in the presence of small multiaccess interference the results are illustrated with numerical examples which show that optimal quantum detectors can significantly outperform conventional detectors even for moderate levels of crosstalk

nondegeneracy and quantitative stability of parameterized optimization problems with multiple solutions this paper presents some results on quantitative stability of optimal solutions of parameterized optimization problems having nonisolated optima the analysis is based on the introduced concept of nondegeneracy which may be of independent interest examples of nonlinear and semidefinite programming are discussed

directional differentiability of the optimal value function in convex semi infinite programming in this paper directional differentiability properties of the optimal value function of a parameterized semi infinite programming problem are studied it is shown that if the unperturbed semi infinite programming problem is convex then the corresponding optimal value function is directionally differentiable under mild regularity assumptions a max min formula for the directional derivatives well known in the finite convex case is given

stability of elliptic optimal control problems abstract we consider optimal problems governed by elliptical equations on parameters and give sufficient conditions for the continuous dependence of the solutions on the parameters the techniques are based on variational methods

first and second order analysis of nonlinear semidefinite programs in this paper we study nonlinear semidefinite programming problems convexity duality and first order optimality conditions for such problems are presented a second order analysis is also given second order necessary and sufficient optimality conditions are derived finally sensitivity analysis of such programs is discussed

perturbed optimization in banach spaces i a general theory based on a weak directional constraint qualification using a directional form of constraint qualification weaker than robinson s we derive an implicit function theorem for inclusions and use it for first and second order sensitivity analyses of the value function in perturbed constrained optimization we obtain holder and lipschitz properties and under a no gap condition first order expansions for exact and approximate solutions as an application differentiability properties of metric projections in hilbert spaces are obtained using a condition generalizing polyhedricity we also present in the appendix a short proof of a generalization of the convex duality theorem in banach spaces

on sensitivity analysis of nonlinear programs in banach spaces the approach via composite unconstrained optimization this article studies local sensitivity analysis of nonlinear programming problems in banach spaces with arbitrary sets of solutions and no a priori regularity properties this is done by first developing a corresponding theory for unconstrained optimization involving simple composite functions and then applying this theory to general nonlinear programs by way of certain reduction principles

building sparse multiple kernel svm classifiers the support vector machines svms have been very successful in many machine learning problems however they can be slow during testing because of the possibly large number of support vectors obtained recently wu 2005 proposed a sparse formulation that restricts the svm to use a small number of expansion vectors in this paper we further extend this idea by integrating with techniques from multiple kernel learning mkl the kernel function in this sparse svm formulation no longer needs to be fixed but can be automatically learned as a linear combination of kernels two formulations of such sparse multiple kernel classifiers are proposed the first one is based on a convex combination of the given base kernels while the second one uses a convex combination of the so called ldquoequivalentrdquo kernels empirically the second formulation is particularly competitive experiments on a large number of toy and real world data sets show that the resultant classifier is compact and accurate and can also be easily trained by simply alternating linear program and standard svm solver

optimality conditions for irregular inequality constrained problems we consider feasible sets given by conic constraints where the cone defining the constraints is convex with nonempty interior we study the case where the feasible set is not assumed to be regular in the classical sense of robinson and obtain a constructive description of the tangent cone under a certain new second order regularity condition this condition contains classical regularity as a special case while being weaker when constraints are twice differentiable assuming that the cone defining the constraints is finitely generated we also derive a special form of primal dual optimality conditions for the corresponding constrained optimization problem our results subsume optimality conditions for both the classical regular and second order regular cases while still being meaningful in the more general setting in the sense that the multiplier associated with the objective function is nonzero

scalar curvature equation on sn part ii analytic characterizations this is the second part of a series devoting to the study of the prescribing scalar curvature problem on the standard sphere of any dimension by studying topological degrees for certain abstract maps we will give explicit analytic conditions on the scalar curvature function which verify the topological degree conditions given in the first part of the series to ensure the solvability of the problem general existence results for the prescribing scalar curvature equation will be given on both h symmetric and sub h symmetric solutions corresponding to h symmetric scalar curvature functions as well as on non symmetric solutions corresponding to symmetric like scalar curvature functions special axisymmetric and axisymmetric like cases will be also considered our analysis will be based on a general approach of dimension reductions and degree calculations by taking advantage of symmetries and symmetric like properties

weak sharp minima in mathematical programming the notion of a sharp or strongly unique minimum is extended to include the possibility of a nonunique solution set these minima will be called weak sharp minima conditions necessary for the solution set of a minimization problem to be a set of weak sharp minima are developed in both the unconstrained and constrained cases these conditions are also shown to be sufficient under the appropriate convexity hypotheses the existence of weak sharp minima is characterized in the cases of linear and quadratic convex programming and for the linear complementarity problem in particular a result of mangasarian and meyer is reproduced that shows that the solution set of a linear program is always a set of weak sharp minima whenever it is nonempty consequences for the convergence theory of algorithms are also examined especially conditions yielding finite termination

conjugate duality in set valued vector optimization abstract in this paper conjugate duality results for convexlike set valued vector optimization problems are presented under closedness or boundedness hypotheses some properties of the value mapping of a set valued vector optimization problem are studied a conjugate duality result is also proved for a convex set valued vector optimization problem without the requirements of closedness and boundedness

a family of simple non parametric kernel learning algorithms previous studies of non parametric kernel learning npkl usually formulate the learning task as a semi definite programming sdp problem that is often solved by some general purpose sdp solvers however for n data examples the time complexity of npkl using a standard interior point sdp solver could be as high as o n6 5 which prohibits npkl methods applicable to real applications even for data sets of moderate size in this paper we present a family of efficient npkl algorithms termed simplenpkl which can learn non parametric kernels from a large set of pairwise constraints efficiently in particular we propose two efficient simplenpkl algorithms one is simplenpkl algorithm with linear loss which enjoys a closed form solution that can be efficiently computed by the lanczos sparse eigen decomposition technique another one is simplenpkl algorithm with other loss functions including square hinge loss hinge loss square loss that can be re formulated as a saddle point optimization problem which can be further resolved by a fast iterative algorithm in contrast to the previous npkl approaches our empirical results show that the proposed new technique maintaining the same accuracy is significantly more efficient and scalable finally we also demonstrate that the proposed new technique is also applicable to speed up many kernel learning tasks including colored maximum variance unfolding minimum volume embedding and structure preserving embedding

variational analysis of a composite function a formula for the lower second order epi derivative an exact formula is established for the lower second order epi derivative of a function of the form g f x where f is a smooth map from one banach space into another and g is a convex function generally not everywhere finite unconstrained minimization of such functions typically arise as an equivalent in one or another sense reduction form for many important classes of constrained optimization problems the formula is further applied to study epi differentiability of the max function x max q x q q

sufficient optimality conditions for optimal control subject to state constraints strong second order sufficient optimality conditions ssc are derived for optimal control problems of systems described by nonlinear odes subject to mixed control state and pure state constraints the obtained ssc are expressed in terms of a modified legendre clebsch condition and the associated riccati equation the role of ssc in stability analysis of solutions to parametric optimal control problems is briefly discussed

second order sensitivity analysis and asymptotic theory of parametrized nonlinear programs in this paper we study second order differential properties of an optimal value function x it is shown that under certain conditions x possesses second order directional derivatives which can be calculated by solving corresponding quadratic programs also upper and lower bounds on these derivatives are introduced under weaker assumptions in particular we show that the second order directional derivative is infinite if the corresponding quadratic program is unbounded finally sensitivity results are applied to investigate asymptotics of estimators in parametrized nonlinear programs

stability and generalization we define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave one out error the methods we use can be applied in the regression framework as well as in the classification one when the classifier is obtained by thresholding a real valued function we study the stability properties of large classes of learning algorithms such as regularization based algorithms in particular we focus on hilbert space regularization and kullback leibler regularization we demonstrate how to apply the results to svm for regression and classification

a statistical multiscale framework for poisson inverse problems this paper describes a statistical multiscale modeling and analysis framework for linear inverse problems involving poisson data the framework itself is founded upon a multiscale analysis associated with recursive partitioning of the underlying intensity a corresponding multiscale factorization of the likelihood induced by this analysis and a choice of prior probability distribution made to match this factorization by modeling the splits in the underlying partition the class of priors used here has the interesting feature that the noninformative member yields the traditional maximum likelihood solution other choices are made to reflect prior belief as to the smoothness of the unknown intensity adopting the expectation maximization em algorithm for use in computing the maximum a posteriori map estimate corresponding to our model we find that our model permits remarkably simple closed form expressions for the em update equations the behavior of our em algorithm is examined and it is shown that convergence to the global map estimate can be guaranteed applications in emission computed tomography and astronomical energy spectral analysis demonstrate the potential of the new approach

quantitative stability of variational systems ii a framework for nonlinear conditioning stability results of lipschitz and holder type are obtained for the solutions and optimal values of optimization problems when perturbations are measured in terms of the rho epi distance

efficient methods for overlapping group lasso the group lasso is an extension of the lasso for feature selection on predefined nonoverlapping groups of features the nonoverlapping group structure limits its applicability in practice there have been several recent attempts to study a more general formulation where groups of features are given potentially with overlaps between the groups the resulting optimization is however much more challenging to solve due to the group overlaps in this paper we consider the efficient optimization of the overlapping group lasso penalized problem we reveal several key properties of the proximal operator associated with the overlapping group lasso and compute the proximal operator by solving the smooth and convex dual problem which allows the use of the gradient descent type of algorithms for the optimization our methods and theoretical results are then generalized to tackle the general overlapping group lasso formulation based on the eq norm we further extend our algorithm to solve a nonconvex overlapping group lasso formulation based on the capped norm regularization which reduces the estimation bias introduced by the convex penalty we have performed empirical evaluations using both a synthetic and the breast cancer gene expression dataset which consists of 8 141 genes organized into overlapping gene sets experimental results show that the proposed algorithm is more efficient than existing state of the art algorithms results also demonstrate the effectiveness of the nonconvex formulation for overlapping group lasso

leave one out bounds for support vector regression model selection minimizing bounds of leave one out errors is an important and efficient approach for support vector machine svm model selection past research focuses on their use for classification but not regression in this letter we derive various leave one out bounds for support vector regression svr and discuss the difference from those for classification experiments demonstrate that the proposed bounds are competitive with bayesian svr for parameter selection we also discuss the differentiability of leave one out bounds

sensitivity analysis of optimization problems under second order regular constraints we present a perturbation theory for finite dimensional optimization problems subject to abstract constraints satisfying a second order regularity condition this is a technical condition that is always satisfied in the case of semi definite optimization we derive lipschitz and holder expansions of approximate optimal solutions under a directional constraint qualification hypothesis and various second order sufficient conditions that take into account the curvature of the set defining the constraints of the problem we show how the theory applies to semi infinite programs in which the contact set is a smooth manifold and the quadratic growth condition in the constraint space holds and discuss the differentiability of metric projections as well as the moreau yosida regularization finally we show how the theory applies to semi definite optimization

on uniqueness of lagrange multipliers in optimization problems subject to cone constraints in this paper we study uniqueness of lagrange multipliers in optimization problems subject to cone constraints the main tool in our investigation of this question will be a calculus of dual polar cones we give sufficient and in some cases necessary conditions for uniqueness of lagrange multipliers in general banach spaces general results are then applied to two particular examples of the semidefinite and semi infinite programming problems respectively

on lipschitzian stability of optimal solutions of parametrized semi infinite programs we study continuity properties of optimal solutions of parametrized semi infinite programming problems the involved constraints are formulated in a form of cone constraints and then a slightly modified general result of shapiro and bonnans 1992 on lipschitzian stability of optimal solutions is applied it is shown that under certain second order sufficient conditions optimal solutions of the semi infinite programs are lipschitzian stable provided a regularity assumption related to a linearization of the considered programs is satisfied

on the marginal function in nonlinear programming for several types of finite or infinite dimensional optimization problems the marginal function or optimal value function is characterized by different local approximations such as generalized gradients generalized directional derivatives directional hadamard or dini derivatives we give estimates for these terms which are determined by multipliers satisfying necessary optimality conditions when the functions which define the optimization problem are more than once continuously differentiable then higher order necessary conditions are employed to obtain refined estimates for the marginal function as a by product we give a new equivalent formulation of clarke s multiplier rule for nonsmooth optimization problems this shows that the set of all multipliers satisfying these necessary conditions is the union of a finite number of closed convex cones

the riccati equation for optimal control problems with mixed state control constraints necessity and sufficiency the goal of this paper is to conduct a complete study of second order conditions for the optimal control problem with mixed state control constraints the conjugate point theory is presented and a necessary condition in terms of the corresponding riccati equation is obtained sufficiency criteria are developed in terms of strengthened necessary conditions including the riccati equation the results generalize the known ones for pure control constraints as well as for the mixed state control constraints

pseudopower expansion of solutions of generalized equations and constrained optimization problems we show that the solution of a strongly regular generalized equation subject to a scalar perturbation expands in pseudopower series in terms of the perturbation parameter i e the expansion of orderk is the solution of generalized equations expanded to orderk and thus depends itself on the perturbation parameter in the polyhedral case this expansion reduces to a usual taylor expansion these results are applied to the problem of regular perturbation in constrained optimization we show that if the strong regularity condition is satisfied the property of quadratic growth holds and at least locally the solutions of the optimization problem and of the associated optimality system coincide if in addition the number of inequality constraints is finite the solution and the lagrange multiplier can be expanded in taylor series if the data are analytic the solution and the multiplier are analytic functions of the perturbation parameter

second order sufficiency and quadratic growth for nonisolated minima for standard nonlinear programming problems the weak second order sufficient condition is equivalent to the quadratic growth condition as far as the set of minima consists of isolated points and some qualification hypothesis holds this kind of condition is instrumental in the study of numerical algorithms and sensitivity analysis the arm of the paper is to study the relations between various types of sufficient conditions and quadratic growth in cases when the set of minima may have nonisolated points

generalized semi infinite programming a tutorial this tutorial presents an introduction to generalized semi infinite programming gsip which in recent years became a vivid field of active research in mathematical programming a gsip problem is characterized by an infinite number of inequality constraints and the corresponding index set depends additionally on the decision variables there exist a wide range of applications which give rise to gsip models some of them are discussed in the present paper furthermore geometric and topological properties of the feasible set and in particular the difference to the standard semi infinite case are analyzed by using first order approximations of the feasible set corresponding constraint qualifications are developed then necessary and sufficient first and second order optimality conditions are presented where directional differentiability properties of the optimal value function of the so called lower level problem are used finally an overview of numerical methods is given

viscosity solutions of minimization problems viscosity methods for minimization problems are revisited from some modern perspectives in variational analysis variational convergences for sequences of functions epi convergence gamma convergence mosco convergence and for sequences of operators graph convergence provide a flexible tool for such questions it is proved in a rather large setting that the solutions of the approximate problems converge to a viscosity solution of the original problem that is a solution that is minimal among all the solutions with respect to some viscosity criteria various examples coming from mathematical programming calculus of variations semicoercive elliptic equations phase transition theory hamilton jacobi equations singular perturbations and optimal control theory are considered

optimization problems with perturbations a guided tour this paper presents an overview of some recent and significant progress in the theory of optimization problems with perturbations we put the emphasis on methods based on upper and lower estimates of the objective function of the perturbed problems these methods allow one to compute expansions of the optimal value function and approximate optimal solutions in situations where the set of lagrange multipliers is not a singleton may be unbounded or is even empty we give rather complete results for nonlinear programming problems and describe some extensions of the method to more general problems we illustrate the results by computing the equilibrium position of a chain that is almost vertical or horizontal