generalized bundle methods we study a class of generalized bundle methods for which the stabilizing term can be any closed convex function satisfying certain properties this setting covers several algorithms from the literature that have been so far regarded as distinct under a different hypothesis on the stabilizing term and or the function to be minimized we prove finite termination asymptotic convergence and finite convergence to an optimal point with or without limits on the number of serious steps and or requiring the proximal parameter to go to infinity the convergence proofs leave a high degree of freedom in the crucial implementative features of the algorithm i e the management of the bundle of subgradients beta strategy and of the proximal parameter t strategy we extensively exploit a dual view of bundle methods which are shown to be a dual ascent approach to one nonlinear problem in an appropriate dual space where nonlinear subproblems are approximately solved at each step with an inner linearization approach this allows us to precisely characterize the changes in the subproblems during the serious steps since the dual problem is not tied to the local concept of varepsilon subdifferential for some of the proofs a generalization of inf compactness called compactness is required this concept is related to that of asymptotically well behaved functions

an infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter global convergence in constrained optimization algorithms has traditionally been enforced by the use of parametrized penalty functions recently the filter strategy has been introduced as an alternative at least part of the motivation for using filter methods consists of avoiding the need for estimating a suitable penalty parameter which is often a delicate task in this paper we demonstrate that the use of a parametrized penalty function in nonsmooth convex optimization can be avoided without using the relatively complex filter methods we propose an approach which appears to be more direct and easier to implement in the sense that it is closer in spirit and structure to the well developed unconstrained bundle methods preliminary computational results are also reported

validation analysis of mirror descent stochastic approximation method the main goal of this paper is to develop accuracy estimates for stochastic programming problems by employing stochastic approximation sa type algorithms to this end we show that while running a mirror descent stochastic approximation procedure one can compute with a small additional effort lower and upper statistical bounds for the optimal objective value we demonstrate that for a certain class of convex stochastic programs these bounds are comparable in quality with similar bounds computed by the sample average approximation method while their computational cost is considerably smaller

the efficiency of ballstep subgradient level methods for convex optimization we study subgradient methods for convex optimization that use projections onto successive approximations of level sets of the objective corresponding to estimates of the optimal value we establish convergence and efficiency estimates for simple ballstep level controls without requiring that the feasible set be compact our framework may handle accelerations based on cheap projections surrogate constraints and conjugate subgradient techniques

a trust region spectral bundle method for nonconvex eigenvalue optimization we present a nonsmooth optimization technique for nonconvex maximum eigenvalue functions and for nonsmooth functions which are infinite maxima of eigenvalue functions we prove global convergence of our method in the sense that for an arbitrary starting point every accumulation point of the sequence of iterates is critical the method is tested on several problems in feedback control synthesis

a doubly stabilized bundle method for nonsmooth convex optimization we propose a bundle method for minimizing nonsmooth convex functions that combines both the level and the proximal stabilizations most bundle algorithms use a cutting plane model of the objective function to formulate a subproblem whose solution gives the next iterate proximal bundle methods employ the model in the objective function of the subproblem while level methods put the model in the subproblem s constraints the proposed algorithm defines new iterates by solving a subproblem that employs the model in both the objective function and in the constraints one advantage when compared to the proximal approach is that the level set constraint provides a certain lagrange multiplier which is used to update the proximal parameter in a novel manner we also show that in the case of inexact function and subgradient evaluations no additional procedure needs to be performed by our variant to deal with inexactness as opposed to the proximal bundle methods that require special modifications numerical experiments on almost 1 000 instances of different types of problems are presented our experiments show that the doubly stabilized bundle method inherits useful features of the level and the proximal versions and compares favorably to both of them

an aggregate subgradient method for nonsmooth convex minimization a class of implementable algorithms is described for minimizing any convex not necessarily differentiable functionf of several variables the methods require only the calculation off and one subgradient off at designated points they generalize lemarechal s bundle method more specifically instead of using all previously computed subgradients in search direction finding subproblems that are quadratic programming problems the methods use an aggregate subgradient which is recursively updated as the algorithms proceed each algorithm yields a minimizing sequence of points and iff has any minimizers then this sequence converges to a solution of the problem particular members of this algorithm class terminate whenf is piecewise linear the methods are easy to implement and have flexible storage requirements and computational effort per iteration that can be controlled by a user

biorthogonal systems and quotient spaces via baire category methods we show that every banach space x of density smaller that the baire category number open image in new window admits a quotient with a long schauder basis that can be taken of length 1 if x is not separable so assuming that the baire category number open image in new window does not take its minimal possible value a banach space x is separable if and only if all biorthogonal systems of x are countable

an optimal variant of kelley s cutting plane method we propose a new variant of kelley s cutting plane method for minimizing a nonsmooth convex lipschitz continuous function over the euclidean space we derive the method through a constructive approach and prove that it attains the optimal rate of convergence for this class of problems

computational experience with a bundle approach for semidefinite cutting plane relaxations of max cut and equipartition we propose a dynamic version of the bundle method to get approximate solutions to semidefinite programs with a nearly arbitrary number of linear inequalities our approach is based on lagrangian duality where the inequalities are dualized and only a basic set of constraints is maintained explicitly this leads to function evaluations requiring to solve a relatively simple semidefinite program our approach provides accurate solutions to semidefinite relaxations of the max cut and the equipartition problem which are not achievable by direct approaches based only on interior point methods

bundle level type methods uniformly optimal for smooth and nonsmooth convex optimization the main goal of this paper is to develop uniformly optimal first order methods for convex programming cp by uniform optimality we mean that the first order methods themselves do not require the input of any problem parameters but can still achieve the best possible iteration complexity bounds by incorporating a multi step acceleration scheme into the well known bundle level method we develop an accelerated bundle level method and show that it can achieve the optimal complexity for solving a general class of black box cp problems without requiring the input of any smoothness information such as whether the problem is smooth nonsmooth or weakly smooth as well as the specific values of lipschitz constant and smoothness level we then develop a more practical restricted memory version of this method namely the accelerated prox level apl method we investigate the generalization of the apl method for solving certain composite cp problems and an important class of saddle point problems recently studied by nesterov math program 103 127 152 2005 we present promising numerical results for these new bundle level methods applied to solve certain classes of semidefinite programming and stochastic programming problems

a bundle filter method for nonsmooth convex constrained optimization for solving nonsmooth convex constrained optimization problems we propose an algorithm which combines the ideas of the proximal bundle methods with the filter strategy for evaluating candidate points the resulting algorithm inherits some attractive features from both approaches on the one hand it allows effective control of the size of quadratic programming subproblems via the compression and aggregation techniques of proximal bundle methods on the other hand the filter criterion for accepting a candidate point as the new iterate is sometimes easier to satisfy than the usual descent condition in bundle methods some encouraging preliminary computational results are also reported

proximal level bundle methods for convex nondifferentiable optimization saddle point problems and variational inequalities we study proximal level methods for convex optimization that use projections onto successive approximations of level sets of the objective corresponding to estimates of the optimal value we show that they enjoy almost optimal efficiency estimates we give extensions for solving convex constrained problems convex concave saddle point problems and variational inequalities with monotone operators we present several variants establish their efficiency estimates and discuss possible implementations in particular our methods require bounded storage in contrast to the original level methods of lemarechal nemirovskii and nesterov

benders decomposition without separability a computational study for capacitated facility location problems abstract benders is one of the most famous decomposition tools for mathematical programming and it is the method of choice e g in mixed integer stochastic programming its hallmark is the capability of decomposing certain types of models into smaller subproblems each of which can be solved individually to produce local information notably cutting planes to be exploited by a centralized master problem as its name suggests the power of the technique comes essentially from the decomposition effect i e the separability of the problem into a master problem and several smaller subproblems in this paper we address the question of whether the benders approach can be useful even without separability of the subproblem i e when its application yields a single subproblem of the same size as the original problem in particular we focus on the capacitated facility location problem in two variants the classical linear case and a congested case where the objective function contains convex but non separable quadratic terms we show how to embed the benders approach within a modern branch and cut mixed integer programming solver addressing explicitly all the ingredients that are instrumental for its success in particular we discuss some computational aspects that are related to the negative effects derived from the lack of separability extensive computational results on various classes of instances from the literature are reported with a comparison with the state of the art exact and heuristic algorithms the outcome is that a clever but simple implementation of the benders approach can be very effective even without separability as its performance is comparable and sometimes even better than that of the most effective and sophisticated algorithms proposed in the previous literature

primal convergence from dual subgradient methods for convex optimization when solving a convex optimization problem through a lagrangian dual reformulation subgradient optimization methods are favorably utilized since they often find near optimal dual solutions quickly however an optimal primal solution is generally not obtained directly through such a subgradient approach unless the lagrangian dual function is differentiable at an optimal solution we construct a sequence of convex combinations of primal subproblem solutions a so called ergodic sequence which is shown to converge to an optimal primal solution when the convexity weights are appropriately chosen we generalize previous convergence results from linear to convex optimization and present a new set of rules for constructing the convexity weights that define the ergodic sequence of primal solutions in contrast to previously proposed rules they exploit more information from later subproblem solutions than from earlier ones we evaluate the proposed rules on a set of nonlinear multicommodity flow problems and demonstrate that they clearly outperform the ones previously proposed

dynamic bundle methods lagrangian relaxation is a popular technique to solve difficult optimization problems however the applicability of this technique depends on having a relatively low number of hard constraints to dualize when there are many hard constraints it may be preferable to relax them dynamically according to some rule depending on which multipliers are active from the dual point of view this approach yields multipliers with varying dimensions and a dual objective function that changes along iterations we discuss how to apply a bundle methodology to solve this kind of dual problems our framework covers many separation procedures to generate inequalities that can be found in the literature including but not limited to the most violated inequality we analyze the resulting dynamic bundle method giving a positive answer for its primal dual convergence properties and under suitable conditions show finite termination for polyhedral problems

new variants of bundle methods in this paper we describe a number of new variants of bundle methods for nonsmooth unconstrained and constrained convex optimization convex concave games and variational inequalities we outline the ideas underlying these methods and present rate of convergence estimates

discriminative semi supervised feature selection via manifold regularization feature selection has attracted a huge amount of interest in both research and application communities of data mining we consider the problem of semi supervised feature selection where we are given a small amount of labeled examples and a large amount of unlabeled examples since a small number of labeled samples are usually insufficient for identifying the relevant features the critical problem arising from semi supervised feature selection is how to take advantage of the information underneath the unlabeled data to address this problem we propose a novel discriminative semi supervised feature selection method based on the idea of manifold regularization the proposed approach selects features through maximizing the classification margin between different classes and simultaneously exploiting the geometry of the probability distribution that generates both labeled and unlabeled data in comparison with previous semi supervised feature selection algorithms our proposed semi supervised feature selection method is an embedded feature selection method and is able to find more discriminative features we formulate the proposed feature selection method into a convex concave optimization problem where the saddle point corresponds to the optimal solution to find the optimal solution the level method a fairly recent optimization method is employed we also present a theoretic proof of the convergence rate for the application of the level method to our problem empirical evaluation on several benchmark data sets demonstrates the effectiveness of the proposed semi supervised feature selection method

on a problem of rolewicz about banach spaces that admit support sets we construct an example of a nonseparable banach space which does not admit a support set 2 it is a consistent and necessarily independent from the axioms of zfc example of a space c k of continuous functions on a compact hausdorff k with the supremum norm the construction depends on a construction of a boolean algebra with some combinatorial properties the space is also hereditarily lindelof in the weak topology but it doesn t have any nonseparable subspace nor any nonseparable quotient which is a c k space for k dispersed

a proximal cutting plane method using chebychev center for nonsmooth convex optimization an algorithm is developed for minimizing nonsmooth convex functions this algorithm extends elzinga moore cutting plane algorithm by enforcing the search of the next test point not too far from the previous ones thus removing compactness assumption our method is to elzinga moore s algorithm what a proximal bundle method is to kelley s algorithm instead of lower approximations used in proximal bundle methods the present approach is based on some objects regularizing translated functions of the objective function we propose some variants and using some academic test problems we conduct a numerical comparative study with elzinga moore algorithm and two other well known nonsmooth methods

a method of projection onto an acute cone with level control in convex minimization we study a projection method for convex minimization problems in each step we construct a certain polyhedral function that approximates the objective function from below the affine pieces of the approximating function are linearizations of the objective function obtained in previous steps the linearization are determined by a linearly independent system of subgradients which generates an obtuse cone the current approximation of the solution is projected onto a sublevel set of the polyhedral function with level which estimates the optimal value and is updated in each iteration the projection can be easily obtained by solving a system of linear equations and is actually the projection onto a translated acute cone dual to the constructed obtuse cone the construction ensures long steps between succesive approximations of the solution such long steps are important for the good behavior of the method

comparison of bundle and classical column generation when a column generation approach is applied to decomposable mixed integer programming problems it is standard to formulate and solve the master problem as a linear program seen in the dual space this results in the algorithm known in the nonlinear programming community as the cutting plane algorithm of kelley and cheney goldstein however more stable methods with better theoretical convergence rates are known and have been used as alternatives to this standard one of them is the bundle method our aim is to illustrate its differences with kelley s method in the process we review alternative stabilization techniques used in column generation comparing them from both primal and dual points of view numerical comparisons are presented for five applications cutting stock which includes bin packing vertex coloring capacitated vehicle routing multi item lot sizing and traveling salesman we also give a sketchy comparison with the volume algorithm

non euclidean restricted memory level method for large scale convex optimization we propose a new subgradient type method for minimizing extremely large scale nonsmooth convex functions over simple domains the characteristic features of the method are a the possibility to adjust the scheme to the geometry of the feasible set thus allowing to get nearly dimension independent and nearly optimal in the large scale case rate of convergence results for minimization of a convex lipschitz continuous function over a euclidean ball a standard simplex and a spectahedron the set of positive semidefinite symmetric matrices of given size with unit trace b flexible handling of accumulated information allowing for tradeoff between the level of utilizing this information and iteration s complexity we present extensions of the scheme for the cases of minimizing non lipschitzian convex objectives finding saddle points of convex concave functions and solving variational inequalities with monotone operators finally we report on encouraging numerical results of experiments with test problems of dimensions up to 66 000

analysis of stochastic dual dynamic programming method abstract in this paper we discuss statistical properties and convergence of the stochastic dual dynamic programming sddp method applied to multistage linear stochastic programming problems we assume that the underline data process is stagewise independent and consider the framework where at first a random sample from the original true distribution is generated and consequently the sddp algorithm is applied to the constructed sample average approximation saa problem then we proceed to analysis of the sddp solutions of the saa problem and their relations to solutions of the true problem finally we discuss an extension of the sddp method to a risk averse formulation of multistage stochastic programs we argue that the computational complexity of the corresponding sddp algorithm is almost the same as in the risk neutral case

accuracy certificates for computational problems with convex structure the goal of this paper is to introduce the notion of certificates which verify the accuracy of solutions of computational problems with convex structure such problems include minimizing convex functions variational inequalities with monotone operators computing saddle points of convex concave functions and solving convex nash equilibrium problems we demonstrate how the implementation of the ellipsoid method and other cutting plane algorithms can be augmented with the computation of such certificates without significant increase of the computational effort further we show that computable certificates exist for any algorithm that is capable of producing solutions of guaranteed accuracy

an adaptive partition based approach for solving two stage stochastic programs with fixed recourse we study an adaptive partition based approach for solving two stage stochastic programs with fixed recourse a partition based formulation is a relaxation of the original stochastic program and we study a finitely converging algorithm in which the partition is adaptively adjusted until it yields an optimal solution a solution guided refinement strategy is developed to refine the partition by exploiting the relaxation solution obtained from a partition in addition to refinement we show that in the case of stochastic linear programs it is possible to merge some components in a partition without weakening the corresponding relaxation bound thus allowing the partition size to be kept small we also show that for stochastic linear programs with simple recourse there exists a small partition that yields an optimal solution the size of this partition is independent of the number of scenarios used in the model our computational results show that the proposed adaptive partition based approach converges v

complexity estimates of some cutting plane methods based on the analytic barrier in this paper we establish the efficiency estimates for two cutting plane methods based on the analytic barrier we prove that the rate of convergence of the second method is optimal uniformly in the number of variables we present a modification of the second method in this modified version each test point satisfies an approximate centering condition we also use the standard strategy for updating approximate hessians of the logarithmic barrier function we prove that the rate of convergence of the modified scheme remains optimal and demonstrate that the number of newton steps in the auxiliary minimization processes is bounded by an absolute constant we also show that the approximate hessian strategy significantly improves the total arithmetical complexity of the method

applying oracles of on demand accuracy in two stage stochastic programming a computational study abstract traditionally two variants of the l shaped method based on benders decomposition principle are used to solve two stage stochastic programming problems the aggregate and the disaggregate version in this study we report our experiments with a special convex programming method applied to the aggregate master problem the convex programming method is of the type that uses an oracle with on demand accuracy we use a special form which when applied to two stage stochastic programming problems is shown to integrate the advantages of the traditional variants while avoiding their disadvantages on a set of 105 test problems we compare and analyze parallel implementations of regularized and unregularized versions of the algorithms the results indicate that solution times are significantly shortened by applying the concept of on demand accuracy

spectral bundle methods for non convex maximum eigenvalue functions second order methods we study constrained and unconstrained optimization programs for nonconvex maximum eigenvalue functions we show how second order techniques may be introduced as soon as it is possible to reliably guess the multiplicity of the maximum eigenvalue at a limit point we examine in which way standard and projected newton steps may be combined with a nonsmooth first order method to obtain a globally convergent algorithm with a fair chance to local superlinear or quadratic convergence

a generalized subgradient method with relaxation step we study conditions for convergence of a generalized subgradient algorithm in which a relaxation step is taken in a direction which is a convex combination of possibly all previously generated subgradients a simple condition for convergence is given and conditions that guarantee a linear convergence rate are also presented we show that choosing the steplength parameter and convex combination of subgradients in a certain sense optimally is equivalent to solving a minimum norm quadratic programming problem it is also shown that if the direction is restricted to be a convex combination of the current subgradient and the previous direction then an optimal choice of stepsize and direction is equivalent to the camerini fratta maffioli modification of the subgradient method

conditional gradient algorithms for norm regularized smooth convex optimization motivated by some applications in signal processing and machine learning we consider two convex optimization problems where given a cone k k a norm vert cdot vert and a smooth convex function f f we want either 1 to minimize the norm over the intersection of the cone and a level set of f f or 2 to minimize over the cone the sum of f f and a multiple of the norm we focus on the case where a the dimension of the problem is too large to allow for interior point algorithms b vert cdot vert is too complicated to allow for computationally cheap bregman projections required in the first order proximal gradient algorithms on the other hand we assume that it is relatively easy to minimize linear forms over the intersection of k k and the unit vert cdot vert ball motivating examples are given by the nuclear norm with k k being the entire space of matrices or the positive semidefinite cone in the space of symmetric matrices and the total variation norm on the space of 2d images we discuss versions of the conditional gradient algorithm capable to handle our problems of interest provide the related theoretical efficiency estimates and outline some applications

solving variational inequalities with monotone operators on domains given by linear minimization oracles the standard algorithms for solving large scale convex concave saddle point problems or more generally variational inequalities with monotone operators are proximal type algorithms which at every iteration need to compute a prox mapping that is to minimize over problem s domain x the sum of a linear form and the specific convex distance generating function underlying the algorithms in question relative computational simplicity of prox mappings which is the standard requirement when implementing proximal algorithms clearly implies the possibility to equip x with a relatively computationally cheap linear minimization oracle lmo able to minimize over x linear forms there are however important situations where a cheap lmo indeed is available but where no proximal setup with easy to compute prox mappings is known this fact motivates our goal in this paper which is to develop techniques for solving variational inequalities with monotone operators on domains given by lmo the techniques we discuss can be viewed as a substantial extension of the proposed in cox et al math program ser b 148 1 2 143 180 2014 method of nonsmooth convex minimization over an lmo represented domain

a version of the bundle idea for minimizing a nonsmooth function conceptual idea convergence analysis numerical results during recent years various proposals for the minimization of a nonsmooth functional have been made amongst these the bundle concept turned out to be an especially fruitful idea based on this concept a number of authors have developed codes that can successfully deal with nonsmooth problems the aim of the paper is to show that by adding some features of the trust region philosophy to the bundle concept the end result is a distinguished member of the bundle family with a more stable behaviour than some other bundle versions the reliability and efficiency of this code is demonstrated on the standard academic test examples and on some real life problems

a minimax resource allocation problem with variable resources abstract the problem of optimally allocating the resources to competing activities where the amounts of the resources are not only previosly given but are also to be determined subject to certain linear constraints is considered the objective is to find such activity levels such that their weighted deviation from a prespecified target is as small as possible a primal decomposition approach for the problem and derive an explicit formula for the piecewise affine linear convex objective function of the upper level problem are suggested the lower level problem is a parametric optimization problem with a bottleneck objective function if the constraints on the amounts of the resources are all of knapsack type or if there is only one such constraint then the upper level problem is equivalent to the lower level problem with fixed right hand sides in the general case the problem can efficiently be solved by means of the level method

bundle methods for regularized risk minimization a wide variety of machine learning problems can be described as minimizing a regularized risk functional with different algorithms using different notions of risk and different regularizers examples include linear support vector machines svms gaussian processes logistic regression conditional random fields crfs and lasso amongst others this paper describes the theory and implementation of a scalable and modular convex solver which solves all these estimation problems it can be parallelized on a cluster of workstations allows for data locality and can deal with regularizers such as l1 and l2 penalties in addition to the unified framework we present tight convergence bounds which show that our algorithm converges in o 1 e steps to e precision for general convex problems and in o log 1 e steps for continuously differentiable problems we demonstrate the performance of our general purpose solver on a variety of publicly available data sets

asynchronous level bundle methods in this paper we consider nonsmooth convex optimization problems with additive structure featuring independent oracles black boxes working in parallel existing methods for solving these distributed problems in a general form are synchronous in the sense that they wait for the responses of all the oracles before performing a new iteration in this paper we propose level bundle methods handling asynchronous oracles these methods require original upper bounds using upper models or scarce coordinations to deal with asynchronicity we prove their convergence using variational analysis techniques and illustrate their practical performance on a lagrangian decomposition problem

exact penalization level function method and modified cutting plane method for stochastic programs with second order stochastic dominance constraints level function methods and cutting plane methods have been recently proposed to solve stochastic programs with stochastic second order dominance ssd constraints a level function method requires an exact penalization setup because it can only be applied to the objective function not the constraints slater constraint qualification scq is often needed for deriving exact penalization it is well known that ssd usually does not satisfy scq and various relaxation schemes have been proposed so that the relaxed problem satisfies the scq in this paper we show that under some moderate conditions the desired constraint qualification can be guaranteed through some appropriate reformulation of the constraints rather than relaxation exact penalization schemes based on l 1 norm and l infty norm are subsequently derived through robinson s error bound on convex systems and clarke s exact penalty function theorem moreover we propose a modified cutting plane method which constructs a cutting plane through t

ergodic primal convergence in dual subgradient schemes for convex programming ii the case of inconsistent primal problems consider the utilization of a lagrangian dual method which is convergent for consistent convex optimization problems when it is used to solve an infeasible optimization problem its inconsistency will then manifest itself through the divergence of the sequence of dual iterates will then the sequence of primal subproblem solutions still yield relevant information regarding the primal program we answer this question in the affirmative for a convex program and an associated subgradient algorithm for its lagrange dual we show that the primal dual pair of programs corresponding to an associated homogeneous dual function is in turn associated with a saddle point problem in which in the inconsistent case the primal part amounts to finding a solution in the primal space such that the euclidean norm of the infeasibility in the relaxed constraints is minimized the dual part amounts to identifying a feasible steepest ascent direction for the lagrangian dual function we present convergence results for a conditional varepsilon subgradient optimization algorithm applied to the lagrangian dual problem and the construction of an ergodic sequence of primal subproblem solutions this composite algorithm yields convergence of the primal dual sequence to the set of saddle points of the associated homogeneous lagrangian function for linear programs convergence to the subset in which the primal objective is at minimum is also achieved

dual subgradient algorithms for large scale nonsmooth learning problems classical first order fo algorithms of convex optimization such as mirror descent algorithm or nesterov s optimal algorithm of smooth convex optimization are well known to have optimal theoretical complexity estimates which do not depend on the problem dimension however to attain the optimality the domain of the problem should admit a good proximal setup the latter essentially means that 1 the problem domain should satisfy certain geometric conditions of favorable geometry and 2 the practical use of these methods is conditioned by our ability to compute at a moderate cost proximal transformation at each iteration more often than not these two conditions are satisfied in optimization problems arising in computational learning what explains why proximal type fo methods recently became methods of choice when solving various learning problems yet they meet their limits in several important problems such as multi task learning with large number of tasks where the problem domain does not exhibit favorable geometry and learning and matrix completion problems with nuclear norm constraint when the numerical cost of computing proximal transformation becomes prohibitive in large scale problems we propose a novel approach to solving nonsmooth optimization problems arising in learning applications where fenchel type representation of the objective function is available the approach is based on applying fo algorithms to the dual problem and using the accuracy certificates supplied by the method to recover the primal solution while suboptimal in terms of accuracy guaranties the proposed approach does not rely upon good proximal setup for the primal problem but requires the problem domain to admit a linear optimization oracle the ability to efficiently maximize a linear form on the domain of the primal problem

piecewise quadratic approximations in convex numerical optimization we present a bundle method for convex nondifferentiable minimization where the model is a piecewise quadratic convex approximation of the objective function unlike standard bundle approaches the model only needs to support the objective function from below at a properly chosen small subset of points as opposed to everywhere we provide the convergence analysis for the algorithm with a general form of master problem which combines features of trust region stabilization and proximal stabilization taking care of all the important practical aspects such as proper handling of the proximity parameters and the bundle of information numerical results are also reported

aspects of the behaviour of solutions of nonlinear abel equations abstract in this paper we study properties about the behaviour uniqueness locally boundedness attracting behaviour etc of positive solutions for nonlinear abel integral equations and we present examples of nonlinear abel integral equations with two nontrivial solutions these examples are related to the existence of nonlocally bounded solutions

solving nonlinear multicommodity flow problems by the analytic center cutting plane method the paper deals with nonlinear multicommodity flow problems with convex costs a decomposition method is proposed to solve them the approach applies a potential reduction algorithm to solve the master problem approximately and a column generation technique to define a sequence of primal linear programming problems each subproblem consists of finding a minimum cost flow between an origin and a destination node in an uncapacited network it is thus formulated as a shortest path problem and solved with dijkstra s d heap algorithm an implementation is described that takes full advantage of the supersparsity of the network in the linear algebra operations computational results show the efficiency of this approach on well known nondifferentiable problems and also large scale randomly generated problems up to 1000 arcs and 5000 commodities

handling cvar objectives and constraints in two stage stochastic models abstract based on the polyhedral representation of kunzi bay and mayer kunzi bay a mayer j 2006 computational aspects of minimizing conditional value at risk computational management science 3 3 27 we propose decomposition frameworks for handling cvar objectives and constraints in two stage stochastic models for the solution of the decomposed problems we propose special level type methods

directed sets and cofinal types we show that 1 s 1 x x xl and l are the only cofinal types of directed sets of size sl but that there exist many cofinal types of directed sets of size continuum a partially ordered set d is directed if every two elements of d have an upper bound in d in this note we consider some basic problems concerning directed sets which have their origin in the theory of moore smith convergence in topology 12 3 19 9 one such problem is to determine all essential kind of directed sets needed for defining the closure operator in a given class of spaces 3 p 47 concerning this problem the following important notion was introduced by j tukey 19 two directed sets d and e are cofinally similar if there is a partially ordered set c in which both can be embedded as cofinal subsets he showed that this is an equivalence relation and that d and e are cofinally similar iff there is a convergent map from d into e and also a convergent map from e into d the equivalence classes of this relation are called cofinal types this concept has been extensively studied since then by various authors 4 13 7 8 already from the first introduction of this concept it has been known that 1 1 x x 1 and x1 represent different cofinal types of directed sets of size btl but no more than five such types were known the main result of this paper shows that 1 1 x x 1 and x1 are the only cofinal types of convergence in spaces of character 81 which can be constructed without additional set theoretic assumptions on the other hand we shall construct many different cofinal types of directed sets of size continuum this gives a solution to problem 1 of j isbell 7 the paper also contains several results about the structure of the class of all cofinal types as well as a result about decomposing arbitrary partially ordered sets into directed sets the results of this note were proved in february march 1982 and presented to the asl in january 1983 1 a decomposition theorem in this section we show that an arbitrary partially ordered set can be decomposed into a number of its directed subsets depending on the sizes of its antichains this result is connected with an unpublished problem of f galvin concerning the dilworth decomposition theorem 6 and it generalizes a similar result of e milner and k prikry 11 the transitivity condition of a partial ordering is not used in our proof so we state our result so as to apply to an arbitrary received by the editors december 3 1984 1980 mathpmaties subjeet classification primary 03e05 03e35 secondary 06a10 18b35 54a15 c1985 american mathematical society 0002 9947 85 1 00 25 per page

piecewise linear approximations in nonconvex nonsmooth optimization we present a bundle type method for minimizing nonconvex nondifferentiable functions of several variables the algorithm is based on the construction of both a lower and an upper polyhedral approximation of the objective function in particular at each iteration a search direction is computed by solving a quadratic program aiming at maximizing the difference between the lower and the upper model a proximal approach is used to guarantee convergence to a stationary point under the hypothesis of weak semismoothness

a new exact penalty function for constrained smooth or nonsmooth optimization problems new continuously differentiable penalty functions are derived they are proved exact in the sense that under some nondegeneracy assumption local optimizers of a nonlinear program are precisely the optimizers of the associated penalty function this is achieved by augmenting the dimension of the program by a variable that controls both the weight of the penalty terms and the regularization of the nonsmooth terms

irredundant sets in boolean algebras it is shown that every uncountable boolean algebra a contains an uncountable subset i such that no a of i is in the subalgebra generated by i a using an additional axiom of set theory it is also shown that a use of some such axiom is necessary

the approximation algorithm for solving a sort of non smooth programming an approximation algorithm for solving a sort of non smooth programming is proposed in the programming the objective function is the holder function and the feasible region is contained in a rectangle viz hyper rectangle to establish the algorithm the properties of the holder function and an approximation of the function by using bernstein polynomial are studied according to the properties of the approximation polynomial and the algorithm for solving geometric programming the strategy for branching and bounding and the branch and bound algorithm are constructed to solve the programming the convergence of the algorithm can be guaranteed by the exhaustive of the bisection of the rectangle the feasibility of the algorithm is validated by solving an example

an inexact proximal path following algorithm for constrained convex minimization many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets in this paper we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self concordant barrier we provide a new joint treatment of proximal and self concordant barrier concepts and illustrate that such problems can be efficiently solved without the need of lifting the problem dimensions as in disciplined convex optimization approach we propose an inexact path following algorithmic framework and theoretically characterize the worst case analytical complexity of this framework when the proximal subproblems are solved inexactly to show the merits of our framework we apply its instances to both synthetic and real world applications where it shows advantages over standard interior point methods as a byproduct we describe how our framework can obtain points on t

a bundle method for a class of bilevel nonsmooth convex minimization problems we consider the bilevel problem of minimizing a nonsmooth convex function over the set of minimizers of another nonsmooth convex function standard convex constrained optimization is a particular case in this framework corresponding to taking the lower level function as a penalty of the feasible set we develop an explicit bundle type algorithm for solving the bilevel problem where each iteration consists of making one descent step for a weighted sum of the upper and lower level functions after which the weight can be updated immediately convergence is shown under very mild assumptions we note that in the case of standard constrained optimization the method does not require iterative solution of any penalization subproblems not even approximately and does not assume any regularity of constraints e g the slater condition we also present some computational experiments for minimizing a nonsmooth convex function over a set defined by linear complementarity constraints

a sequential linear programming algorithm for solving monotone variational inequalities applied to strongly monotone variational inequalities newton s algorithm achieves local quadratic convergence in this paper it is shown how the basic newton method can be modified to yield an algorithm whose global convergence can be guaranteed by monitoring the monotone decrease of the gap function associated with the variational inequality each iteration consists in the solution of a linear program in the space of primal dual variables and of a linesearch convergence does not depend on strong monotonicity however under strong monotonicity and geometric stability assumptions the set of active constraints at the solution is implicitly identified and quadratic convergence is achieved

proximity control in bundle methods for convex proximal bundle methods for minimizing a convex functionf generate a sequence x k by takingxk 1 to be the minimizer of hat f k x u k x x k 2 2 where hat f k is a sufficiently accurate polyhedral approximation tof andu k 0 the usual choice ofu k 1 may yield very slow convergence a technique is given for choosing u k adaptively that eliminates sensitivity to objective scaling some encouraging numerical experience is reported

two stage linear decision rules for multi stage stochastic programming multi stage stochastic linear programs mslps are notoriously hard to solve in general linear decision rules ldrs yield an approximation of an mslp by restricting the decisions at each stage to be an affine function of the observed uncertain parameters finding an optimal ldr is a static optimization problem that provides an upper bound on the optimal value of the mslp and under certain assumptions can be formulated as an explicit linear program similarly as proposed by kuhn et al math program 130 1 177 209 2011 a lower bound for an mslp can be obtained by restricting decisions in the dual of the mslp to follow an ldr we propose a new approximation approach for mslps two stage ldrs the idea is to require only the state variables in an mslp to follow an ldr which is sufficient to obtain an approximation of an mslp that is a two stage stochastic linear program 2slp we similarly propose to apply ldr only to a subset of the variables in the dual of the mslp which yields a 2slp approximation of the dual that provides a lower bound on the optimal value of the mslp although solving the corresponding 2slp approximations exactly is intractable in general we investigate how approximate solution approaches that have been developed for solving 2slp can be applied to solve these approximation problems and derive statistical upper and lower bounds on the optimal value of the mslp in addition to potentially yielding better policies and bounds this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static ldr approach a computational study on two example problems demonstrates that using a two stage ldr can yield significantly better primal policies and modestly better dual policies than using policies based on a static ldr

bundle methods for sum functions with easy components applications to multicommodity network design we propose a version of the bundle scheme for convex nondifferentiable optimization suitable for the case of a sum function where some of the components are easy that is they are lagrangian functions of explicitly known compact convex programs this corresponds to a stabilized partial dantzig wolfe decomposition where suitably modified representations of the easy convex subproblems are inserted in the master problem as an alternative to iteratively inner approximating them by extreme points thus providing the algorithm with exact information about a part of the dual objective function the resulting master problems are potentially larger and less well structured than the standard ones ruling out the available specialized techniques and requiring the use of general purpose solvers for their solution this strongly favors piecewise linear stabilizing terms as opposed to the more usual quadratic ones which in turn may have an adverse effect on the convergence speed of the algorithm so that the overall performance may depend on appropriate tuning of all these aspects yet very good computational results are obtained in at least one relevant application the computation of tight lower bounds for fixed charge multicommodity min cost flow problems

nonconvolution nonlinear integral volterra equations with monotone operators abstract some results about existence uniqueness and attractive behaviour of solutions for nonlinear volterra integral equations with nonconvolution kernels are presented in this paper these results are based on similar ones about nonlinear volterra integral equations with convolutions kernels and some comparison techniques therefore this paper is devoted to find a wide class of nonconvolutions volterra integral equations where their solutions behave like those of volterra equations with convolution kernels

using regularization and second order information in outer approximation for convex minlp in this paper we present two new methods for solving convex mixed integer nonlinear programming problems based on the outer approximation method the first method is inspired by the level method and uses a regularization technique to reduce the step size when choosing new integer combinations the second method combines ideas from both the level method and the sequential quadratic programming technique and uses a second order approximation of the lagrangean when choosing the new integer combinations the main idea behind the methods is to choose the integer combination more carefully at each iteration in order to obtain the optimal solution in fewer iterations compared to the original outer approximation method we prove rigorously that both methods will find and verify the optimal solution in a finite number of iterations furthermore we present a numerical comparison of the methods based on 109 test problems to illustrate their advantages

a bundle method for solving equilibrium problems we present a bundle method for solving nonsmooth convex equilibrium problems based on the auxiliary problem principle first we consider a general algorithm that we prove to be convergent then we explain how to make this algorithm implementable the strategy is to approximate the nonsmooth convex functions by piecewise linear convex functions in such a way that the subproblems are easy to solve and the convergence is preserved in particular we introduce a stopping criterion which is satisfied after finitely many iterations and which gives rise to stationary points finally we apply our implementable algorithm for solving the particular case of singlevalued and multivalued variational inequalities and we find again the results obtained recently by salmon et al 18