convex proximal bundle methods in depth a unified analysis for inexact oracles the last few years have seen the advent of a new generation of bundle methods capable to handle inexact oracles polluted by noise proving convergence of a bundle method is never simple and coping with inexact oracles substantially increases the technicalities besides several variants exist to deal with noise each one needing an ad hoc proof to show convergence we state a synthetic convergence theory in which we highlight the main arguments and specify which assumption is used to establish each intermediate result the framework is comprehensive and generalizes in various ways a number of algorithms proposed in the literature based on the ingredients of our synthetic theory we consider various bundle methods adapted to oracles for which high accuracy is possible yet it is preferable not to make exact calculations often because they are too time consuming

a doubly stabilized bundle method for nonsmooth convex optimization we propose a bundle method for minimizing nonsmooth convex functions that combines both the level and the proximal stabilizations most bundle algorithms use a cutting plane model of the objective function to formulate a subproblem whose solution gives the next iterate proximal bundle methods employ the model in the objective function of the subproblem while level methods put the model in the subproblem s constraints the proposed algorithm defines new iterates by solving a subproblem that employs the model in both the objective function and in the constraints one advantage when compared to the proximal approach is that the level set constraint provides a certain lagrange multiplier which is used to update the proximal parameter in a novel manner we also show that in the case of inexact function and subgradient evaluations no additional procedure needs to be performed by our variant to deal with inexactness as opposed to the proximal bundle methods that require special modifications numerical experiments on almost 1 000 instances of different types of problems are presented our experiments show that the doubly stabilized bundle method inherits useful features of the level and the proximal versions and compares favorably to both of them

box dimensions and topological pressure for some expanding maps we consider an expanding map defined on an open region of the plane and study the box dimensions of its invariant sets under the condition that the map leaves invariant a strong unstable foliation we prove that the box dimension of an invariant set is given by f t where t is its dimension transverse to and f is the root of a certain function involving topological pressure

convergence analysis of deflected conditional approximate subgradient methods subgradient methods for nondifferentiable optimization benefit from deflection i e defining the search direction as a combination of the previous direction and the current subgradient in the constrained case they also benefit from projection of the search direction onto the feasible set prior to computing the steplength that is from the use of conditional subgradient techniques however combining the two techniques is not straightforward especially if an inexact oracle is available which can only compute approximate function values and subgradients we present a convergence analysis of several different variants both conceptual and implementable of approximate conditional deflected subgradient methods our analysis extends the available results in the literature by using the main stepsize rules presented so far while allowing deflection in a more flexible way furthermore to allow for diminishing square summable rules where the stepsize is tightly controlled a priori we propose a new class of deflection restricted approaches where it is the deflection parameter rather than the stepsize which is dynamically adjusted using the target value of the optimization sequence for both polyak type and diminishing square summable stepsizes we propose a correction of the standard formula which shows that in the inexact case knowledge about the error computed by the oracle which is available in several practical applications can be exploited in order to strengthen the convergence properties of the method the analysis allows for several variants of the algorithm at least one of them is likely to show numerical performances similar to these of heavy ball subgradient methods popular within backpropagation approaches to train neural networks while possessing stronger convergence properties

dynamic bundle methods lagrangian relaxation is a popular technique to solve difficult optimization problems however the applicability of this technique depends on having a relatively low number of hard constraints to dualize when there are many hard constraints it may be preferable to relax them dynamically according to some rule depending on which multipliers are active from the dual point of view this approach yields multipliers with varying dimensions and a dual objective function that changes along iterations we discuss how to apply a bundle methodology to solve this kind of dual problems our framework covers many separation procedures to generate inequalities that can be found in the literature including but not limited to the most violated inequality we analyze the resulting dynamic bundle method giving a positive answer for its primal dual convergence properties and under suitable conditions show finite termination for polyhedral problems

comparison of bundle and classical column generation when a column generation approach is applied to decomposable mixed integer programming problems it is standard to formulate and solve the master problem as a linear program seen in the dual space this results in the algorithm known in the nonlinear programming community as the cutting plane algorithm of kelley and cheney goldstein however more stable methods with better theoretical convergence rates are known and have been used as alternatives to this standard one of them is the bundle method our aim is to illustrate its differences with kelley s method in the process we review alternative stabilization techniques used in column generation comparing them from both primal and dual points of view numerical comparisons are presented for five applications cutting stock which includes bin packing vertex coloring capacitated vehicle routing multi item lot sizing and traveling salesman we also give a sketchy comparison with the volume algorithm

a geometric study of duality gaps with applications lagrangian relaxation is often an efficient tool to solve large scale optimization problems even nonconvex however it introduces a duality gap which should be small for the method to be really efficient here we make a geometric study of the duality gap given a nonconvex problem we formulate in a first part a convex problem having the same dual this formulation involves a convexification in the product of the three spaces containing respectively the variables the objective and the constraints we apply our results to several relaxation schemes especially one called lagrangean decomposition in the combinatorial optimization community or operator splitting elsewhere we also study a specific application highly nonlinear the unit commitment problem

a proximal bundle method with approximate subgradient linearizations we give a proximal bundle method for minimizing a convex function f over a closed convex set it only requires evaluating f and its subgradients with an accuracy epsilon 0 which is fixed but possibly unknown it asymptotically finds points that are epsilon optimal when applied to lagrangian relaxation it allows for epsilon accurate solutions of lagrangian subproblems and finds epsilon optimal solutions of convex programs

a version of the bundle idea for minimizing a nonsmooth function conceptual idea convergence analysis numerical results during recent years various proposals for the minimization of a nonsmooth functional have been made amongst these the bundle concept turned out to be an especially fruitful idea based on this concept a number of authors have developed codes that can successfully deal with nonsmooth problems the aim of the paper is to show that by adding some features of the trust region philosophy to the bundle concept the end result is a distinguished member of the bundle family with a more stable behaviour than some other bundle versions the reliability and efficiency of this code is demonstrated on the standard academic test examples and on some real life problems

constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems joint chance constrained problems give rise to many algorithmic challenges even in the convex case i e when an appropriate transformation of the probabilistic constraint is a convex function its cutting plane linearization is just an approximation produced by an oracle providing subgradient and function values that can only be evaluated inexactly as a result the cutting plane model may lie above the true constraint for dealing with such upper inexact oracles and still solving the problem up to certain precision a special numerical algorithm must be put in place we introduce a family of constrained bundle methods based on the so called improvement functions that is shown to be convergent and encompasses many previous approaches as well as new algorithms depending on the oracle accuracy we analyze to what extent the considered methods solve the joint chance constrained program the approach is assessed on real life energy problems arising when dealing with stochastic hydroreservoir management

piecewise quadratic approximations in convex numerical optimization we present a bundle method for convex nondifferentiable minimization where the model is a piecewise quadratic convex approximation of the objective function unlike standard bundle approaches the model only needs to support the objective function from below at a properly chosen small subset of points as opposed to everywhere we provide the convergence analysis for the algorithm with a general form of master problem which combines features of trust region stabilization and proximal stabilization taking care of all the important practical aspects such as proper handling of the proximity parameters and the bundle of information numerical results are also reported

an algorithm for nonsmooth convex minimization with errors a readily implementable algorithm is given for minimizing any convex not necessarily differentiable function f of several variables at each iteration the method requires only one approximate evaluation of f and its e subgradient and finds a search direction by solving a small quadratic programming problem the algorithm generates a minimizing sequence of points which converges to a solution wheneverf has any minimizers

schrodinger operators on graphs and geometry i essentially bounded potentials the inverse spectral problem for schrodinger operators on finite compact metric graphs is investigated the relations between the spectral asymptotics and geometric properties of the underlying graph are studied it is proven that the euler characteristic of the graph can be calculated from the spectrum of the schrodinger operator in the case of essentially bounded real potentials and standard boundary conditions at the vertices several generalizations of the presented results are discussed

nonanticipative duality relaxations and formulations for chance constrained stochastic programs we propose two new lagrangian dual problems for chance constrained stochastic programs based on relaxing nonanticipativity constraints we compare the strength of the proposed dual bounds and demonstrate that they are superior to the bound obtained from the continuous relaxation of a standard mixed integer programming mip formulation for a given dual solution the associated lagrangian relaxation bounds can be calculated by solving a set of single scenario subproblems and then solving a single knapsack problem we also derive two new primal mip formulations and demonstrate that for chance constrained linear programs the continuous relaxations of these formulations yield bounds equal to the proposed dual bounds we propose a new heuristic method and two new exact algorithms based on these duals and formulations the first exact algorithm applies to chance constrained binary programs and uses either of the proposed dual bounds in concert with cuts that eliminate solutions found by the subproblems the second exact method is a branch and cut algorithm for solving either of the primal formulations our computational results indicate that the proposed dual bounds and heuristic solutions can be obtained efficiently and the gaps between the best dual bounds and the heuristic solutions are small

dual applications of proximal bundle methods including lagrangian relaxation of nonconvex problems we exhibit useful properties of proximal bundle methods for finding min sf where f and s are convex we show that they asymptotically find objective subgradients and constraint multipliers involved in optimality conditions multipliers of objective pieces for max type functions and primal and dual solutions in lagrangian decomposition of convex programs when applied to lagrangian relaxation of nonconvex programs they find solutions to relaxed convexified versions of such programs numerical results are presented for unit commitment in power production scheduling

divide to conquer decomposition methods for energy optimization modern electricity systems provide a plethora of challenging issues in optimization the increasing penetration of low carbon renewable sources of energy introduces uncertainty in problems traditionally modeled in a deterministic setting the liberalization of the electricity sector brought the need of designing sound markets ensuring capacity investments while properly reflecting strategic interactions in all these problems hedging risk possibly in a dynamic manner is also a concern the fact of representing uncertainty and or competition of different companies in a multi settlement power market considerably increases the number of variables and constraints for this reason usually a trade off needs to be found between modeling and numerical tractability the more details are brought into the model the harder becomes the optimization problem for structured optimization and generalized equilibrium problems we explore some variants of solution methods based on lagrangian relaxation and on benders decomposition throughout we keep as a leading thread the actual practical value of such techniques in terms of their efficiency to solve energy related problems

the effect of deterministic noise in subgradient methods in this paper we study the influence of noise on subgradient methods for convex constrained optimization the noise may be due to various sources and is manifested in inexact computation of the subgradients and function values assuming that the noise is deterministic and bounded we discuss the convergence properties for two cases the case where the constraint set is compact and the case where this set need not be compact but the objective function has a sharp set of minima for example the function is polyhedral in both cases using several different stepsize rules we prove convergence to the optimal value within some tolerance that is given explicitly in terms of the errors in the first case the tolerance is nonzero but in the second case the optimal value can be obtained exactly provided the size of the error in the subgradient computation is below some threshold we then extend these results to objective functions that are the sum of a large number of convex functions in which case an incremental subgradient method can be used

restricted step and levenberg marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization two methods are given for minimizing locally lipschitzian upper semidifferentiable functions they employ extensions of restricted step trust region and levenberg marquardt techniques that are widely used in other contexts extensions to linearly constrained optimization are discussed preliminary numerical experience is reported

proximity control in bundle methods for convex proximal bundle methods for minimizing a convex functionf generate a sequence x k by takingxk 1 to be the minimizer of hat f k x u k x x k 2 2 where hat f k is a sufficiently accurate polyhedral approximation tof andu k 0 the usual choice ofu k 1 may yield very slow convergence a technique is given for choosing u k adaptively that eliminates sensitivity to objective scaling some encouraging numerical experience is reported

a spectral bundle method with bounds semidefinite relaxations of quadratic 0 1 programming or graph partitioning problems are well known to be of high quality however solving them by primal dual interior point methods can take much time even for problems of moderate size the recent spectral bundle method of helmberg and rendl can solve quite efficiently large structured equality constrained semidefinite programs if the trace of the primal matrix variable is fixed as happens in many applications we extend the method so that it can handle inequality constraints without seriously increasing computation time in addition we introduce inexact null steps this abolishes the need of computing exact eigenvectors for subgradients which brings along significant advantages in theory and in practice encouraging preliminary computational results are reported

inexact bundle methods for two stage stochastic programming stochastic programming problems arise in many practical situations in general the deterministic equivalents of these problems can be very large and may not be solvable directly by general purpose optimization approaches for the particular case of two stage stochastic programs we consider decomposition approaches akin to a regularized l shaped method that can handle inexactness in the subproblem solution from a nonsmooth optimization perspective these variants amount to applying a proximal bundle method to an oracle that gives inaccurate values for the objective function and a subgradient rather than forcing early termination of the subproblem optimization to define inexact oracles we select a small subset of scenarios for which the subproblem solution is exact and we replace the information for the remaining scenarios by a fast procedure that does not involve solving an optimization problem the inaccurate oracle information creates inexact cuts in the master program which are well handled by th