a polynomial time approximation algorithm for the permanent of a matrix with nonnegative entries we present a polynomial time randomized algorithm for estimating the permanent of an arbitrary n n matrix with nonnegative entries this algorithm technically a fully polynomial randomized approximation scheme computes an approximation that is with high probability within arbitrarily small specified relative error of the true value of the permanent

approximate primal solutions and rate analysis for dual subgradient methods in this paper we study methods for generating approximate primal solutions as a byproduct of subgradient methods applied to the lagrangian dual of a primal convex possibly nondifferentiable constrained optimization problem our work is motivated by constrained primal problems with a favorable dual problem structure that leads to efficient implementation of dual subgradient methods such as the recent resource allocation problems in large scale networks for such problems we propose and analyze dual subgradient methods that use averaging schemes to generate approximate primal optimal solutions these algorithms use a constant stepsize in view of its simplicity and practical significance we provide estimates on the primal infeasibility and primal suboptimality of the generated approximate primal solutions these estimates are given per iteration thus providing a basis for analyzing the trade offs between the desired level of error and the selection of the stepsize value our analysis relies on the slater condition and the inherited boundedness properties of the dual problem under this condition it also relies on the boundedness of subgradients which is ensured by assuming the compactness of the constraint set

cooling schedules for optimal annealing a monte carlo optimization technique called simulated annealing is a descent algorithm modified by random ascent moves in order to escape local minima which are not global minima the level of randomization is determined by a control parameter t called temperature which tends to zero according to a deterministic cooling schedule we give a simple necessary and sufficient condition on the cooling schedule for the algorithm state to converge in probability to the set of globally minimum cost states in the special case that the cooling schedule has parametric form tt c log1 t the condition for convergence is that c be greater than or equal to the depth suitably defined of the deepest local minimum which is not a global minimum state

ergodic primal convergence in dual subgradient schemes for convex programming lagrangean dualization and subgradient optimization techniques are frequently used within the field of computational optimization for finding approximate solutions to large structured optimization prob lems the dual subgradient scheme does not automatically produce primal feasible solutions there is an abundance of techniques for computing such solutions via penalty functions tangential approximation schemes or the solution of auxiliary primal programs all of which require a fair amount of computational effort we consider a subgradient optimization scheme applied to a lagrangean dual formulation of a convex program and construct at minor cost an ergodic sequence of subproblem solutions which converges to the primal solution set numerical experiments performed on a traffic equilibrium assignment problem under road pricing show that the computation of the ergodic sequence results in a considerable improvement in the quality of the primal solutions obtained compared to those generated in the basic subgradient scheme

survey paper research on probabilistic methods for control system design a novel approach based on probability and randomization has emerged to synergize with the standard deterministic methods for control of systems with uncertainty the main objective of this paper is to provide a broad perspective on this area of research known as probabilistic robust control and to address in a systematic manner recent advances the focal point is on design methods based on the interplay between uncertainty randomization and convex optimization and on the illustration of specific control applications

submodular functions from discrete to continous domains submodular set functions have many applications in combinatorial optimization as they can be minimized and approximately maximized in polynomial time a key element in many of the algorithms and analyses is the possibility of extending the submodular set function to a convex function which opens up tools from convex optimization submodularity goes beyond set functions and has naturally been considered for problems with multiple labels or for functions defined on continuous domains where it corresponds essentially to cross second derivatives being nonpositive in this paper we show that most results relating submodularity and convexity for set functions can be extended to all submodular functions in particular a we naturally define a continuous extension in a set of probability measures b show that the extension is convex if and only if the original function is submodular c prove that the problem of minimizing a submodular function is equivalent to a typically non smooth convex optimization problem and d propose another convex optimization problem with better computational properties e g a smooth dual problem most of these extensions from the set function situation are obtained by drawing links with the theory of multi marginal optimal transport which provides also a new interpretation of existing results for set functions we then provide practical algorithms to minimize generic submodular functions on discrete domains with associated convergence rates

distributed sub optimal resource allocation over weight balanced graph via singular perturbation in this paper we consider distributed optimization design for resource allocation problems over weight balanced graphs with the help of singular perturbation analysis we propose a simple sub optimal continuous time optimization algorithm moreover we prove the existence and uniqueness of the algorithm equilibrium and then show the convergence with an exponential rate finally we verify the sub optimality of the algorithm which can approach the optimal solution as an adjustable parameter tends to zero

distributed robust adaptive equilibrium computation for generalized convex games this paper considers a class of generalized convex games where each player is associated with a convex objective function a convex inequality constraint and a convex constraint set the players aim to compute a nash equilibrium through communicating with neighboring players the particular challenge we consider is that the component functions are unknown a priori to associated players we study two distributed computation algorithms and analyze their convergence properties in the presence of data transmission delays and dynamic changes of network topologies the algorithm performance is verified through demand response on the ieee 30 bus test system our technical tools integrate convex analysis variational inequalities and simultaneous perturbation stochastic approximation

convergence analysis of a lasserre hierarchy of upper bounds for polynomial minimization on the sphere we study the convergence rate of a hierarchy of upper bounds for polynomial minimization problems proposed by lasserre siam j optim 21 3 864 885 2011 for the special case when the feasible set is the unit hyper sphere the upper bound at level r n of the hierarchy is defined as the minimal expected value of the polynomial over all probability distributions on the sphere when the probability density function is a sum of squares polynomial of degree at most 2r with respect to the surface measure we show that the rate of convergence is o 1 r2 and we give a class of polynomials of any positive degree for which this rate is tight in addition we explore the implications for the related rate of convergence for the generalized problem of moments on the sphere

computational complexity certification for real time mpc with input constraints based on the fast gradient method this paper proposes to use nesterov s fast gradient method for the solution of linear quadratic model predictive control mpc problems with input constraints the main focus is on the method s a priori computational complexity certification which consists of deriving lower iteration bounds such that a solution of pre specified suboptimality is obtained for any possible state of the system we investigate cold and warm starting strategies and provide an easily computable lower iteration bound for cold starting and an asymptotic characterization of the bounds for warm starting moreover we characterize the set of mpc problems for which small iteration bounds and thus short solution times are expected the theoretical findings and the practical relevance of the obtained lower iteration bounds are underpinned by various numerical examples and compared to certification results for a primal dual interior point method

primal convergence from dual subgradient methods for convex optimization when solving a convex optimization problem through a lagrangian dual reformulation subgradient optimization methods are favorably utilized since they often find near optimal dual solutions quickly however an optimal primal solution is generally not obtained directly through such a subgradient approach unless the lagrangian dual function is differentiable at an optimal solution we construct a sequence of convex combinations of primal subproblem solutions a so called ergodic sequence which is shown to converge to an optimal primal solution when the convexity weights are appropriately chosen we generalize previous convergence results from linear to convex optimization and present a new set of rules for constructing the convexity weights that define the ergodic sequence of primal solutions in contrast to previously proposed rules they exploit more information from later subproblem solutions than from earlier ones we evaluate the proposed rules on a set of nonlinear multicommodity flow problems and demonstrate that they clearly outperform the ones previously proposed

approximate projected consensus for convex intersection computation convergence analysis and critical error angle in this paper we study an approximate projected consensus algorithm for a network to cooperatively compute the intersection of convex sets where each set corresponds to one network node instead of assuming exact convex projection that each node can compute we allow each node to compute an approximate projection with respect to its own set after receiving the approximate projection information nodes update their states by weighted averaging with the neighbors over a directed and time varying communication graph the approximate projections are related to projection angle errors which introduces state dependent disturbance in the iterative algorithm projection accuracy conditions are presented for the considered algorithm to converge the results indicate how much projection accuracy is required to ensure global consensus to a point in the intersection set when the communication graph is uniformly jointly strongly connected in addition we show that pi 4 is a critical angle for the error of the projection approximation to ensure the boundedness finally the results are illustrated by simulations

convergence of limited communication gradient methods distributed optimization increasingly plays a central role in economical and sustainable operation of cyber physical systems nevertheless the complete potential of the technology has not yet been fully exploited in practice due to communication limitations posed by the real world infrastructures this work investigates fundamental properties of distributed optimization based on gradient methods where gradient information is communicated using a limited number of bits in particular a general class of quantized gradient methods are studied where the gradient direction is approximated by a finite quantization set sufficient and necessary conditions are provided on such a quantization set to guarantee that the methods minimize any convex objective function with lipschitz continuous gradient and a nonempty and bounded set of optimizers a lower bound on the cardinality of the quantization set is provided along with specific examples of minimal quantizations convergence rate results are established that connect the fineness of the quantization and the number of iterations needed to reach a predefined solution accuracy generalizations of the results to a relevant class of constrained problems using projections are considered finally the results are illustrated by simulations of practical systems

application of a smoothing technique to decomposition in convex optimization dual decomposition is a powerful technique for deriving decomposition schemes for convex optimization problems with separable structure although the augmented lagrangian is computationally more stable than the ordinary lagrangian the ldquoprox termrdquo destroys the separability of the given problem in this technical note we use another approach to obtain a smooth lagrangian based on a smoothing technique developed by nesterov which preserves separability of the problem with this approach we derive a new decomposition method called ldquoproximal center algorithm rdquo which from the viewpoint of efficiency estimates improves the bounds on the number of iterations of the classical dual gradient scheme by an order of magnitude

computational complexity certification for dual gradient method application to embedded mpc abstract in this paper we analyze the computational complexity of the dual gradient method for solving linearly constrained convex problems when it is difficult to project on the primal feasible set described by linear constraints we use the lagrangian relaxation to handle the complicated constraints and then we apply the dual gradient algorithm for solving the corresponding dual we give a unified convergence rate analysis for the dual gradient algorithm we provide sublinear or linear estimates on the primal suboptimality and feasibility violation of the generated approximate primal solutions our analysis relies on the lipschitz property of the dual function or an error bound property furthermore the iteration complexity analysis is based on two types of approximate primal solutions an average primal sequence or the last primal iterate sequence we also discuss complexity certifications and implementation aspects of the dual gradient algorithm on constrained mpc problems for embedded linear systems

exact complexity certification of active set methods for quadratic programming active set methods are recognized to often outperform other methods in terms of speed and solution accuracy when solving small size quadratic programming qp problems making them very attractive in embedded linear model predictive control mpc applications a drawback of active set methods is the lack of tight bounds on the worst case number of iterations a fundamental requirement for their implementation in a real time system extensive simulation campaigns provide an indication of the expected worst case computation load but not a complete guarantee this paper solves such a certification problem by proposing an algorithm to compute the exact bound on the maximum number of iterations and floating point operations required by a state of the art dual active set qp solver the algorithm is applicable to a given qp problem whose linear term of the cost function and right hand side of the constraints depend linearly on a vector of parameters as in the case of linear mpc in addition a new solver is presented that combines explicit and implicit mpc ideas guaranteeing improvements of the worst case computation time the ability of the approach to exactly quantify memory and worst case computation requirements is tested on a few mpc examples also highlighting when online optimization should be preferred to explicit mpc

non euclidean restricted memory level method for large scale convex optimization we propose a new subgradient type method for minimizing extremely large scale nonsmooth convex functions over simple domains the characteristic features of the method are a the possibility to adjust the scheme to the geometry of the feasible set thus allowing to get nearly dimension independent and nearly optimal in the large scale case rate of convergence results for minimization of a convex lipschitz continuous function over a euclidean ball a standard simplex and a spectahedron the set of positive semidefinite symmetric matrices of given size with unit trace b flexible handling of accumulated information allowing for tradeoff between the level of utilizing this information and iteration s complexity we present extensions of the scheme for the cases of minimizing non lipschitzian convex objectives finding saddle points of convex concave functions and solving variational inequalities with monotone operators finally we report on encouraging numerical results of experiments with test problems of dimensions up to 66 000

hit and run from a corner we show that the hit and run random walk mixes rapidly starting from any interior point of a convex body this is the first random walk known to have this property in contrast the ball walk can take exponentially many steps from some starting points the proof extends to sampling an exponential density over a convex body

truncated second main theorem with moving targets we prove a truncated second main theorem for holomorphic curves intersecting a finite set of moving or fixed hyperplanes the set of hyperplanes is assumed to be non degenerate previously only general position or subgeneral position was considered

adaptive simulated annealing a near optimal connection between sampling and counting we present a near optimal reduction from approximately counting the cardinality of a discrete set to approximately sampling elements of the set an important application of our work is to approximating the partition function z of a discrete system such as the ising model matchings or colorings of a graph the typical approach to estimating the partition function z at some desired inverse temperature is to define a sequence which we call a cooling schedule 0 e 0 for well studied problems such as estimating the partition function of the ising model or approximating the number of colorings or matchings of a graph our cooling schedule is of length o s n which implies an overall savings of o n in the running time of the approximate counting algorithm since roughly e samples are needed to estimate each ratio a similar improvement in the length of the cooling schedule was recently obtained by lovasz and vempala in the context of estimating the volume of convex bodies while our reduction is inspired by theirs the discrete analogue of their result turns out to be significantly more difficult whereas a fixed schedule suffices in their setting we prove that in the discrete setting we need an adaptive schedule that is the schedule depends on z more precisely we prove any nonadaptive cooling schedule has length at least o ln a and we present an algorithm to find an adaptive schedule of length o s ln a

projective re normalization for improving the behavior of a homogeneous conic linear system in this paper we study the homogeneous conic system f ax 0 x in c setminus 0 we choose a point bar s in rm int c that serves as a normalizer and consider computational properties of the normalized system f bar s ax 0 bar s t x 1 x in c we show that the computational complexity of solving f via an interior point method depends only on the complexity value vartheta of the barrier for c and on the symmetry of the origin in the image set h bar s ax bar s tx 1 x in c where the symmetry of 0 in h bar s is rm sym 0 h bar s rm max alpha y in h bar s rightarrow alpha y in h bar s interior point iterations in order to improve the theoretical and practical computation of a solution of f we next present a general theory for projective re normalization of the feasible region f bar s and the image set h bar s and prove the existence of a normalizer bar s such that rm sym 0 h bar s ge 1 m provided that f has an interior solution we develop a methodology for constructing a normalizer bar s such that rm sym 0 h bar s ge 1 m with high probability based on sampling on a geometric random walk with associated probabilistic complexity analysis while such a normalizer is not itself computable in strongly polynomial time the normalizer will yield a conic system that is solvable in o sqrt vartheta rm ln m vartheta iterations which is strongly polynomial time finally we implement this methodology on randomly generated homogeneous linear programming feasibility problems constructed to be poorly behaved our computational results indicate that the projective re normalization methodology holds the promise to markedly reduce the overall computation time for conic feasibility problems for instance we observe a 46 decrease in average ipm iterations for 100 randomly generated poorly behaved problem instances of dimension 1 000 5 000

incremental subgradient methods for nondifferentiable optimization we consider a class of subgradient methods for minimizing a convex function that consists of the sum of a large number of component functions this type of minimization arises in a dual context from lagrangian relaxation of the coupling constraints of large scale separable problems the idea is to perform the subgradient iteration incrementally by sequentially taking steps along the subgradients of the component functions with intermediate adjustment of the variables after processing each component function this incremental approach has been very successful in solving large differentiable least squares problems such as those arising in the training of neural networks and it has resulted in a much better practical rate of convergence than the steepest descent method in this paper we establish the convergence properties of a number of variants of incremental subgradient methods including some that are stochastic based on the analysis and computational experiments the methods appear very promising and effective for important classes of large problems a particularly interesting discovery is that by randomizing the order of selection of component functions for iteration the convergence rate is substantially improved

on distributed convex optimization under inequality and equality constraints we consider a general multi agent convex optimization problem where the agents are to collectively minimize a global objective function subject to a global inequality constraint a global equality constraint and a global constraint set the objective function is defined by a sum of local objective functions while the global constraint set is produced by the intersection of local constraint sets in particular we study two cases one where the equality constraint is absent and the other where the local constraint sets are identical we devise two distributed primal dual subgradient algorithms based on the characterization of the primal dual optimal solutions as the saddle points of the lagrangian and penalty functions these algorithms can be implemented over networks with dynamically changing topologies but satisfying a standard connectivity property and allow the agents to asymptotically agree on optimal solutions and optimal values of the optimization problem under the slater s condition

duality between subgradient and conditional gradient methods given a convex optimization problem and its dual there are many possible first order algorithms in this paper we show the equivalence between mirror descent algorithms and algorithms generalizing the conditional gradient method this is done through convex duality and implies notably that for certain problems such as for supervised machine learning problems with nonsmooth losses or problems regularized by nonsmooth regularizers the primal subgradient method and the dual conditional gradient method are formally equivalent the dual interpretation leads to a form of line search for mirror descent as well as guarantees of convergence for primal dual certificates

distributed constrained optimization by consensus based primal dual perturbation method various distributed optimization methods have been developed for solving problems which have simple local constraint sets and whose objective function is the sum of local cost functions of distributed agents in a network motivated by emerging applications in smart grid and distributed sparse regression this paper studies distributed optimization methods for solving general problems which have a coupled global cost function and have inequality constraints we consider a network scenario where each agent has no global knowledge and can access only its local mapping and constraint functions to solve this problem in a distributed manner we propose a consensus based distributed primal dual perturbation pdp algorithm in the algorithm agents employ the average consensus technique to estimate the global cost and constraint functions via exchanging messages with neighbors and meanwhile use a local primal dual perturbed subgradient method to approach a global optimum the proposed pdp method not only can handle smooth inequality constraints but also non smooth constraints such as some sparsity promoting constraints arising in sparse optimization we prove that the proposed pdp algorithm converges to an optimal primal dual solution of the original problem under standard problem and network assumptions numerical results illustrating the performance of the proposed algorithm for a distributed demand response control problem in smart grid are also presented

second main theorem and uniqueness theorem with moving targets on parabolic manifolds abstract in this paper with the motivation from diophantine approximation a truncated second main theorem is established for meromorphic maps from m into p v with moving targets g j m p v 1 j q where m is a parabolic manifold and v is a hermitian vector space as an application of this second main theorem a uniqueness theorem without counting multiplicities is given

a decomposition method for large scale milps with performance guarantees and a power system application lagrangian duality in mixed integer optimization is a useful framework for problem decomposition and for producing tight lower bounds to the optimal objective however in contrast to the convex case it is generally unable to produce optimal solutions directly in fact solutions recovered from the dual may not only be suboptimal but even infeasible in this paper we concentrate on large scale mixed integer programs with a specific structure that appears in a variety of application domains such as power systems and supply chain management we propose a solution method for these structures in which the primal problem is modified in a certain way guaranteeing that the solutions produced by the corresponding dual are feasible for the original unmodified primal problem the modification is simple to implement and the method is amenable to distributed computation we also demonstrate that the quality of the solutions recovered using our procedure improves as the problem size increases making it particularly useful for large scale problem instances for which commercial solvers are inadequate we illustrate the efficacy of our method with extensive experimentations on a problem stemming from power systems

stabilizing linear model predictive control under inexact numerical optimization this note describes a model predictive control mpc formulation for discrete time linear systems with hard constraints on control and state variables under the assumption that the solution of the associated quadratic program is neither optimal nor satisfies the inequality constraints this is common in embedded control applications for which real time constraints and limited computing resources dictate restrictions on the possible number of on line iterations that can be performed within a sampling period the proposed approach is rather general in that it does not refer to a particular optimization algorithm and is based on the definition of an alternative mpc problem that we assume can only be solved within bounded levels of suboptimality and violation of the inequality constraints by showing that the inexact solution is a feasible suboptimal one for the original problem asymptotic or exponential stability is guaranteed for the closed loop system based on the above general results we focus on a specific dual accelerated gradient projection method to obtain a stabilizing mpc law that only requires a predetermined maximum number of on line iterations

a random sampling based algorithm for learning intersections of halfspaces we give an algorithm to learn an intersection of k halfspaces in rn whose normals span an l dimensional subspace for any input distribution with a logconcave density such that the bounding hyperplanes of the k halfspaces pass through its mean the algorithm e learns with time and sample complexity bounded by nkl e o l log 1 e the hypothesis found is an intersection of o k log 1 e halfspaces this improves on blum and kannan s algorithm for the uniform distribution over a ball in the time and sample complexity previously doubly exponential and in the generality of the input distribution

a class of convergent primal dual subgradient algorithms for decomposable convex programs in this paper we develop a primal dual subgradient algorithm for preferably decomposable generally nondifferentiable convex programming problems under usual regularity conditions the algorithm employs a lagrangian dual function along with a suitable penalty function which satisfies a specified set of properties in order to generate a sequence of primal and dual iterates for which some subsequence converges to a pair of primal dual optimal solutions several classical types of penalty functions are shown to satisfy these specified properties a geometric convergence rate is established for the algorithm under some additional assumptions this approach has three principal advantages firstly both primal and dual solutions are available which prove to be useful in several contexts secondly the choice of step sizes which plays an important role in subgradient optimization is guided more determinably in this method via primal and dual information thirdly typical subgradient algorithms suffer from the lack of an appropriate stopping criterion and so the quality of the solution obtained after a finite number of steps is usually unknown in contrast by using the primal dual gap the proposed algorithm possesses a natural stopping criterion

on the convergence of a distributed augmented lagrangian method for nonconvex optimization in this paper we propose a distributed algorithm for optimization problems that involve a separable possibly nonconvex objective function subject to convex local constraints and linear coupling constraints the method is based on the accelerated distributed augmented lagrangians adal algorithm that was recently developed by the authors to address convex problems here we extend this line of work in two ways first we establish convergence of the method to a local minimum of the problem using assumptions that are common in the analysis of nonconvex optimization methods to the best of our knowledge this is the first work that shows convergence to local minima specifically for a distributed augmented lagrangian al method applied to nonconvex optimization problems distributed al methods are known to perform very well when used to solve convex problems second we propose a more general and decentralized rule to select the stepsizes of the method this improves on the authors original adal method where the stepsize selection used global information at initialization numerical results are included to verify the correctness and efficiency of the proposed distributed method

dual decomposition for multi agent distributed optimization with coupling constraints abstract we study distributed optimization in a cooperative multi agent setting where agents have to agree on the usage of shared resources and can communicate via a time varying network to this purpose each agent has its own decision variables that should be set so as to minimize its individual objective function subject to local constraints resource sharing is modeled via coupling constraints that involve the non positivity of the sum of agents individual functions each one depending on the decision variables of one single agent we propose a novel distributed algorithm to minimize the sum of the agents objective functions subject to both local and coupling constraints where dual decomposition and proximal minimization are combined in an iterative scheme notably privacy of information is guaranteed since only the dual optimization variables associated with the coupling constraints are exchanged by the agents under convexity assumptions jointly with suitable connectivity properties of the communication network we are able to prove that agents reach consensus to some optimal solution of the centralized dual problem counterpart while primal variables converge to the set of optimizers of the centralized primal problem the efficacy of the proposed approach is demonstrated on a plug in electric vehicles charging problem

solving convex programs by random walks minimizing a convex function over a convex set in n dimensional space is a basic general problem with many interesting special cases here we present a simple new algorithm for convex optimization based on sampling by a random walk it extends naturally to minimizing quasi convex functions and to other generalizations

accelerated dual descent for network flow optimization we present a fast distributed solution to the convex network flow optimization problem our approach uses a family of dual descent algorithms that approximate the newton direction to achieve faster convergence rates than existing distributed methods the approximate newton directions are obtained through matrix splitting techniques and sparse taylor approximations of the inverse hessian we couple this descent direction with a distributed line search algorithm which requires the same information as our descent direction to compute we show that similarly to conventional newton methods the proposed algorithm exhibits super linear convergence within a neighborhood of the optimal value numerical experiments corroborate that convergence times are between one to two orders of magnitude faster than existing distributed optimization methods a connection with recent developments that use consensus to compute approximate newton directions is also presented

simulated annealing for convex optimization we apply the method known as simulated annealing to the following problem in convex optimization minimize a linear function over an arbitrary convex set where the convex set is specified only by a membership oracle using distributions from the boltzmann gibbs family leads to an algorithm that needs only o n phases for instances in rn this gives an optimization algorithm that makes o n4 5 calls to the membership oracle in the worst case compared to the previous best guarantee of o n5 the benefits of using annealing here are surprising because such problems have no local minima that are not also global minima hence we conclude that one of the advantages of simulated annealing in addition to avoiding poor local minima is that in these problems it converges faster to the minima that it finds we also give a proof that under certain general conditions the boltzmann gibbs distributions are optimal for annealing on these convex problems

on multivariate quantiles under partial orders this paper focuses on generalizing quantiles from the ordering point of view we propose the concept of partial quantiles which are based on a given partial order we establish that partial quantiles are equivariant under order preserving transformations of the data robust to outliers characterize the probability distribution if the partial order is sufficiently rich generalize the concept of efficient frontier and can measure dispersion from the partial order perspective we also study several statistical aspects of partial quantiles we provide estimators associated rates of convergence and asymptotic distributions that hold uniformly over a continuum of quantile indices furthermore we provide procedures that can restore monotonicity properties that might have been disturbed by estimation error establish computational complexity bounds and point out a concentration of measure phenomenon the latter under independence and the componentwise natural order finally we illustrate the concepts by discussing several theoretical examples and simulations empirical applications to compare intake nutrients within diets to evaluate the performance of investment funds and to study the impact of policies on tobacco awareness are also presented to illustrate the concepts and their use

the volume algorithm producing primal solutions with a subgradient method we present an extension to the subgradient algorithm to produce primal as well as dual solutions it can be seen as a fast way to carry out an approximation of dantzig wolfe decomposition this gives a fast method for producing approximations for large scale linear programs it is based on a new theorem in linear programming duality we present successful experience with linear programs coming from set partitioning set covering max cut and plant location

on the convergence time of dual subgradient methods for strongly convex programs this paper studies the convergence time of dual gradient methods for general possibly nondifferentiable strongly convex programs for general convex programs the convergence time of dual subgradient gradient methods with simple running averages running averages started from iteration 0 is known to be o frac 1 epsilon 2 this paper shows that the convergence time for general strongly convex programs is o frac 1 epsilon this paper also considers a variation of the average scheme called the sliding running averages and shows that if the dual function of the strongly convex program is locally quadratic then the convergence time of the dual gradient method with sliding running averages is o log frac 1 epsilon the convergence time analysis is further verified by numerical experiments

rate analysis of inexact dual first order methods application to dual decomposition we propose and analyze two dual methods based on inexact gradient information and averaging that generate approximate primal solutions for smooth convex problems the complicating constraints are moved into the cost using the lagrange multipliers the dual problem is solved by inexact first order methods based on approximate gradients for which we prove sublinear rate of convergence in particular we provide a complete rate analysis and estimates on the primal feasibility violation and primal and dual suboptimality of the generated approximate primal and dual solutions moreover we solve approximately the inner problems with a linearly convergent parallel coordinate descent algorithm our analysis relies on the lipschitz property of the dual function and inexact dual gradients further we combine these methods with dual decomposition and constraint tightening and apply this framework to linear model predictive control obtaining a suboptimal and feasible control scheme

discrete hit and run for sampling points from arbitrary distributions over subsets of integer hyperrectangles we consider the problem of sampling a point from an arbitrary distribution over an arbitrary subset s of an integer hyperrectangle neither the distribution nor the support set s are assumed to be available as explicit mathematical equations but may only be defined through oracles and in particular computer programs this problem commonly occurs in black box discrete optimization as well as counting and estimation problems the generality of this setting and high dimensionality of s precludes the application of conventional random variable generation methods as a result we turn to markov chain monte carlo mcmc sampling where we execute an ergodic markov chain that converges to so that the distribution of the point delivered after sufficiently many steps can be made arbitrarily close to unfortunately classical markov chains such as the nearest neighbor random walk or the coordinate direction random walk fail to converge to because they can get trapped in isolated regions of the support set to surmount this difficulty we propose discrete hit and run dhr a markov chain motivated by the hit and run algorithm known to be the most efficient method for sampling from log concave distributions over convex bodies in rn we prove that the limiting distribution of dhr is as desired thus enabling us to sample approximately from by delivering the last iterate of a sufficiently large number of iterations of dhr in addition to this asymptotic analysis we investigate finite time behavior of dhr and present a variety of examples where dhr exhibits polynomial performance

random vectors in the isotropic position abstract letybe a random vector in r n satisfying e y y id letmbe a natural number and lety1 ymbe independent copies ofy we study the question of approximation of the identity operator by finite sums of the tensorsyi yi we prove that for some absolute constantc e 1 m i 1 m y i y i id c log n m e y log m 1 log m provided that the last expression is smaller than 1 we apply this estimate to improve a result of bourgain concerning the number of random points needed to bring a convex body into a nearly isotropic position

lagrangian relaxation via ballstep subgradient methods we exhibit useful properties of ballstep subgradient methods for convex optimization using level controls for estimating the optimal value augmented with simple averaging schemes they asymptotically find objective and constraint subgradients involved in optimality conditions when applied to lagrangian relaxation of convex programs they find both primal and dual solutions and have practicable stopping criteria up until now similar results have only been known for proximal bundle methods and for subgradient methods with divergent series stepsizes whose convergence can be slow encouraging numerical results are presented for large scale nonlinear multicommodity network flow problems

on linear convergence of a distributed dual gradient algorithm for linearly constrained separable convex problems in this paper we propose a fully distributed dual gradient algorithm for minimizing linearly constrained separable convex problems and analyze its rate of convergence in particular we prove that under the assumption of strong convexity and lipschitz continuity of the gradient of the primal objective function we have a global error bound type property for the dual problem using this error bound property we devise a fully distributed dual gradient scheme i e a gradient scheme based on a weighted step size for which we derive global linear rate of convergence for both dual and primal suboptimality and for primal feasibility violation numerical simulations are also provided to confirm our theory

fast global kernel density mode seeking applications to localization and tracking tracking objects in video using the mean shift ms technique has been the subject of considerable attention in this work we aim to remedy one of its shortcomings ms like other gradient ascent optimization methods is designed to find local modes in many situations however we seek the global mode of a density function the standard ms tracker assumes that the initialization point falls within the basin of attraction of the desired mode when tracking objects in video this assumption may not hold particularly when the target s displacement between successive frames is large in this case the local and global modes do not correspond and the tracker is likely to fail a novel multibandwidth ms procedure is proposed which converges to the global mode of the density function regardless of the initialization point we term the procedure annealed ms as it shares similarities with the annealed importance sampling procedure the bandwidth of the procedure plays the same role as the temperature in conventional annealing we observe that an over smoothed density function with a sufficiently large bandwidth is unimodal using a continuation principle the influence of the global peak in the density function is introduced gradually in this way the global maximum is more reliably located since it is imperative that the computational complexity is minimal for real time applications such as visual tracking we also propose an accelerated version of the algorithm this significantly decreases the number of iterations required to achieve convergence we show on various data sets that the proposed algorithm offers considerable promise in reliably and rapidly finding the true object location when initialized from a distant point

a saddle point algorithm for networked online convex optimization an algorithm to learn optimal actions in convex distributed online problems is developed learning is online because cost functions are revealed sequentially and distributed because they are revealed to agents of a network that can exchange information with neighboring nodes only learning is measured in terms of the global network regret which is defined here as the accumulated loss of causal prediction with respect to a centralized clairvoyant agent to which the information of all times and agents is revealed at the initial time a variant of the arrow hurwicz saddle point algorithm is proposed to control the growth of global network regret this algorithm uses lagrange multipliers to penalize the discrepancies between agents and leads to an implementation that relies on local operations and exchange of variables between neighbors we show that decisions made with this saddle point algorithm lead to regret whose order is not larger than o sqrt t where t is the total operating time numerical behavior is illustrated for the particular case of distributed recursive least squares an application to computer network security in which service providers cooperate to detect the signature of malicious users is developed to illustrate the practical value of the proposed algorithm

computational results of an o n4 volume algorithm abstract recently an o n4 volume algorithm has been presented for convex bodies by lovasz and vempala where n is the number of dimensions of the convex body essentially the algorithm is a series of monte carlo integrations in this paper we describe a computer implementation of the volume algorithm where we improved the computational aspects of the original algorithm by adding variance decreasing modifications a stratified sampling strategy double point integration and orthonormalised estimators formulas and methodology were developed so that the errors in each phase of the algorithm can be controlled some computational results for convex bodies in dimensions ranging from 2 to 10 are presented as well

computational complexity of inexact gradient augmented lagrangian methods application to constrained mpc we study the computational complexity certification of inexact gradient augmented lagrangian methods for solving convex optimization problems with complicated constraints we solve the augmented lagrangian dual problem that arises from the relaxation of complicating constraints with gradient and fast gradient methods based on inexact first order information moreover since the exact solution of the augmented lagrangian primal problem is hard to compute in practice we solve this problem up to some given inner accuracy we derive relations between the inner and the outer accuracy of the primal and dual problems and we give a full convergence rate analysis for both gradient and fast gradient algorithms we provide estimates on the primal and dual suboptimality and on primal feasibility violation of the generated approximate primal and dual solutions our analysis relies on the lipschitz property of the dual function and on inexact dual gradients we also discuss implementation aspects of the proposed algor

exponential convergence of distributed primal dual convex optimization algorithm without strong convexity abstract this paper establishes exponential convergence rates for a class of primal dual gradient algorithms in distributed optimization without strong convexity the convergence analysis is based on a carefully constructed lyapunov function by evaluating metric subregularity of the primal dual gradient map we present a general criterion under which the algorithm achieves exponential convergence to facilitate practical applications of this criterion several simplified sufficient conditions are derived we also prove that although these results are developed for the continuous time algorithms they carry over in a parallel manner to the discrete time algorithms constructed by using euler s approximation method