combinatorial interior point methods for generalized network flow problems we present combinatorial interior point methods for the generalized minimum cost flow and the generalized circulation problems based on wallacher and zimmermann s combinatorial interior point method for the minimum cost network flow problem the algorithms have features of both a combinatorial algorithm and an interior point method they work towards optimality by iteratively reducing the value of a potential function while maintaining interior point solutions at each iteration flow is augmented along a generalized circulation which is computed by solving a tvpi two variables per inequality system the algorithms run in time where m and n are respectively the number of arcs and nodes in the graph and l is the length of the input data

global convergence of a cass of quasi newton methods on convex problems we study the global convergence properties of the restricted broyden class of quasi newton methods when applied to a convex objective function we assume that the line search satisfies a standard sufficient decrease condition and that the initial hessian approximation is any positive definite matrix we show global and superlinear convergence for this class of methods except for dfp this generalizes powell s well known result for the bfgs method the analysis gives us insight into the properties of these algorithms in particular it shows that dfp lacks a very desirable self correcting property possessed by bfgs

analyzing fractional horn constraint systems abstract in this paper we study linear constraint systems in which each constraint is a fractional horn constraint a constraint is fractional horn if it can be written in the form i 1 n a i x i c where each a i is integral c is integral at most one of the a i s is positive and all negative a i s are equal to 1 a conjunction of fractional horn constraints is called a fractional horn constraint systems fhs fhss generalize a number of specialized constraint systems such as difference constraint systems dcs and horn constraint systems hcs we know that the problem of checking lattice point feasibility in dcss and hcss is in p in contrast we show that the problem of checking lattice point feasibility in fhss is np complete we then study a sub class of fractional horn constraint systems called binary fractional horn constraint systems bfhs in a bfhs each constraint has at most two non zero coefficients in this case we show that the problem of checking integer feasibility is in p furthermore we establish several properties of the least integer point and the least linear point of a bfhs if they exist

bimonotone linear inequalities and sublattices of rn abstract a bimonotone linear inequality is a linear inequality with at most two nonzero coefficients that are of opposite signs if both different from zero a linear inequality defines a halfspace that is a sublattice of r n a subset closed with respect to componentwise maximum and minimum if and only if it is bimonotone veinott has shown that a polyhedron is a sublattice if and only if it can be defined by a finite system of bimonotone linear inequalities whereas topkis has shown that every sublattice of r n and of more general product lattices is the solution set of a system of nonlinear bimonotone inequalities in this paper we prove that a subset of r n is the solution set of a countable system of bimonotone linear inequalities if and only if it is a closed convex sublattice similarly we note that a subset of r n is closed and convex if and only if it is the solution set of a countable system of linear inequalities we also present necessary and or sufficient conditions for a sublattice to be the intersection of the cartesian product of its projections on the coordinate axes with the solution set of a possibly infinite system of bimonotone linear inequalities we provide explicit constructions of such systems of bimonotone linear inequalities under certain assumptions on the sublattice we obtain veinott s polyhedral representation theorem and a 0 1 version of birkhoff s representation theorem as corollaries we also point out a few potential pitfalls regarding properties of sublattices of r n

res regularized stochastic bfgs algorithm res a regularized stochastic version of the broyden fletcher goldfarb shanno bfgs quasi newton method is proposed to solve strongly convex optimization problems with stochastic objectives the use of stochastic gradient descent algorithms is widespread but the number of iterations required to approximate optimal arguments can be prohibitive in high dimensional problems application of second order methods on the other hand is impracticable because the computation of objective function hessian inverses incurs excessive computational cost bfgs modifies gradient descent by introducing a hessian approximation matrix computed from finite gradient differences res utilizes stochastic gradients in lieu of deterministic gradients for both the determination of descent directions and the approximation of the objective function s curvature since stochastic gradients can be computed at manageable computational cost res is realizable and retains the convergence rate advantages of its deterministic counterparts convergence results show that lower and upper bounds on the hessian eigenvalues of the sample functions are sufficient to guarantee almost sure convergence of a subsequence generated by res and convergence of the sequence in expectation to optimal arguments numerical experiments showcase reductions in convergence time relative to stochastic gradient descent algorithms and non regularized stochastic versions of bfgs an application of res to the implementation of support vector machines is developed

querying temporal and spatial constraint networks in ptime we start with the assumption that temporal and spatial knowledge usually captured by constraint networks can be represented and queried more effectively by using the scheme of indefinite constraint databases because query evaluation in this scheme is in general a hard computational problem we seek tractable instances of query evaluation we assume that we have a class of constraints c with some reasonable computational and closure properties the computational properties of interest are that the satisfiability problem and an appropriate version of the variable elimination problem for c should be solvable in ptime under this assumption we exhibit general classes of indefinite constraint databases and first order modal queries for which query evaluation can be done with ptime data complexity we then search for tractable instances of c among the subclasses of horn disjunctive linear constraints over the rationals from previous research we know that the satisfiability problem for horn disjunctive linear constraints is solvable in ptime but not the variable elimination problem thus we try to discover subclasses of horn disjunctive linear constraints with tractable variable elimination problems the class of utvpi constraints is the largest class that we show to have this property finally we restate our general tractability results with c ranging over the newly discovered tractable classes interesting tractable query answering problems for indefinite temporal and spatial constraint databases are identified in this way we close our complexity analysis by precisely outlining the frontier between tractable and possibly intractable query answering problems

a polynomial combinatorial algorithm for generalized minimum cost flow we propose the first combinatorial solution to the generalized minimum cost flow problem flow with losses and gains despite a rich history dating back to kantorovich and dantzig until now the only known way to solve the problem in polynomial time was via general purpose linear programming techniques polynomial combinatorial algorithms were previously known only for the version of our problem without costs we design the first such algorithms for the version with costs our algorithms also find provably good solutions faster than optimal ones providing the first strongly polynomial approximation schemes for the problem our techniques extend to optimize linear programs with two variables per inequality polynomial combinatorial algorithms were previously developed for testing the feasibility of such linear programs we propose the first such methods for the optimization version

an accelerated linearly convergent stochastic l bfgs algorithm the limited memory version of the broyden fletcher goldfarb shanno l bfgs algorithm is the most popular quasi newton algorithm in machine learning and optimization recently it was shown that the stochastic l bfgs sl bfgs algorithm with the variance reduced stochastic gradient converges linearly in this paper we propose a new sl bfgs algorithm by importing a proper momentum we prove an accelerated linear convergence rate under mild conditions the experimental results on different data sets also verify this acceleration advantage

fast and simple approximation schemes for generalized flow we present fast and simple fully polynomial time approximation schemes fptas for generalized versions of maximum flow multicommodity flow minimum cost maximum flow and minimum cost multicommodity flow we extend and refine fractional packing frameworks introduced in fptas s for traditional multicommodity flow and packing linear programs our fptas s dominate the previous best known complexity bounds for all of these problems some by more than a factor of n2 where n is the number of nodes this is accomplished in part by introducing an efficient method of solving a sequence of generalized shortest path problems our generalized multicommodity fptas s are now as fast as the best non generalized ones we believe our improvements make it practical to solve generalized multicommodity flow problems via combinatorial methods

simple and fast algorithms for linear and integer programs with two variables per inequality the authors present an o mn 2 log m algorithm for solving feasibility in linear programs with up to two variables per inequality which is derived directly from the fourier motzkin elimination method the number of variables and inequalities are denoted by n and m respectively the running time of the algorithm dominates that of the best known algorithm for the problem and is far simpler integer programming on monotone inequalities i e inequalities where the coefficients are of opposite sign is then considered this problem includes as a special case the simultaneous approximation of a rational vector with specified accuracy which is known to be np complete however it is shown that both a feasible solution and an optimal solution with respect to an arbitrary objective function can be computed in pseudo polynomial time

incremental closure for systems of two variables per inequality subclasses of linear inequalities where each inequality has at most two vari ables are popular in abstract interpretation and model checking because they strike a balance between what can be described and what can be efficiently computed this paper focuses on the tvpi class of inequalities for which each coefficient of each two variable inequality is unrestricted an implied tvpi in equality can be generated from a pair of tvpi inequalities by eliminating a given common variable echoing resolution on clauses this operation called result can be applied to derive tvpi inequalities which are entailed implied by a given tvpi system the key operation on tvpi is calculating closure satisfiability can be observed from a closed system and a closed system also simplifies the calculation of other operations a closed system can be derived by repeatedly applying the result operator the process of adding a single tvpi inequality to an already closed input tvpi system and then finding the closure of this augmented system is called incremental closure this too can be calcu lated by the repeated application of the result operator this paper studies the calculus defined by result the structure of result derivations and how deriva tions can be combined and controlled a series of lemmata on derivations are presented that collectively provide a pathway for synthesising an algorithm for incremental closure the complexity of the incremental closure algorithm is analysed and found to be o n 2 m 2 lg m where n is the number of variables and m the number of inequalities of the input tvpi system

on the limited memory bfgs method for large scale optimization we study the numerical performance of a limited memory quasi newton method for large scale optimization which we call the l bfgs method we compare its performance with that of the method developed by buckley and lenir 1985 which combines cycles of bfgs steps and conjugate direction steps our numerical tests indicate that the l bfgs method is faster than the method of buckley and lenir and is better able to use additional storage to accelerate convergence we show that the l bfgs method can be greatly accelerated by means of a simple scaling we then compare the l bfgs method with the partitioned quasi newton method of griewank and toint 1982a the results show that for some problems the partitioned quasi newton method is clearly superior to the l bfgs method however we find that for other problems the l bfgs method is very competitive due to its low iteration cost we also study the convergence properties of the l bfgs method and prove global convergence on uniformly convex problems

the simplex method is strongly polynomial for deterministic markov decision processes we prove that the simplex method with the highest gain most negative reduced cost pivoting rule converges in strongly polynomial time for deterministic markov decision processes mdps regardless of the discount factor for a deterministic mdp with n states and m actions we prove the simplex method runs in o n3 m2 log2 n iterations if the discount factor is uniform and o n5 m3 log2 n iterations if each action has a distinct discount factor previously the simplex method was known to run in polynomial time only for discounted mdps where the discount was bounded away from 1

local spectral convergence in rcd k n spaces abstract in this paper we give necessary and sufficient conditions for the validity of the local spectral convergence in balls in the setting of rcd spaces

solving integer programs over monotone inequalities in three variables a framework for half integrality and good approximations abstract we define a class of monotone integer programs with constraints that involve up to three variables each a generic constraint in such integer program is of the form ax by z c where a and b are nonnegative and the variable z appears only in that constraint we devise an algorithm solving such problems in time polynomial in the length of the input and the range of variables u the solution is obtained from a minimum cut on a graph with o nu nodes and o mu arcs where n is the number of variables of the types x and y and m is the number of constraints our algorithm is also valid for nonlinear objective functions nonmonotone integer programs are optimization problems with constraints of the type ax by z c without restriction on the signs of a and b such problems are in general np hard we devise here an algorithm relying on a transformation to the monotone case that delivers half integral superoptimal solutions in polynomial time such solutions provide bounds on the optimum value that can only be superior to bounds provided by linear programming relaxation when the half integral solution can be rounded to an integer feasible solution this is a 2 approximate solution in that the technique is a unified 2 approximation technique for a large class of problems the results apply also for general integer programming problems with worse approximation factors that depend on a quantifier measuring how far the problem is from the class of problems we describe the algorithm described here has a wide array of problem applications an additional important consequence of our results is that nonmonotone problems in the framework are max snp hard and at least as hard to approximate as vertex cover problems that are amenable to the analysis provided here are easily recognized the analysis itself is entirely technical and involves manipulating the constraints and transforming them to a totally unimodular system while losing no more than a factor of 2 in the integrality

a stochastic quasi newton method for large scale optimization the question of how to incorporate curvature information into stochastic approximation methods is challenging the direct application of classical quasi newton updating techniques for deterministic optimization leads to noisy curvature estimates that have harmful effects on the robustness of the iteration in this paper we propose a stochastic quasi newton method that is efficient robust and scalable it employs the classical bfgs update formula in its limited memory form and is based on the observation that it is beneficial to collect curvature information pointwise and at spaced intervals one way to do this is through subsampled hessian vector products this technique differs from the classical approach that would compute differences of gradients at every iteration and where controlling the quality of the curvature estimates can be difficult we present numerical results on problems arising in machine learning that suggest that the proposed method shows much promise

tight bounds and 2 approximation algorithms for integer programs with two variables per inequality the problem of integer programming in bounded variables over constraints with no more than two variables in each constraint is np complete even when all variables are binary this paper deals with integer linear minimization problems inn variables subject tom linear constraints with at most two variables per inequality and with all variables bounded between 0 andu for such systems a 2 approximation algorithm is presented that runs in time o mnu2 log un2m so it is polynomial in the input size if the upper boundu is polynomially bounded the algorithm works by finding first a super optimal feasible solution that consists of integer multiples of 1 2 that solution gives a tight bound on the value of the minimum it furthermore has an identifiable subset of integer components that retain their value in an integer optimal solution of the problem these properties are a generalization of the properties of the vertex cover problem the algorithm described is in particular a 2 approximation algorithm for the problem of minimizing the total weight of true variables among all truth assignments to the 2 satisfiability problem

iqn an incremental quasi newton method with local superlinear convergence rate the problem of minimizing an objective that can be written as the sum of a set of n smooth and strongly convex functions is challenging because the cost of evaluating the function and its derivat

the linear complementarity problems with a few variables per constraint in this paper we consider the sparse linear complementarity problem denoted by k lcp the coefficient matrices are restricted to have at most k nonzero entries per row it is known that the 1 lcp is solvable in linear time and the 3 lcp is strongly np hard we show that the 2 lcp is strongly np hard and it can be solved in polynomial time if it is sign balanced i e each row of the matrix has at most one positive and one negative entry our second result matches the currently best known complexity bound for the corresponding sparse linear feasibility problem in addition we show that an integer variant of the sign balanced 2 lcp is weakly np hard and pseudo polynomially solvable and the generalized 1 lcp is strongly np hard

optimizing over consecutive 1 s and circular 1 s constraints we consider packing and covering optimization problems over constraints in consecutive and circular 1 s such problems arise in the context of shift scheduling and in problems related to interval graphs previous approaches to this problem depended on solving several minimum cost network flow problems we devise here substantially more efficient and strongly polynomial algorithms based on parametric shortest paths approaches the objective function in the covering and packing problems is to either minimize or maximize the number of sets that satisfy the constraints the various problems studied are classified according to whether the constraints are all consecutive 1 s or if there are also circular 1 s constraints and according to whether the constraints are all of covering type all of packing type or mixed the running time of our algorithm for a pure covering all consecutive 1 s constraints problem on n variables and m constraints is o m n for the pure packing problem with consecutive 1 s constraints we present an o m n log n time algorithm for the mixed case with both covering and packing consecutive 1 s constraints we present an o mn time algorithm an o mn n 2 log n time algorithm is presented for the case where the constraints are circular consecutive 1 s constraint is also circular of pure type either all covering constraints or all packing constraints finally we show an o n min mn n 2 log n m log 2 n time algorithm for the most general problem of mixed covering and packing case where the constraints are circular all our algorithms are strongly polynomial and improve on the nonstrongly polynomial parametric minimum cost network flow or the strongly polynomial linear programming known approaches

max ones generalized to larger domains we study a family of problems called maximum solution where the objective is to maximize a linear goal function over the feasible integer assignments to a set of variables subject to a set of constraints when the domain is boolean i e restricted to 0 1 the maximum solution problem is identical to the well studied max ones problem and the approximability is completely understood for all restrictions on the underlying constraints s khanna m sudan l trevisan and d p williamson siam j comput 30 2001 pp 1863 1920 we continue this line of research by considering domains containing more than two elements we present two main results a complete classification for the approximability of all maximal constraint languages over domains of cardinality at most 4 and a complete classification of the approximability of the problem when the set of allowed constraints contains all permutation constraints under the assumption that a conjecture due to szczepara minimal clones generated by groupoids ph d thesis universite de montreal montreal qc 1996 holds we give a complete classification for all maximal constraint languages these classes of languages are well studied in universal algebra and computer science they have for instance been considered in connection with machine learning and constraint satisfaction our results are proved by using algebraic results from clone theory and the results indicate that this approach is very powerful for classifying the approximability of certain optimization problems

adaptive subgradient methods for online learning and stochastic optimization we present a new family of subgradient methods that dynamically incorporate knowledge of the geometry of the data observed in earlier iterations to perform more informative gradient based learning metaphorically the adaptation allows us to find needles in haystacks in the form of very predictive but rarely seen features our paradigm stems from recent advances in stochastic optimization and online learning which employ proximal functions to control the gradient steps of the algorithm we describe and analyze an apparatus for adaptively modifying the proximal function which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal function that can be chosen in hindsight we give several efficient algorithms for empirical risk minimization problems with common and important regularization functions and domain constraints we experimentally study our theoretical analysis and show that adaptive subgradient methods outperform state of the art yet non adaptive subgradient algorithms

polynomial time computation of strong and n present value optimal policies in markov decision chains this paper studies the problem of finding a stationary strong present value optimal and more generally an n present value optimal policy for an infinite horizon stationary finite state action substochastic markov decision chain a policy is strong present value optimal if it has maximum present value for all small positive interest rates a policy is n present value optimal if its present value falls below the maximum present value for all small positive by o n 1 the importance of stationary n present value optimal policies arises from the following known facts the set of such policies diminishes with n and for n s is precisely the set of stationary strong present value optimal policies for 0 n s stationary n present value optimal policies are nearly strong present value optimal and are of independent interest the best algorithms for finding stationary strong present value optimal policies find stationary n present value policies for n 1 s in that order this paper decomposes th

global convergence of online limited memory bfgs global convergence of an online stochastic limited memory version of the broyden fletcher goldfarb shanno bfgs quasi newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established lower and upper bounds on the hessian eigenvalues of the sample functions are shown to suffice to guarantee that the curvature approximation matrices have bounded determinants and traces which in turn permits establishing convergence to optimal arguments with probability 1 experimental evaluation on a search engine advertising problem showcase reductions in convergence time relative to stochastic gradient descent algorithms

tractability conditions for numeric csps abstract the computational complexity of the constraint satisfaction problem csp with semilinear relations over the reals has gained recent attraction as a result its complexity is known for all finite sets of semilinear relations containing the relation r x y z r 3 x y z we consider larger and more expressive classes of relations such as semialgebraic and o minimal relations we present a general result for characterising computationally hard fragments and under certain side conditions this result implies that polynomial time solvable fragments are only to be found within two limited families of sets of relations in the setting of semialgebraic relation our result takes on a simplified form and we provide a full complexity classification for constraint languages that consist of algebraic varieties full classifications like the one obtained here for algebraic varieties or the one for semilinear relations appear to be rare and we discuss several barriers for obtaining further such results these barriers have strong connections with well known open problems concerning the complexity of various restrictions of convex programming