las vegas algorithms for linear and integer programming when the dimension is small this paper gives an algorithm for solving linear programming problems for a problem with n constraints and d variables the algorithm requires an expected o d2n logn o d d 2 o 1 o d4 n logn arithmetic operations as n the constant factors do not depend on d also an algorithm is given for integer linear programming let 4 bound the number of bits required to specify the rational numbers defining an input constraint or the objective function vector let n and d be as before then the algorithm requires expected o 2ddn 8dd n l n n ln n do d 4 ln n operations on numbers with do 1 4 bits as n where the constant factors do not depend on d or 4 to other convex programming problems for example an algorithm for finding the smallest sphere enclosing a set of n points in ed has the same time bound

a simplex algorithm whose average number of steps is bounded between two quadratic functions of the smaller dimension it has been a challenge for mathematicians to confirm theoretically the extremely good performance of simplex type algorithms for linear programming in this paper the average number of steps performed by a simplex algorithm the so called self dual method is analyzed the algorithm is not started at the traditional point 1 l t but points of the form 1 e e 2 t with e sufficiently small are used the result is better in two respects than those of the previous analyses first it is shown that the expected number of steps is bounded between two quadratic functions c 1 min m n 2 and c 2 min m n 2 of the smaller dimension of the problem this should be compared with the previous two major results in the field borgwardt proves an upper bound of o n 4 m 1 n 1 under a model that implies that the zero vector satisfies all the constraints and also the algorithm under his consideration solves only problems from that particular subclass smale analyzes the self dual algorithm starting at 1 1 t he shows that for any fixed m there is a constant c m such the expected number of steps is less than c m ln n m m 1 megiddo has shown that under smale s model an upper bound c m exists thus for the first time a polynomial upper bound with no restrictions except for nondegeneracy on the problem is proved and for the first time a nontrivial lower bound of precisely the same order of magnitude is established both borgwardt and smale require the input vectors to be drawn from spherically symmetric distributions in the model in this paper invariance is required only under certain

linear programming in linear time when the dimension is fixed it is demonstrated that the linear programming problem in d variables and n constraints can be solved in o n time when d is fixed this bound follows from a multidimensional search technique which is applicable for quadratic programming as well there is also developed an algorithm that is polynomial in both n and d provided d is bounded by a certain slowly growing function of n

optimal distributed execution of join queries it is proposed that the execution of a chain query in a distributed system can be usefully and appropriately modeled as an integer linear program in response to a user request information in the form of relational tables scattered across the network is to be combined and made available to the user the formulation initially attained by considering the behavior of the distributed system in processing such a query is then reduced by removing redundant linear constraints to produce a model of minimal transmission cost execution in view of varying properties displayed by the possibly many optima of this problem further attention is devoted to discriminating between them by perturbing the objective function those solutions requiring fewer network transmissions can be favored at the expense of equal cost but more complicated strategies this includes those strategies that may specify the transmission of a relation around a cycle when the costs of transmission between sites forming the cycle are zero such a solution might otherwise be optimal many different ways have been devised to solve programs having some number of variables restricted to taking only integer values in some interval and virtually any of these might be used to solve the join query model one possible method using a tree search approach is discussed here

the hirsch conjecture for dual transportation polyhedra an algorithm is given that joins any pair of extreme points of a dual transportation polyhedron by a path of at most m 1 n 1 extreme edges

a quasi polynomial bound for the diameter of graphs of polyhedra the diameter of the graph of a d dimensional polyhedron with n facets is at most nlog d 2 let p be a convex polyhedron the graph of p denoted by g p is an abstract graph whose vertices are the extreme points of p and two vertices u and v are adjacent if the interval v u is an extreme edge 1 dimensional face of p the diameter of the graph of p is denoted by p let d n be the maximal diameter of the graphs of d dimensional polyhedra p with n facets a facet is a d 1 dimensional face thus p is the set of solutions of n linear inequalities in d variables it is an old standing problem to determine the behavior of the function d n the value of d n is a lower bound for the number of iterations needed for dantzig s simplex algorithm for linear programming with any pivot rule in 1957 hirsch conjectured 2 that d n n d klee and walkup 6 showed that the hirsch conjecture is false for unbounded polyhedra they proved that for n 2d d n n d d 5 this is the best known lower bound for d n the statement of the hirsch conjecture for bounded polyhedra is still open for a recent survey on the hirsch conjecture and its relatives see 5 in 1967 barnette proved 1 3 that d n n3 an improved upper bound d n n2 was proved in 1970 by larman 7 barnette s and larman s bounds are linear in n but exponential in the dimension d in 1990 the first author 4 proved a subexponential bound d n 2 n d the purpose of this paper is to announce and to give a complete proof of a quasipolynomial upper bound for d n such a bound was proved by the first author in march 1991 the proof presented here is a substantial simplification that was subsequently found by the second author see 4 for the original proof and related results the existence of a polynomial or even linear upper bound for d n is still open recently the first author found a randomized pivot rule for linear programming which requires an expected n d or less arithmetic operations for every linear programming problem with d variables and n constraints 1991 mathematics subject classification primary 52a25 90c05 received by the editors july 1 1991 the first author was supported in part by a bsf grant by a gif grant the second author was supported by an afosr grant c 1992 american mathematical society 0273 0979 92 1 00 25 per page

on the length of simplex paths the assignment case this paper reports on an experimental study of the distribution of the length of simplex paths for the optimal assignment problem we study the distribution of the pivot counts for a version of the simplex method that with essentially equal probabilities introduces any variable with negative reduced cost into the basis in this situation the distribution of the pivot counts turns out to be normally distributed and independent of the actual cost coefficients provided these are sufficiently spread out further the mean and standard deviation grow only moderately with the size of the problem namely asd1 8 andd1 5 respectively for ad d problem implying in particular that the pivot counts concentrate around the mean with growingd the usual simplex method on the other hand gives a growth ofd1 6 hence a large part of the favourable polynomial growth experienced on practical problems may be attributed to the fact that the simplex paths are rather short on the average at least for assignment problems

on design of a survivable network architecture for dynamic routing optimal solution strategy and an efficient heuristic abstract we investigate network planning and design under volatile conditions of link failures and traffic overload our model is a non simultaneous multi commodity problem with any particular two link failure being considered as one scenario we show that the optimal solution model is not practically solvable for real world problems and hence an efficient heuristic is provided which is o n 6 faster than the optimal model and is based on synthesizing a modified maximum spanning tree using an algorithm due to gomory and hu the output of this procedure is then used to solve a much smaller linear program simulation results indicate that the heuristic is near optimal for problems with up to 40 nodes

low order polynomial bounds on the expected performance of local improvement algorithms we present a general abstract model of local improvement applicable to such diverse cases as principal pivoting methods for the linear complementarity problem and hill climbing in artificial intelligence the model accurately predicts the behavior of the algorithms and allows for a variety of probabilistic assumptions that permit degeneracy simulation indicates an approximately linear average number of iterations under a variety of probability assumptions we derive theoretical bounds of 2en logn and en2 for different distributions respectively as well as polynomial bounds for a broad class of probability distributions we conclude with a discussion of the applications of the model to lcp and linear programming

an algorithm for computing shape preserving interpolating splines of arbitrary degree abstract the aim of this paper is to describe an algorithm for computing co monotone and or co convex splines of degree m and deficiency m k at the knots 0 k m k which are interpolant or osculatory to a given set of data the method is based upon some existence properties recently developed graphical examples and a listing of the fortran code spisp1 are given

on the average speed of lemke s algorithm for quadratic programming in this paper we show that the average number of steps of the lemke algorithm for the quadratic programming problems grows at most linearly in the number of variables while fixing the number of constraints the result and method were motivated by smale s result on linear programming problems cf 4 we also give the probability that a quadratic programming problem indeed possesses a finite optimal solution

toward probabilistic analysis of interior point algorithms for linear programming we propose an approach based on interior point algorithms for linear programming lp we show that the algorithm solves a class of lp problems in strongly polynomial time o n log n iteration where each iteration solves a system of linear equations with n variables the recent statistical data of the solutions of the netlib lp problems seem to indicate that virtually all of these problems are in this class then we show that some random lp problems with high probability probability converges to one as n approaches infinity are in this class these random lp problems include recent todd s probabilistic models with the standard gauss distribution

convex cubic hermite spline interpolation we present a method to construct convex cubic c1 splines which interpolate a given convex data set the problem is reduced to the solution of a system of linear inequalities the existence of such convex interpolation splines is assured if the data fulfill slight additional conditions for stronger conditions some easier methods are developed finally error estimations are given

a strongly polynomial algorithm to solve combinatorial linear programs khachiyan and recently karmarkar gave polynomial algorithms to solve the linear programming problem these algorithms have a small theoretical drawback namely the number of arithmetic steps depends on the size of the input numbers we present a polynomial linear programming algorithm whose number of arithmetic steps depends only on the size of the numbers in the constraint matrix but is independent of the size of the numbers in the right hand side and objective vectors in particular it gives a polynomial algorithm for the minimum cost flow and multicommodity flow problems in which the number of arithmetic steps is independent of the size of the costs and capacities the algorithm makes use of an existing polynomial linear programming algorithm the problem of whether any algorithm has a running time that is independent even of the size of the numbers in the constraint matrix remains open

a note on gauss hermite quadrature summary for gauss hermite quadrature we consider a systematic method for transforming the variable of integration so that the integrand is sampled in an appropriate region the effectiveness of the quadrature then depends on the ratio of the integrand to some gaussian density being a smooth function well approximated by a low order polynomial it is pointed out that in this approach order one gauss hermite quadrature becomes the laplace approximation thus the quadrature as implemented here can be thought of as a higher order laplace approximation

on the average number of steps of the simplex method of linear programming the goal is to give some theoretical explanation for the efficiency of the simplex method of george dantzig fixing the number of constraints and using dantzig s self dual parametric algorithm we show that the number of pivots required to solve a linear programming problem grows in proportion to the number of variables on the average

variational fourier features for gaussian processes this work brings together two powerful concepts in gaussian processes the variational approach to sparse approximation and the spectral representation of gaussian processes this gives rise to an approximation that inherits the benefits of the variational approach but with the representational power and computational scalability of spectral representations the work hinges on a key result that there exist spectral features related to a finite domain of the gaussian process which exhibit almost independent covariances we derive these expressions for matern kernels in one dimension and generalize to more dimensions using kernels with specific structures under the assumption of additive gaussian noise our method requires only a single pass through the data set making for very fast and accurate computation we fit a model to 4 million training points in just a few minutes on a standard laptop with non conjugate likelihoods our mcmc scheme reduces the cost of computation from o nm2 for a sparse gaussian process to o nm per iteration where n is the number of data and m is the number of features

on the complexity of a pivot step of the revised simplex algorithm abstract a modification of the revised simplex algorithm is considered where every step involves o m2 arithmetical operations the error analysis and the bit complexity estimates presented show very strong stability of the modified algorithm the same modification can be included into some other linear algebra algorithms

fast finite methods for a system of linear inequalities abstract 1 several algorithms are known that solve a system of m linear inequalities in n variables using a polynomial number of steps in m and n for all inputs the simplex method runs in polynomial time in the average case proven and requires only linear in m number of steps in practice we consider a family of finite algorithms some of them involve o m n log3 n iteration steps in the average case under a certain additional assumption about the behavior of the computed approximations to a solution and only o m m n arithmetical operations are required per step the theoretical study is confirmed by some numerical experiments also two methods are presented that enable us to turn any converging algorithm for a system of linear inequalities with real inputs into an algorithm that converges in a finite number of steps

gaussian kullback leibler approximate inference we investigate gaussian kullback leibler g kl variational approximate inference techniques for bayesian generalised linear models and various extensions in particular we make the following novel contributions sufficient conditions for which the g kl objective is differentiable and convex are described constrained parameterisations of gaussian covariance that make g kl methods fast and scalable are provided the lower bound to the normalisation constant provided by g kl methods is proven to dominate those provided by local lower bounding methods complexity and model applicability issues of g kl versus other gaussian approximate inference methods are discussed numerical results comparing g kl and other deterministic gaussian approximate inference methods are presented for robust gaussian process regression models with either student t or laplace likelihoods large scale bayesian binary logistic regression models and bayesian sparse linear models for sequential experimental design

a new conjugate gradient method with guaranteed descent and an efficient line search a new nonlinear conjugate gradient method and an associated implementation based on an inexact line search are proposed and analyzed with exact line search our method reduces to a nonlinear version of the hestenes stiefel conjugate gradient scheme for any inexact line search our scheme satisfies the descent condition bf g k sf t bf d k le frac 7 8 bf g k 2 moreover a global convergence result is established when the line search fulfills the wolfe conditions a new line search scheme is developed that is efficient and highly accurate efficiency is achieved by exploiting properties of linear interpolants in a neighborhood of a local minimizer high accuracy is achieved by using a convergence criterion which we call the approximate wolfe conditions obtained by replacing the sufficient decrease criterion in the wolfe conditions with an approximation that can be evaluated with greater precision in a neighborhood of a local minimum than the usual sufficient decrease criterion numerical comparisons are given with both l bfgs and conjugate gradient methods using the unconstrained optimization problems in the cute library

solving lp problems via weighted centers the feasibility problem for a system of linear inequalities can be converted into an unconstrained optimization problem by using ideas from the ellipsoid method which can be viewed as a very simple minimization technique for the resulting nonlinear function this function is related to the volume of an ellipsoid containing all feasible solutions which is parametrized by certain weights which we choose to minimize the function the center of the resulting ellipsoid turns out to be a feasible solution to the inequalities using more sophisticated nonlinear minimization algorithms we develop and investigate more efficient methods which lead to two kinds of weighted centers for the feasible set using these centers we develop new algorithms for solving linear programming problems

semiparametric mean field variational bayes general principles and numerical issues we introduce the term semiparametric mean field variational bayes to describe the relaxation of mean field variational bayes in which some density functions in the product density restriction are pre specified to be members of convenient parametric families this notion has appeared in various guises in the mean field variational bayes literature during its history and we endeavor to unify this important topic we lay down a general framework and explain how previous relevant methodologies fall within this framework a major contribution is elucidation of numerical issues that impact semiparametric mean field variational bayes in practice

new results on the average behavior of simplex algorithms it has been a challenge for mathematicians to theoreti cally confirm the extremely good performance of simplex algorithms for linear programming we have confirmed that a certain variant of the simplex method solves problems of order m x n in an expected num ber of steps which is bounded between two quadratic functions of the smaller dimension of the problem our probabilistic assumptions are rather weak 1 introduction we consider the linear programming problem of order m x n in the form maximize ctx over all x 0 in rn such that ax 5 6 where a e rmxn it has been observed that the simplex algorithms for linear programming developed by george dantzig dl work extremely well a central question in the field of analysis of algorithms is to estimate the expected number of steps that these algorithms perform relative to different probability distributions of inputs some background is given in 52 in am1 and t we consider a variant of the simplex method called the lexicographic self dual method we find that the expected number of steps of this variant relative to a rather weak probabilistic model is only o min m n 2 this is the first polynomial average case upper bound for a simplex algorithm which is capable of solving any linear programming problem the first two authors am21 have also determined a quadratic lower bound so the behavior of this variant is indeed quadratic whereas it has been conjectured that other variants take a linear expected number of steps with various choices of the starting point this variant can simulate different constraint by constraint and variable dimension algorithms as well as combinations thereof me2 adler karp and shamir aks provide a different proof of similar upper bound for a certain class of constraint by constraint algorithms it follows from me21 that their variant can also be simulated by the self dual algorithm with an appropriate starting point

some distribution independent results about the asymptotic order of the average number of pivot steps of the simplex method this paper is concerned with the average number of pivot steps of the simplex method which are required to solve linear programming problems of the following kind mbox max v t x quad mbox subject to a t i x leq 1 quad mbox for i 1 ldots m mbox where a i v x in mathbb r n we investigate a certain variant of the simplex method which is particularly suited for theoretical considerations for every pair m nm n we introduce probability spaces whose random samples are the linear programming problems of the above mentioned type the vectors ai v are supposed to be distributed independently identically and symmetrically under rotations on a bounded subset of rn we deduce upper and lower bounds for the expectation values of the number of steps required in phase ii of the simplex method the evaluation of these bounds yields the following result for m and for fixed n 1 the expectation values become om1 n 1 2 there exists a distribution whose expectation values increase like m1 n 1 3 for any distribution which satisfies our conditions the expectation values tend to infinity 4 for every e 0 there are distributions whose expectation values become ome

approximate riemannian conjugate gradient learning for fixed form variational bayes variational bayesian vb methods are typically only applied to models in the conjugate exponential family using the variational bayesian expectation maximisation vb em algorithm or one of its variants in this paper we present an efficient algorithm for applying vb to more general models the method is based on specifying the functional form of the approximation such as multivariate gaussian the parameters of the approximation are optimised using a conjugate gradient algorithm that utilises the riemannian geometry of the space of the approximations this leads to a very efficient algorithm for suitably structured approximations it is shown empirically that the proposed method is comparable or superior in efficiency to the vb em in a case where both are applicable we also apply the algorithm to learning a nonlinear state space model and a nonlinear factor analysis model for which the vb em is not applicable for these models the proposed algorithm outperforms alternative gradient based methods by a significant margin

frequency response algorithms for h sub infinity optimization with time domain constraints a very broad framework for control system design is considered that encompasses frequency response methodologies for h sub infinity optimization that solve various aspects of the control design problem and that are less well known that state space methods the focus is on linear programming lawson s algorithm and trefethen s algorithm a modified lawson s algorithm is proposed and related to trefethan s method the modified algorithm is shown to be significantly faster than linear programming and lawson s algorithm it is also shown how to extend the modified lawson s algorithm so as to handle time domain constraints in addition to frequency domain specifications which distinguishes it from other h sub infinity optimization methods some steps are taken toward dealing with time domain constraints within an h sub infinity optimization framework

random polytopes their definition generation and aggregate properties the definition of random polytope adopted in this paper restricts consideration to those probability measures satisfying two properties first the measure must induce an absolutely continuous distribution over the positions of the bounding hyperplanes of the random polytope and second it must result in every point in the space being equally as likely as any other point of lying within the random polytope an efficient monte carlo method for their computer generation is presented together with analytical formulas characterizing their aggregate properties in particular it is shown that the expected number of extreme points for such random polytopes increases monotonically in the number of constraints to the limiting case of a polytope topologically equivalent to a hypercube the implied upper bound of 2n wheren is the dimensionality of the space is significantly less than mcmullen s attainable bound on the maximal number of vertices even for a moderate number of constraints

criss cross methods a fresh view on pivot algorithms criss cross methods are pivot algorithms that solve linear programming problems in one phase starting with any basic solution the first finite criss cross method was invented by chang terlaky and wang independently unlike the simplex method that follows a monotonic edge path on the feasible region the trace of a criss cross method is neither monotonic with respect to the objective function nor feasibility preserving the main purpose of this paper is to present mathematical ideas and proof techniques behind finite criss cross pivot methods a recent result on the existence of a short admissible pivot path to an optimal basis is given indicating shortest pivot paths from any basis might be indeed short for criss cross type algorithms the origins and the history of criss cross methods are also touched upon

frontiers of stochastically nondominated portfolios we consider the problem of constructing a portfolio of finitely many assets whose returns are described by a discrete joint distribution we propose mean risk models that are solvable by linear programming and generate portfolios whose returns are nondomi nated in the sense of second order stochastic dominance next we develop a specialized parametric method for recovering the entire mean risk efficient frontiers of these models and we illustrate its operation on a large data set involving thousands of assets and real izations the problem of optimizing a portfolio of finitely many assets is a classical problem in theoretical and computational finance since the seminal work of markowitz 1952 it is generally agreed that portfolio performance should be measured in two distinct dimen sions the mean describing the expected return and the risk which measures the uncer tainty of the return in the mean risk approach we select from the universe of all possible portfolios those that are efficient for a given value of the mean they minimize the risk or equivalently for a given value of risk they maximize the mean such an approach has many advantages it allows one to formulate the problem as a parametric optimization problem and it facilitates the trade off analysis between mean and risk markowitz used the variance of the return as the measure of the risk it is easy to compute and it reduces the portfolio selection problem to a parametric quadratic pro gramming problem one can however construct simple counterexamples that show the imperfection of the variance as the risk measure it treats over performance equally as under performance and more importantly its use may suggest a portfolio that is always outperformed by another portfolio the use of the semivariance rather than the variance was already recommended by markowitz 1959 but even in this case significant deficien cies remain as we shall explain another theoretical approach to the portfolio selection problem is that of stochastic dominance see whitmore and findlay 1978 levy 1992 the usual first order defi nition of stochastic dominance gives a partial order in the space of real random variables more important from the portfolio point of view is the notion of second order dominance which is also defined as a partial order but which is equivalent to this statement a random variable y dominates the random variable z if e u y e u z for all nondecreasing concave functions u for which these expected values are finite thus no risk averse decision marker will prefer a portfolio with return z over a portfolio with return y while

sparse approximate inference for spatio temporal point process models abstractspatio temporal log gaussian cox process models play a central role in the analysis of spatially distributed systems in several disciplines yet scalable inference remains computationally challenging both due to the high resolution modeling generally required and the analytically intractable likelihood function here we exploit the sparsity structure typical of spatially discretized log gaussian cox process models by using approximate message passing algorithms the proposed algorithms scale well with the state dimension and the length of the temporal horizon with moderate loss in distributional accuracy they hence provide a flexible and faster alternative to both nonlinear filtering smoothing type algorithms and to approaches that implement the laplace method or expectation propagation on block sparse latent gaussian models we infer the parameters of the latent gaussian model using a structured variational bayes approach we demonstrate the proposed framework on simulation studies with bo

efficient simplex like methods for equilibria of nonsymmetric analog networks what is the complexity of computing equilibria for physically implementable analog networks hopfield 1984 sejnowski 1981 with arbitrary connectivity we show that if the amplifiers are piecewise linear then such networks are instances of a game theoretic model known as polymatrix games in contrast with the usual gradient descent methods for symmetric networks equilibria for polymatrix games may be computed by vertex pivoting algorithms similar to the simplex method for linear programming like the simplex method these algorithms have characteristic low order polynomial behavior in virtually all practical cases though not certain theoretical ones while these algorithms cannot be applied to models requiring evolution from an initial point they are applicable to clamping models whose input is expressed purely as a bias thus we have an a priori indication that such models are computationally tractable

a family of simplex variants solving an m d linear program in expected number of pivot steps depending on d only we present a family of variants of the simplex method which are based on a constraint by constraint procedure the solution to a linear program is obtained by solving a sequence of subproblems with an increasing number of constraints we discuss several probabilistic models for generating linear programs in all of them the underlying distribution is assumed to be invariant under changing the signs of rows or columns in the problem data a weak regularity condition is also assumed under these models for linear programs with d variables and m d inequality constraints the expected number of pivots required by these algorithms is bounded by a function of minm d only in particular this means that for a fixed number of variables the expected number of pivots is bounded by a constant when the number of constraints tends to infinity since smale s original model smale s 1983 on the average speed of the simplex method of linear programming math programming27 satisfies our probabilistic assumptions the same results apply to his model although not to the particular algorithm he analyzes we also present some results for models generating only feasible linear programs and for bland s pivoting rule we conclude with a discussion of our probabilistic models and show why they are inadequate for obtaining meaningful results unless d and m are of the same order of magnitude

modifications and implementation of the ellipsoid algorithm for linear programming we give some modifications of the ellipsoid algorithm for linear programming and describe a numerically stable implementation we are concerned with practical problems where user supplied bounds can usually be provided our implementation allows constraint dropping and updates bounds on the optimal value and should be able to terminate with an indication of infeasibility or with a provably good feasible solution in a moderate number of iterations

probabilistic models for linear programming we propose and investigate new probabilistic models for linear programming in contrast to previous models ours guarantee the existence of optimal solutions and are symmetric under duality while in some respects our distributions are very special there is sufficient flexibility to permit an arbitrary degree of primal and or dual degeneracy either just at the optimal solution or throughout the feasible region using null variables moreover the precision of the distributions allows us to compute the probability that the feasible region is bounded as well as the distribution of the distance to a constraint hyperplane and that of the components of a vertex interest in these measures stems from karmarkar s algorithm and we also introduce a model for generating random linear programming problems on a simplex

the expected number of extreme points of a random linear program there has been increasing attention recently on average case algorithmic performance measures since worst case measures can be qualitatively quite different an important characteristic of a linear program relating to simplex method performance is the number of vertices of the feasible region we show 2n to be an upper bound on the mean number of extreme points of a randomly generated feasible region with arbitrary probability distributions on the constraint matrix and right hand side vector the only assumption made is that inequality directions are chosen independently in accordance with a series of independent fair coin tosses

a probabilistic analysis of a measure of combinatorial complexity for the central curve we investigate certain combinatorial properties of the central curve associated with interior point methods for linear optimization we define a measure of complexity for the curve in terms of the number of turns or changes of direction that it makes in a geometric sense and then perform an average case analysis of this measure for p matrix linear complementarity problems we show that the expected number of nondegenerate turns taken by the central curve is bounded by n 2 n where the expectation is taken with respect to a sign invariant probability distribution on the problem data as an alternative measure of complexity we also consider the number of times the central curve intersects with a wide class of algebraic hypersurfaces including such objects as spheres and boxes as an example of the results obtained we show that the primal and dual variables in each coordinate of the central curve cross each other at most once on average as a further example we show that the central curve intersects any sphere centered at the origin at most twice on average

worst case comparison of valid inequalities for the tsp we consider most of the known classes of valid inequalities for the graphical travelling salesman polyhedron and compute the worst case improvement resulting from their addition to the subtour polyhedron for example we show that the comb inequalities cannot improve the subtour bound by a factor greater than 10 9 the corresponding factor for the class of clique tree inequalities is 8 7 while it is 4 3 for the path configuration inequalities

efficiency of the primal network simplex algorithm for the minimum cost circulation problem we study the number of pivots required by the primal network simplex algorithm to solve the minimum cost circulation problem we propose a pivot selection rule with a bound of n logn 2 o 1 on the number of pivots for an n vertex network this is the first known subexponential bound the network simplex algorithm with this rule can be implemented to run in n logn 2 o 1 time in the special case of planar graphs we obtain a polynomial bound on the number of pivots and the running time we also consider the relaxation of the network simplex algorithm in which cost increasing pivots are allowed as well as cost decreasing ones for this algorithm we propose a pivot selection rule with a bound of o nm min log nc m log n on the number of pivots for a network with n vertices m arcs and integer arc costs bounded in magnitude by c the total running time is o nm log n min log nc m log n this bound is within a logarithmic factor of those of the best previously known algorithms for the minimum

a monotonic build up simplex algorithm for linear programming we devise a new simplex pivot rule which has interesting theoretical properties beginning with a basic feasible solution and any nonbasic variable having a negative reduced cost the pivot rule produces a sequence of pivots such that ultimately the originally chosen nonbasic variable enters the basis and all reduced costs which were originally nonnegative remain nonnegative the pivot rule thus monotonically builds up to a dual feasible and hence optimal basis a surprising property is that the pivot sequence results in intermediate bases which are neither primal nor dual feasible we prove the correctness of the procedure and relate it to other pivoting rules for linear programming

on the expected number of linear complementarity cones intersected by random and semi random rays lemke s algorithm for the linear complementarity problem follows a ray which leads from a certain fixed point traditionally the point 1 1 t to the point given in the problem the problem also induces a set of 2 n cones and a question which is relevant to the probabilistic analysis of lemke s algorithm is to estimate the expected number of times a semi random ray intersects the boundary between two adjacent cones when the problem is sampled from a spherically symmetric distribution this number turns out to be exponential for ann dimensional problem the natural logarithm of this number is equal to ln n o n where is approximately 1 151222 this number stands in sharp contrast with the expected number of cones intersected by a ray which is determined by two random points call itrandom the latter is only n 2 1 the discrepancy between linear behavior under the random assumption and exponential behavior under the semi random assumption has implications with respect to recent analyses of the average complexity of the linear programming problem surprisingly the semi random case is very sensitive to the fixed point of the ray even when that point is confined to the positive orthant we show that for points of the form e e 2 e n t the expected number of facets of cones cut by a semi random ray tends to 1 8n 2 3 8n whene tends to zero

solving large scale sparse semidefinite programs for combinatorial optimization we present a dual scaling interior point algorithm and show how it exploits the structure and sparsity of some large scale problems we solve the positive semidefinite relaxation of combinatorial and quadratic optimization problems subject to boolean constraints we report the first computational results of interior point algorithms for approximating maximum cut semidefinite programs with dimension up to 3 000

quantitative simulations by matrices abstract we introduce notions of simulation between semiring weighted automata as models of quantitative systems our simulations are instances of the categorical coalgebraic notions previously studied by hasuo hence soundness against language inclusion comes for free but are concretely presented as matrices that are subject to linear inequality constraints pervasiveness of these formalisms allows us to exploit existing algorithms in searching for a simulation and hence verifying quantitative correctness that is formulated as language inclusion transformations of automata that aid search for simulations are introduced too this verification workflow is implemented for the plus times and max plus semirings furthermore an extension to weighted tree automata is presented and implemented

the efficiency of the simplex method a survey the linear programming problem is by far the most widely used optimization model its impact on economic and government modeling is immense the simplex method for solving the linear programming lp problem due to george dantzig has been an extremely efficient computational tool for almost four decades the method has been the subject of intense investigations for many years but some major aspects of its behavior are not fully understood yet the purpose of this paper is to survey the body of knowledge on the efficiency of the simplex method from both practical and theoretical points of view adopting the number of iterations pivot steps as the yardstick for efficiency we survey four aspects of the issue 1 reports on practical experience of the method s performance on real life lp problems 2 results on controlled monte carlo experiments solving lp problems which were randomly generated according to some predetermined distributions 3 complexity results including theoretical analyses on both upper and lower bounds for the performance of the simplex as well as non simplex algorithms for lp 4 results of recent theoretical studies using probabilistic analysis to derive bounds on the average behavior of the simplex method we discuss the consequences and limitations of the various studies special emphasis is given to open questions

is binary encoding appropriate for the problem language relationship abstract it is proved that there exist encoding schemes which are arbitrarily as efficient as the binary encoding in terms of compactness and arithmetic operations with respect to which khachiyan s algorithm for linear programming is exponential this constitutes an objection to the standard translation of problems into languages via the binary encoding

average polynomial time complexity of some np complete problems abstract we define clique n n to be the problem clique restricted to the set of graphs g v e such that e n v in this paper it is proved that when n n n e e a rational number 0 clique n n is an np complete problem and under the assumption of a uniform distribution on the set of instances clique n n can be solved by a deterministic algorithm in average and almost everywhere polynomial time some cases of nonuniform distributions on the set of instances for which the usual problem clique is solved in average polynomial time are also considered

potential reduction methods in mathematical programming we provide a survey of interior point methods for linear programming and its extensions that are based on reducing a suitable potential function at each iteration we give a fairly complete overview of potential reduction methods for linear programming focusing on the possibility of taking long steps and the properties of the barrier function that are necessary for the analysis we then describe briefly how the methods and results can be extended to certain convex programming problems following the approach of nesterov and todd we conclude with some open problems

approximate marginals in latent gaussian models we consider the problem of improving the gaussian approximate posterior marginals computed by expectation propagation and the laplace method in latent gaussian models and propose methods that are similar in spirit to the laplace approximation of tierney and kadane 1986 we show that in the case of sparse gaussian models the computational complexity of expectation propagation can be made comparable to that of the laplace method by using a parallel updating scheme in some cases expectation propagation gives excellent estimates where the laplace approximation fails inspired by bounds on the correct marginals we arrive at factorized approximations which can be applied on top of both expectation propagation and the laplace method the factorized approximations can give nearly indistinguishable results from the non factorized approximations and their computational complexity scales linearly with the number of variables we experienced that the expectation propagation based marginal approximations we introduce are typically more accurate than the methods of similar complexity proposed by rue et al 2009

exact semidefinite formulations for a class of random and non random nonconvex quadratic programs we study a class of quadratically constrained quadratic programs qcqps called diagonal qcqps which contain no off diagonal terms x j x k for j ne k and we provide a sufficient condition on the problem data guaranteeing that the basic shor semidefinite relaxation is exact our condition complements and refines those already present in the literature and can be checked in polynomial time we then extend our analysis from diagonal qcqps to general qcqps i e ones with no particular structure by reformulating a general qcqp into diagonal form we establish new polynomial time checkable sufficient conditions for the semidefinite relaxations of general qcqps to be exact finally these ideas are extended to show that a class of random general qcqps has exact semidefinite relaxations with high probability as long as the number of constraints grows no faster than a fixed polynomial in the number of variables to the best of our knowledge this is the first result establishing the exactness of the semidefinite relaxation for random general qcqps

steepest edge simplex algorithms for linear programming we present several new steepest edge simplex algorithms for solving linear programming problems including variants of both the primal and the dual simplex method these algorithms differ depending upon the space in which the problem is viewed as residing and include variants in which this space varies dynamically we present computational results comparing steepest edge simplex algorithms and approximate versions of them against simplex algorithms that use standard pivoting rules on truly large scale realworld linear programs with as many as tens of thousands of rows and columns these results demonstrate unambiguously the superiority of steepest edge pivot selection criteria to other pivot selection criteria in the simplex method

a simple heuristic approach to simplex efficiency abstract consider the standard linear program minimize cx subject to ax b x 0 where a is an m n matrix the simplex algorithm solves this linear program by moving from an extreme point of the feasibility region to a better in terms of the objective function cx extreme point via the pivot operation until the optimal is reached in order to obtain a feel for the number of necessary iterations we consider a simple probabilistic markov chain model as to how the algorithm moves along the extreme points at first we suppose that if at any time the algorithm is at the j th best extreme point then after the next pivot the resulting extreme point is equally likely to be any of the j 1 best under this assumption we show that the time to get from the n th best to the best extreme point has approximately for large n a poisson distribution with mean equal to the algorithm base e of n we also consider a more general probabilistic model in which we drop the uniformity assumption and suppose that when at the j th best the next one is chosen probabilistically according to weights w i i 1 j 1

hard to solve bimatrix games the lemke howson algorithm is the classical method for finding one nash equilibrium of a bimatrix game this paper presents a class of square bimatrix games for which this algorithm takes even in the best case an exponential number of steps in the dimension d of the game using polytope theory the games are constructed using pairs of dual cyclic polytopes with 2d suitably labeled facets in d space the construction is extended to nonsquare games where in addition to exponentially long lemke howson computations finding an equilibrium by support enumeration takes on average exponential time

on the properties of variational approximations of gibbs posteriors the pac bayesian approach is a powerful set of techniques to derive nonasymptotic risk bounds for random estimators the corresponding optimal distribution of estimators usually called the gibbs posterior is unfortunately often intractable one may sample from it using markov chain monte carlo but this is usually too slow for big datasets we consider instead variational approximations of the gibbs posterior which are fast to compute we undertake a general study of the properties of such approximations our main finding is that such a variational approximation has often the same rate of convergence as the original pac bayesian procedure it approximates in addition we show that when the risk function is convex a variational approximation can be obtained in polynomial time using a convex solver we give finite sample oracle inequalities for the corresponding estimator we specialize our results to several learning tasks classification ranking matrix completion discuss how to implement a variational approximation in each case and illustrate the good properties of said approximation on real datasets

random linear programs with many variables and few constraints we extend and simplify smale s work on the expected number of pivots for a linear program with many variables and few constraints our analysis applies to new versions of the simplex algorithm and to new random distributions

improved asymptotic analysis of the average number of steps performed by the self dual simplex algorithm in this paper we analyze the average number of steps performed by the self dual simplex algorithm for linear programming under the probabilistic model of spherical symmetry the model was proposed by smale consider a problem ofn variables withm constraints smale established that for every number of constraintsm there is a constantc m such that the number of pivot steps of the self dual algorithm m n is less thanc m lnn m m 1 we improve upon this estimate by showing that m n is bounded by a function ofm only the symmetry of the function inm andn implies that m n is in fact bounded by a function of the smaller ofm andn

linear programming the simplex algorithm and simple polytopes in the first part of the paper we survey some far reaching applications of the basic facts of linear programming to the combinatorial theory of simple polytopes in the second part we discuss some recent developments concerning the simplex algorithm we describe subexponential randomized pivot rules and upper bounds on the diameter of graphs of polytopes 1997 the mathematical programming society inc published by elsevier science b v

the d step conjecture and its relatives the d step conjecture arose from an attempt to understand the computational complexity of edge following algorithms for linear programming such as the simplex algorithm it can be stated in terms of diameters of graphs of convex polytopes in terms of the existence of nonrevisiting paths in such graphs in terms of an exchange process for simplicial bases of a vector space and in terms of matrix pivot operations first formulated by w m hirsch in 1957 the conjecture remains unsettled though it has been proved in many special cases and counterexamples have been found for slightly stronger conjectures if the conjecture is false as we believe to be the case then finding a counterexample will be merely a small first step in the line of investigation related to the conjecture this report summarizes what is known about the d step conjecture and its relatives a considerable amount of new material is included but it does not seem to come close to settling the conjecture of special interest is the first example of a polytope that is not vertex decomposable showing that a certain natural approach to the conjecture will not work also significant are the quantitative relations among the lengths of paths associated with various forms of the conjecture

randomly generated polytopes for testing mathematical programming algorithms randomly generated polytopes are used frequently to test and compare algorithms for a variety of mathematical programming problems these polytopes are constructed by generating linear inequality constraints with coefficients drawn independently from a distribution such as the uniform or the normal

variable metric relaxation methods part ii the ellipsoid method the deepest or least shallow cut ellipsoid method is a polynomial time and space method which finds an ellipsoid representable by polynomial space integers such that the maximal ellipsoidal distance relaxation method using this fixed ellipsoid is polynomial this is equivalent to finding a linear transforming such that the maximal distance relaxation method of agmon motzkin and schoenberg in this transformed space is polynomial if perfect arithmetic is used then the sequence of ellipsoids generated by the method converges to a set of ellipsoids which share some of the properties of the classical hessian at an optimum point of a function and thus the ellipsoid method is quite analogous to a variable metric quasi newton method

a counterexample to the hirsch conjecture the hirsch conjecture 1957 stated that the graph of a d dimensional polytope with n facets cannot have combinatorial diameter greater than n d that is any two vertices of the polytope can be connected by a path of at most n d edges this paper presents the rst counterexample to the conjecture our polytope has dimension 43 and 86 facets it is obtained from a 5 dimensional polytope with 48 facets that violates a certain generalization of the d step conjecture of klee and walkup

the ellipsoid method generates dual variables we show that the ellipsoid algorithm applied to a system of linear inequalities can be implemented in such a way that at each iteration there is a short proof of the containment of the feasible region in the current ellipsoid moreover the data describing each ellipsoid also generate dual variables that provide bounds on the linear functions appearing in the inequalities

approximations for binary gaussian process classification we provide a comprehensive overview of many recent algorithms for approximate inference in gaussian process models for probabilistic binary classification the relationships between several approaches are elucidated theoretically and the properties of the different algorithms are corroborated by experimental results we examine both 1 the quality of the predictive distributions and 2 the suitability of the different marginal likelihood approximations for model selection selecting hyperparameters and compare to a gold standard based on mcmc interestingly some methods produce good predictive distributions although their marginal likelihood approximations are poor strong conclusions are drawn about the methods the expectation propagation algorithm is almost always the method of choice unless the computational budget is very tight we also extend existing methods in various ways and provide unifying code implementing all approaches

variational inference for latent variables and uncertain inputs in gaussian processes the gaussian process latent variable model gp lvm provides a flexible approach for non linear dimensionality reduction that has been widely applied however the current approach for training gp lvms is based on maximum likelihood where the latent projection variables are maximised over rather than integrated out in this paper we present a bayesian method for training gp lvms by introducing a non standard variational inference framework that allows to approximately integrate out the latent variables and subsequently train a gp lvm by maximising an analytic lower bound on the exact marginal likelihood we apply this method for learning a gp lvm from i i d observations and for learning non linear dynamical systems where the observations are temporally correlated we show that a benefit of the variational bayesian procedure is its robustness to overfitting and its ability to automatically select the dimensionality of the non linear latent space the resulting framework is generic flexible and easy to extend for other purposes such as gaussian process regression with uncertain or partially missing inputs we demonstrate our method on synthetic data and standard machine learning benchmarks as well as challenging real world datasets including high resolution video data

a sharp upper bound for the expected number of shadow vertices in lp polyhedra under orthogonal projection on two dimensional planes consider a set a1 am u v n satisfying a mild nondegeneracy condition when the polyhedron x x a1tx 1 amtx 1 n is orthogonally projected on the two dimensional plane linu v then some of its vertices may be mapped on vertices of the two dimensional image of x these vertices of x will be called shadow vertices under that projection their number will be denoted by s this figure s has an outstanding relevance for the complexity of a special simplex method the dimension by dimension method for solving lps of the type maximize vtx s t a1tx 1 amtx 1 where x v a1 am n and m n this method applies the shadow vertex algorithm in n 1 stages of increasing dimension by constructing a simplex path that visits only shadow vertices in each stage specific versions of the vectors u and v are used to define the projection plane and to control the application of the shadow vertex algorithm so s delivers an upper bound for the length of such a path in one stage we want to study the average case complexity of these algorithms and are therefore interested in em ns the expected value of s for that purpose we assume a distribution of the linear programming problems corresponding to the rotation symmetry model rsm the vectors a1 am u v are distributed on n 0 independently identically and symmetrically under rotations in this paper we improve the upper bounds for em ns from our polynomiality proofs of 1982 and 1987 there we had found an upper bound of m1 n 1 i n3 i const valid for all rsm distributions and all pairs m n based on a refined evaluation technique for space angles on a reorganization and on substantial modifications of the old proof we are now able to confirm a bound em ns m1 n 1 i n2 i const for all pairs m n with m n this result had been desired since 1982 because then a bound m1 n 1 i n2 i const had been derived exclusively for the situation where n is fixed and m tends to infinity all the time the question had been open whether the asymptotical behaviour would be better than the behaviour in moderate dimensions or whether the bound for the moderate case was crude in addition we know from an asymptotic analysis in 1987 that for a special rsm distribution namely the uniform distribution on the unit sphere there is a lower bound for em ns of type m1 n 1 i n2 i const the combination of this result and the new upper bound shows that the new bound is sharp

the many facets of linear programming we examine the history of linear programming from computational geometric and complexity points of view looking at simplex ellipsoid interior point and other methods

nonnegativity monotonicity or convexity preserving cubic and quintic hermite interpolation the hermite polynomials are simple effective interpolants of discrete data these interpolants can preserve local positivity monotonicity and convexity of the data if we restrict their derivatives to satisfy constraints at the data points this paper de scribes the conditions that must be satisfied for cubic and quintic hermite interpolants to preserve these properties when they exist in the discrete data we construct algorithms to ensure that these constraints are satisfied and give numerical examples to illustrate the effectiveness of the algorithms on locally smooth and rough data 1 introduction piecewise polynomial interpolants especially those based on hermite polynomials polynomials determined by their values and values of one or more derivatives at both ends of an interval have a number of desirable properties they are easy to compute once the derivative values are chosen if the derivative values are chosen locally e g by finite difference methods then the interpolant at a given point will depend only on the given data at nearby mesh points if the derivatives are computed by spline methods then the interpolant will have an extra degree of continuity at the mesh points in either case the interpolant is linear in the given function values and has excellent convergence properties as the mesh spacing decreases these methods however do not necessarily preserve the shape of the given data when the data arise from a physical experiment it may be vital that the interpolant preserve nonnegativity f x 0 nonpositivity f x 0 or f x 0 or concavity f x 0 in this and other cases geometric considerations such as preventing spurious behavior near rapid changes in the data may be more important than the asymptotic accuracy of the interpolation method one can construct a shape preserving interpolant by constraining the derivatives for the hermite polynomials to meet conditions which imply the desired properties 4 5 8 11 15 20 by adding new mesh points

polynomial expected behavior of a pivoting algorithm for linear complementarity and linear programming problems we show that a particular pivoting algorithm which we call the lexicographic lemke algorithm takes an expected number of steps that is bounded by a quadratic inn when applied to a random linear complementarity problem of dimensionn we present two probabilistic models both requiring some nondegeneracy and sign invariance properties the second distribution is concerned with linear complementarity problems that arise from linear programming in this case we give bounds that are quadratic in the smaller of the two dimensions of the linear programming problem and independent of the larger similar results have been obtained by adler and megiddo

a quadratically convergent method for linear programming abstract a new method to solve linear programming problems is introduced this method follows a path defined by a system of o d e and for nondegenerate problem is quadratically convergent

a barrier method for dynamic leontief type linear programs abstract in this paper we specialize gill et al s projected newton barrier method to solve a large scale linear program of dynamic i e multistage leontief type constraints we propose an efficient and stable method for solving the least squares subproblems the crucial part of the barrier method the key step is to exploit a special structure of the constraint matrix and reduce the matrix of the normal equation for the least squares problem to a banded matrix by comparing the average case operations count of this specialized barrier method with that of the sparse simplex method we show that this method performs at least o t faster than the simplex method for such stype of linear programs where t is the number of time periods i e stages

the expected number of nash equilibria of a normal form game fix finite pure strategy sets s1 s and let s s x x s n in our model of a random game the agents payoffs are statistically independent with each agent s payoff uniformly distributed on the unit sphere in r a for given nonempty t 1 subset of s 1 t n subset of s n we give a computationally implementable formula for the mean number of nash equilibria in which each agent i s mixed strategy has support t i the formula is the product of two expressions the first is the expected number of totally mixed equilibria for the truncated game obtained by eliminating pure strategies outside the sets ti the second may be construed as the probability that such an equilibrium remains an equilibrium when the strategies in the sets s i t i become available

random inequality constraint systems with few variables we show that a system of many linear inequality constraints will have a high proportion of redundant constraints with high probability implications for expected time of algorithms are indicated

the variational gaussian approximation revisited the variational approximation of posterior distributions by multivariate gaussians has been much less popular in the machine learning community compared to the corresponding approximation by factorizing distributions this is for a good reason the gaussian approximation is in general plagued by an number of variational parameters to be optimized being the number of random variables in this letter we discuss the relationship between the laplace and the variational approximation and we show that for models with gaussian priors and factorizing likelihoods the number of variational parameters is actually the approach is applied to gaussian process regression with nongaussian likelihoods

parametric linear programming and anti cycling pivoting rules the traditional perturbation or lexicographic methods for resolving degeneracy in linear programming impose decision rules that eliminate ties in the simplex ratio rule and therefore restrict the choice of exiting basic variables bland s combinatorial pivoting rule also restricts the choice of exiting variables using ideas from parametric linear programming we develop anticycling pivoting rules that do not limit the choice of exiting variables beyond the simplex ratio rule that is any variable that ties for the ratio rule can leave the basis a similar approach gives pivoting rules for the dual simplex method that do not restrict the choice of entering variables