a lanczos type method for multiple starting vectors given a square matrix and single right and left starting vectors the classical nonsymmetric lanczos process generates two sequences of bior thogonal basis vectors for the right and left krylov subspaces induced by the given matrix and vectors in this paper we propose a lanczos type algorithm that extends the classical lanczos process for single starting vectors to mul tiple starting vectors given a square matrix and two blocks of right and left starting vectors the algorithm generates two sequences of biorthogonal basis vectors for the right and left block krylov subspaces induced by the given data the algorithm can handle the most general case of right and left start ing blocks of arbitrary sizes while all previously proposed extensions of the lanczos process are restricted to right and left starting blocks of identical sizes other features of our algorithm include a built in deation procedure to detect and delete linearly dependent vectors in the block krylov sequences and the option to employ look ahead to remedy the potential breakdowns that may occur in nonsymmetric lanczos type methods

recognition problems for special classes of polynomials in 0 1 variables this paper investigates the complexity of various recognition problems for pseudo boolean functions i e real valued functions defined on the unit hypercubebn 0 1 n when such functions are represented as multilinear polynomials in their variables determining whether a pseudo boolean function a is monotonic or b is supermodular or c is threshold or d has a unique local maximum in each face ofbn or e has a unique local maximum inbn is shown to be np hard a polynomial time recognition algorithm is presented for unimodular functions previously introduced by hansen and simeone as a class of functions whose maximization overbn is reducible to a network minimum cut problem

polynomially computable bounds for the probability of the union of events we consider the problem of finding upper and lower bounds for the probability of the union of events when the probabilities of the single events and the probabilities of the intersections of up to m events are given it is known that the best possible bounds can be obtained by solving linear programming problems with a number of variables that is exponential in the number of events because of their size and structure these large linear programs are known to be very hard to solve in the literature simpler polynomially sized aggregations are considered and numerous closed form or polynomially computable bounds are derived from those we present here a new approach that introduces additional constraints to the dual linear programming problems in such a way that those become polynomially solvable by using different sets of additional constraints we introduce three new classes of polynomially computable upper and lower bounds we show that they dominate almost all efficiently computable bounds known in t

facial reduction in partially finite convex programming we consider the problem of minimizing an extended valued convex function on a locally convex space subject to a finite number of linear in equalities when the standard constraint qualification fails a reduction technique is needed to derive necessary optimality conditions facial reduction is usually applied in the range of the constraints in this paper it is applied in the domain space thus maintaining any structure and in particular lattice properties of the underlying domain applications include constrained approximation and best entropy estimation

convergence of best phi entropy estimates minimization problems involving phi entropy functionals a generalization of boltzmann shannon entropy are studied over a given set a and a sequence of sets a sub n and the properties of their optimal solutions x sub phi x sub n under certain conditions on the objective functional and the sets a and a sub n it is proven that as n increases to infinity the optimal solution x sub n converges in l sub 1 norm to the test phi entropy estimate x sub phi

criteria for quasi convexity and pseudo convexity relationships and comparisons a first order criterion for pseudo convexity and second order criteria for quasi convexity and pseudo convexity are given for twice differentiable functions on open convex sets the relationships between these second order criteria and other known criteria are also analysed finally the numbers of operations required to verify these criteria are calculated and compared

an improved bonferroni inequality and applications summary we present an improved bonferroni inequality which gives an upper bound for the probability of the union of an arbitrary sequence of events the bound is constructed in terms of the joint probability of pairs of events which are represented by edges on a graph examples of applications to periodicity location shift detection kolmogorovsmirnov tests and outlier detection are given

on pade type model order reduction of j hermitian linear dynamical systems a simple yet powerful approach to model order reduction of large scale linear dynamical systems is to employ projection onto block krylov subspaces the transfer functions of the resulting reduced order models of such projection methods can be characterized as pade type approximants of the transfer function of the original large scale system if the original system exhibits certain symmetries then the reduced order models are considerably more accurate than the theory for general systems predicts in this paper the framework of j hermitian linear dynamical systems is used to establish a general result about this higher accuracy in particular it is shown that in the case of j hermitian linear dynamical systems the reduced order transfer functions match twice as many taylor coefficients of the original transfer function as in the general case an application to the sprim algorithm for order reduction of general rcl electrical networks is discussed

high order entropy based closures for linear transport in slab geometry ii a computational study of the optimization problem we present a numerical algorithm to implement entropy based m n moment models in the context of a simple linear kinetic equation for particles moving through a material slab the closure for these models as is the case for all entropy based models is derived through the solution of a constrained convex optimization problem the algorithm has two components the first component is a discretization of the moment equations which preserves the set of realizable moments thereby ensuring that the optimization problem has a solution in exact arithmetic the discretization is a second order kinetic scheme which uses muscl type limiting in space and a strong stability preserving runge kutta time integrator the second component of the algorithm is a newton based solver for the dual optimization problem which uses an adaptive quadrature to evaluate integrals in the dual objective and its derivatives the accuracy of the numerical solution to the dual problem plays a key role in the time step restric

partially finite convex programming part ii explicit lattice models in part i of this work we derived a duality theorem for partially finite convex programs problems for which the standard slater condition fails almost invariably our result depended on a constraint qualification involving the notion ofquasi relative interior the derivation of the primal solution from a dual solution depended on the differentiability of the dual objective function the differentiability of various convex functions in lattices was considered at the end of part i in part ii we shall apply our results to a number of more concrete problems including variants of semi infinite linear programming l1 approximation constrained approximation and interpolation spectral estimation semi infinite transportation problems and the generalized market area problem of lowe and hurter 1976 as in part i we shall use lattice notation extensively but as we illustrated there in concrete examples lattice theoretic ideas can be avoided if preferred by direct calculation

partially finite programming in l_1 and the existence of maximum entropy estimates best entropy estimation is a technique that has been widely applied in many areas of science it consists of estimating an unknown density from some of its moments by maximizing some measure of the entropy of the estimate this problem can be modelled as a partially finite convex program with an integrable function as the variable a complete duality and existence theory is developed for this problem and for an associated extended problem which allows singular measure theoretic solutions this theory explains the appearance of singular components observed in the literature when the burg entropy is used it also provides a unified treatment of existence conditions when the burg boltzmann shannon or some other entropy is used as the objective some examples are discussed

submodular function minimization submodular functions often arise in various fields of operations research including discrete optimization game theory queueing theory and information theory in this survey paper we give overview on the fundamental properties of submodular functions and recent algorithmic devolopments of their minimization

convergence analysis of a lasserre hierarchy of upper bounds for polynomial minimization on the sphere we study the convergence rate of a hierarchy of upper bounds for polynomial minimization problems proposed by lasserre siam j optim 21 3 864 885 2011 for the special case when the feasible set is the unit hyper sphere the upper bound at level r n of the hierarchy is defined as the minimal expected value of the polynomial over all probability distributions on the sphere when the probability density function is a sum of squares polynomial of degree at most 2r with respect to the surface measure we show that the rate of convergence is o 1 r2 and we give a class of polynomials of any positive degree for which this rate is tight in addition we explore the implications for the related rate of convergence for the generalized problem of moments on the sphere

matrix theoretic criteria for the quasiconvexity of twice continuously differentiable functions abstract the author examines matrices and bordered matrices having exactly one or at most one negative eigenvalue these results are then used to state matrix theoretic criteria for the quasiconvexity of twice continuously differentiable functions for quadratic functions a new criterion for quasiconvexity is established and its equivalence to the already known criterion 11 23 is also shown

a maximum entropy method for solving frobenius perron operator equations let s 0 1 0 1 be a nonsingular transformation that preserves an absolutely continuous invariant measure with density f in this paper we develop a general algorithm for approximating f using the maximum entropy approach

a convex polynomial that is not sos convex a multivariate polynomial p x p x 1 x n is sos convex if its hessian h x can be factored as h x m t x m x with a possibly nonsquare polynomial matrix m x it is easy to see that sos convexity is a sufficient condition for convexity of p x moreover the problem of deciding sos convexity of a polynomial can be cast as the feasibility of a semidefinite program which can be solved efficiently motivated by this computational tractability it is natural to study whether sos convexity is also a necessary condition for convexity of polynomials in this paper we give a negative answer to this question by presenting an explicit example of a trivariate homogeneous polynomial of degree eight that is convex but not sos convex

dual methods in entropy maximization application to some problems in crystallography this paper is devoted to some infinite dimensional optimization problems with finitely many constraints these deal with entropy maximization and this paper is particularly concerned with those or

duality and the computation of approximate invariant densities for nonsingular transformations we investigate a class of optimization problems which arise in the approximation of invariant densities for a nonsingular measurable transformation t acting on a finite measure space the problems under consideration have convex integral type objectives and finite moment constraints and include for example the maximum entropy and quadratic programming approaches previously studied in the literature this article is a natural sequel to those investigations and to the paper c bose and r murray discrete contin dyn syst 14 2006 pp 597 615 where a general class of convergent moment approximations were defined such that the limiting optimal solution is an invariant density for t this article mainly concerns the solution of a single finite moment problem arising from this general approximation scheme both theoretical aspects and computational issues are treated although the problem fits easily into the standard theory of duality in convex optimization its dynamical origins lead to technical obstructions in the derivation of optimality conditions in particular the dual functional for our problem is neither strictly convex nor coercive relating in part to the fact that the moment generating functions for the approximation scheme need not be pseudo haar the method of the paper circumvents these obstructions and yields an unexpected benefit each finite moment approximation leads to rigorous bounds on the support of all invariant densities for t

semidefinite characterization of sum of squares cones in algebras we extend nesterov s semidefinite programming characterization of squared functional systems and faybusovich s abstraction to bilinear symmetric maps to cones of sum of squares elements in general abstract algebras using algebraic techniques such as isomorphism linear isomorphism tensor products sums and direct sums we show that many concrete cones are in fact sum of squares cones with respect to some algebra and thus are representable by the cone of positive semidefinite matrices we also consider nonnegativity with respect to a proper cone mathcal k and show that in some cases cones of mathcal k nonnegative functions are either sum of squares or at least semidefinite representable for example we show that some well known chebyshev systems when extended to euclidean jordan algebras induce cones that are semidefinite representable finally we will discuss some concrete examples and applications including minimum ellipsoid enclosing given space curves minimization of eigenvalues of po

on the convergence of moment problems we study the problem of estimating a nonnegative density given a finite number of moments such problems arise in numerous practical applications as the number of moments increases the estimates will always converge weak as measures but need not converge weakly in l this is related to the existence of functions on a compact metric space which are not essentially riemann integrable in some suitable sense we characterize the type of weak convergence we can expect in terms of riemann integrability and in some cases give error bounds when the estimates are chosen to minimize an objective function with weakly compact level sets such as the bolzmann shannon entropy they will converge weakly in l when an lp norm 1 p is used as the objective the estimates actually converge in norm these results provide theoretical support to the growing popularity of such methods in practice

hardness and approximation results for l p ball constrained homogeneous polynomial optimization problems in this paper we establish hardness and approximation results for various lp ball constrained homogeneous polynomial optimization problems where p 2 specifically we prove that for any given d 3 and p 2 both the problem of optimizing a degree d homogeneous polynomial over the lp ball and the problem of optimizing a degree d multilinear form regardless of its super symmetry over lp balls are np hard on the other hand we show that these problems can be approximated to within a factor of log n d 2 p nd 2 1 in deterministic polynomial time where n is the number of variables we further show that with the help of randomization the approximation guarantee can be improved to log n n d 2 1 which is independent of p and is currently the best for the aforementioned problems our results unify and generalize those in the literature which focus either on the quadratic case or the case where p 2 we believe that the wide array of tools used in this paper will have further

semidefinite representation of convex sets let s x in mathbb r n g 1 x geq 0 ldots g m x geq 0 be a semialgebraic set defined by multivariate polynomials g i x assume s is convex compact and has nonempty interior let s i x in mathbb r n g i x geq 0 and s resp s i be the boundary of s resp s i this paper as does the subject of semidefinite programming sdp concerns linear matrix inequalities lmis the set s is said to have an lmi representation if it equals the set of solutions to some lmi and it is known that some convex s may not be lmi representable helton and vinnikov in commun pure appl math 60 5 654 674 2007 a question arising from nesterov and nemirovski siam studies in applied mathematics society for industrial and applied mathematics siam philadelphia 1994 see helton and vinnikov in commun pure appl math 60 5 654 674 2007 and nemirovski in plenary lecture international congress of mathematicians icm madrid spain 2006 is given a subset s of mathbb r n does there exist an lmi representable set in some higher dimensional space mathbb r n n whose projection down onto mathbb r n equals s such s is called semidefinite representable or sdp representable this paper addresses the sdp representability problem the following are the main contributions of this paper i assume g i x are all concave on s if the positive definite lagrange hessian condition holds i e the hessian of the lagrange function for optimization problem of minimizing any nonzero linear function l t x on s is positive definite at the minimizer then s is sdp representable ii if each g i x is either sos concave 2 g i x w x t w x for some possibly nonsquare matrix polynomial w x or strictly quasi concave on s then s is sdp representable iii if each s i is either sos convex or poscurv convex s i is compact convex whose boundary has positive curvature and is nonsingular i e g i x 0 on s i s then s is sdp representable this also holds for s i for which s i s extends smoothly to the boundary of a poscurv convex set containing s iv we give the complexity of schmudgen and putinar s matrix positivstellensatz which are critical to the proofs of i iii

fixed smoothing asymptotics in the generalized empirical likelihood estimation framework this paper concerns the fixed smoothing asymptotics for two commonly used estimators in the generalized empirical likelihood estimation framework for time series data namely the continuous updating estimator and the maximum blockwise empirical likelihood estimator for continuously updating generalized method of moments gmm estimator we show that the results for the two step gmm estimator in sun 2014a continue to hold under suitable assumptions for continuous updating estimator obtained through solving a saddle point problem newey and smith 2004 and the maximum blockwise empirical likelihood estimator kitamura 1997 we show that their fixed smoothing asymptotic distributions up to an unknown linear transformation are mixed normal based on these results we derive the asymptotic distributions of the specification tests including the over identification testing and testing on parameters under the fixed smoothing asymptotics where the corresponding limiting distributions are nonstandard yet pivotal simulation studies show that i the fixed smoothing asymptotics provides better approximation to the sampling distributions of the continuous updating estimator and the maximum blockwise empirical likelihood estimator as compared to the standard normal approximation the testing procedures based on the fixed smoothing critical values are more accurate in size than the conventional chi square based tests ii the continuously updating gmm estimator is asymptotically more efficient and the corresponding specification tests are generally more powerful than the other two competitors finite sample results from an empirical data analysis are also reported

a fast newton algorithm for entropy maximization in phase determination a long standing issue in the bayesian statistical approach to the phase problem in x ray crystallography is to solve an entropy maximization subproblem efficiently in every iteration of phase estimation the entropy maximization problem is a semi infinite convex program and can be solved in a finite dual space by using a standard newton method however the newton method is too expensive for this application since it requires o n3 floating point operations per iteration where n corresponds to the number of phases to be estimated other less expensive methods have been used but they cannot guarantee fast convergence in this paper we present a fast newton algorithm for the entropy maximization problem which uses the sherman morrison woodbury formula and the fast fourier transform to compute the newton step and requires only o n log n floating point operations per iteration yet has the same convergence rate as the standard newton we describe the algorithm and discuss related computational issues numerical results on simple test cases will also be presented to demonstrate the behavior of the algorithm

economic implications of nonlinear pricing kernels based on a family of discrepancy functions we derive nonparametric stochastic discount factor bounds that naturally generalize variance entropy and higher moment bounds these bounds are especially useful to identify how parameters affect pricing kernel dispersion in asset pricing models in particular they allow us to distinguish between models where dispersion comes mainly from skewness from models where kurtosis is the primary source of dispersion we analyze the admissibility of disaster disappointment aversion and long run risk models with respect to these bounds this paper was accepted by jerome detemple finance

on the geometry of maximum entropy problems we show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite and infinite dimensional maximum entropy problems concerning probability distributions spectral densities and covariance matrices these include burg s spectral estimation method and dempster s covariance completion as well as various recent generalizations of the above we then apply this orthogonality principle to the new problem of completing a block circulant covariance matrix when an a priori estimate is available

superresolution in the markov moment problem abstract suppose that bar x is a characteristic function and a 1 a 2 a n are weight functions on a finite measure space recent work of gamboa and gassiat observed conditions guaranteeing that x bar x 1 is small whenever 0 x 1 a e and the moment errors int x bar x a i are small i 1 2 n using concise and elementary techniques we obtain similar results under very mild assumptions we also provide precise error bounds

a new look at entropy for solving linear inverse problems entropy based methods are widely used for solving inverse problems particularly when the solution is known to be positive here we address linear ill posed and noisy inverse problems of the form z ax n with a general convex constraint x spl isin x where x is a convex set although projective methods are well adapted to this context we study alternative methods which rely highly on some information based criteria our goal is to clarify the role played by entropy in this field and to present a new point of view on entropy using general tools and results coming from convex analysis we present then a new and broad scheme for entropic based inversion of linear noisy inverse problems this scheme was introduced by navaza in 1985 in connection with a physical modeling for crystallographic applications and further studied by dacunha castelle and gamboa 1990 important features of this paper are i a unified presentation of many well known reconstruction criteria ii proposal of new criteria for reconstruction under various prior knowledge and with various noise statistics iii a description of practical inversion of data using the aforementioned criteria and iv a presentation of some reconstruction results

application of the method of maximum entropy in the mean to classification problems abstract in this note we propose an application of the method of maximum entropy in the mean to solve a class of inverse problems comprising classification problems and feasibility problems appearing in optimization such problems may be thought of as linear inverse problems with convex constraints imposed on the solution as well as on the data the method of maximum entropy in the mean proves to be a very useful tool to deal with this type of problems

convexity in semialgebraic geometry and polynomial optimization we review several and provide new results on the theory of moments sums of squares and basic semialgebraic sets when convexity is present in particular we show that under convexity the hierarchy of semidefinite relaxations for polynomial optimization simplifies and has finite convergence a highly desirable feature as convex problems are in principle easier to solve in addition if a basic semialgebraic set mathbf k is convex but its defining polynomials are not we provide two algebraic certificates of convexity which can be checked numerically the second is simpler and holds if a sufficient and almost necessary condition is satisfied it also provides a new condition for mathbf k to have semidefinite representation for this we use and extend some of the recent results from the author and helton and nie math program to appear finally we show that when restricting to a certain class of convex polynomials the celebrated jensen s inequality in convex analysis can be extended to linear functionals that are not necessarily probability measures

an implicitly restarted krylov subspace method for real symmetric skew symmetric eigenproblems abstract a new implicitly restarted krylov subspace method for real symmetric skew symmetric generalized eigenvalue problems is presented the new method improves and generalizes the shira method of mehrmann and watkins 2001 37 to the case where the skew symmetric matrix is singular it computes a few eigenvalues and eigenvectors of the matrix pencil close to a given target point several applications from control theory are presented and the properties of the new method are illustrated by benchmark examples

krylov subspace methods for reduced order modeling in circuit simulation the simulation of electronic circuits involves the numerical solution of very large scale sparse in general nonlinear systems of differential algebraic equations often the size of these systems can be reduced considerably by replacing the equations corresponding to linear subcircuits by approximate models of much smaller state space dimension in this paper we describe the use of krylov subspace methods for generating such reduced order models of linear subcircuits particular emphasis is on reduced order modeling techniques that preserve the passivity of linear rlc subcircuits

efficient information theoretic inference for conditional moment restrictions abstract the generalised method of moments estimator may be substantially biased in finite samples especially so when there are large numbers of unconditional moment conditions this paper develops a class of first order equivalent semi parametric efficient estimators and tests for conditional moment restrictions models based on a local or kernel weighted version of the cressie read power divergence family of discrepancies this approach is similar in spirit to the empirical likelihood methods of kitamura et al 2004 empirical likelihood based inference in conditional moment restrictions models econometrica 72 1667 1714 and tripathi and kitamura 2003 testing conditional moment restrictions annals of statistics 31 2059 2095 these efficient local methods avoid the necessity of explicit estimation of the conditional jacobian and variance matrices of the conditional moment restrictions and provide empirical conditional probabilities for the observations

adaptive change of basis in entropy based moment closures for linear kinetic equations entropy based m n moment closures for kinetic equations are defined by a constrained optimization problem that must be solved at every point in a space time mesh making it important to solve these optimization problems accurately and efficiently we present a complete and practical numerical algorithm for solving the dual problem in one dimensional slab geometries the closure is only well defined on the set of moments that are realizable from a positive underlying distribution and as the boundary of the realizable set is approached the dual problem becomes increasingly difficult to solve due to ill conditioning of the hessian matrix to improve the condition number of the hessian we advocate the use of a change of polynomial basis defined using a cholesky factorization of the hessian that permits solution of problems nearer to the boundary of the realizable set we also advocate a fixed quadrature scheme rather than adaptive quadrature since the latter introduces unnecessary expense and changes the computationally realizable set as the quadrature changes for very ill conditioned problems we use regularization to make the optimization algorithm robust we design a manufactured solution and demonstrate that the adaptive basis optimization algorithm reduces the need for regularization this is important since we also show that regularization slows and even stalls convergence of the numerical simulation when refining the space time mesh we also simulate two well known benchmark problems there we find that our adaptive basis fixed quadrature algorithm uses less regularization than alternatives although differences in the resulting numerical simulations are more sensitive to the regularization strategy than to the choice of basis

analysis and design of functionally weighted single input rule modules connected fuzzy inference systems the single input rule modules sirms connected fuzzy inference method can efficiently solve the fuzzy rule explosion phenomenon which usually occurs in the multivariable modeling and or control applications however the performance of the sirms connected fuzzy inference system sirm fis is limited due to its simple input output mapping in this paper to further enhance the performance of sirm fis a functionally weighted sirm fis fwsirm fis which adopts multivariable functional weights to measure the important degrees of the sirms is presented then in order to show the fundamental differences of the sirms methods properties of the traditional sirm fis the type 2 sirm fis t2sirm fis the functional sirm fis fsirm fis the sirms model with single variable functional weights sirm fw and fwsirm fis are explored these properties demonstrate that the proposed fwsirm fis has more general and complex input output mapping than the existing sirms methods such properties theoretically guarantee that better performance can be achieved by fwsirm fis furthermore based on the least squares method a novel data driven optimization method is presented for the parameter learning of fwsirm fis it can also be used to optimize the parameters of sirm fis t2sirm fis fsirm fis and sirm fw due to the properties of the least squares method the proposed parameter learning algorithm can overcome the drawbacks of the gradients based parameter learning methods and obtain both smallest training errors and smallest parameters finally to show the effectiveness and superiority of fwsirm fis and the proposed optimization method six examples and detailed comparisons are given simulation results show that fwsirm fis can obtain better performance than the other sirms methods and compared with some well known methods fwsirm fis can achieve similar or better performance but has much less parameters and faster training speed

which is the better entropy expression for speech processing s log s or log s in maximum entropy spectral analysis mesa one maximizes the integral of logs f where s f is a power spectrum the resulting spectral estimate which is equivalent to that obtained by linear prediction and other methods is popular in speech processing applications an alternative expression s f logs f is used in optical processing and elsewhere this paper considers whether the alternative expression leads to spectral estimates useful in speech processing we investigate the question both theoretically and empirically the theoretical investigation is based on generalizations of file two estimates the generalizations take into account prior estimates of the unknown power spectrum it is shown that both estimates result from applying a generalized version of the principle of maximum entropy but they differ concerning the quantities that are treated as random variables the empirical investigation is based on speech synthesized using the different spectral estimates although both estimates lead to intelligible speech speech based on the mesa estimate is qualitatively superior

kullback leibler upper confidence bounds for optimal sequential allocation we consider optimal sequential allocation in the context of the so called stochastic multi armed bandit model we describe a generic index policy in the sense of gittins 1979 based on upper confidence bounds of the arm payoffs computed using the kullback leibler divergence we consider two classes of distributions for which instances of this general idea are analyzed the kl ucb algorithm is designed for one parameter exponential families and the empirical kl ucb algorithm for bounded and finitely supported distributions our main contribution is a unified finite time analysis of the regret of these algorithms that asymptotically matches the lower bounds of lai and robbins 1985 and burnetas and katehakis 1996 respectively we also investigate the behavior of these algorithms when used with general bounded rewards showing in particular that they provide significant improvements over the state of the art

on hamiltonian rational transfer functions abstract a sequel to a previous paper fuhrmann 1983 this paper contains a detailed study of rational transfer functions having hamiltonian symmetry a detailed realization procedure is described that reflects this symmetry as a corollary the williamson canonical form for hamiltonian maps in symplectic spaces is derived

closed form two sided bounds for probabilities that at least r and exactly r out of n events occur in two previous papers prekopa prekopa a 1986a boole bonferroni inequalities and linear programming oper res 36 145 162 prekopa a 1986b sharp bounds on probabilities using linear programming to appear in oper res gave algorithms to approximate probabilities that at least r and exactly r out of n events occur 1 r n primal and dual linear programming problems were formulated and solved by dual type algorithms the purpose of the present paper is to give closed forms for the basis inverse and the corresponding dual vector in case of an arbitrary basis furthermore to give closed forms for the lower and upper bounds approximating the probability in question in case of a dual feasible basis in the case when the probability that at least one out of n events occurs is approximated it is shown that the absolute values of the components of any dual vector form a monotonically decreasing sequence the paper improves the method of inclusion exclusion proves new probability inequalities and

dc decomposition of nonconvex polynomials with algebraic techniques we consider the problem of decomposing a multivariate polynomial as the difference of two convex polynomials we introduce algebraic techniques which reduce this task to linear second order cone and semidefinite programming this allows us to optimize over subsets of valid difference of convex decompositions dcds and find ones that speed up the convex concave procedure we prove however that optimizing over the entire set of dcds is np hard

some np complete problems in quadratic and nonlinear programming in continuous variable smooth nonconvex nonlinear programming we analyze the complexity of checking whether a a given feasible solution is not a local minimum and b the objective function is not bounded below on the set of feasible solutions we construct a special class of indefinite quadratic programs with simple constraints and integer data and show that checking a or b on this class is np complete as a corollary we show that checking whether a given integer square matrix is not copositive is np complete

optimality conditions and optimization methods for quartic polynomial optimization abstract in this paper multivariate quartic polynomial optimization program qpop is considered quartic optimization problems arise in various practical applications and are proved to be np hard we discuss necessary global optimality conditions for quartic problem qpop and then we present a new strongly or e strongly local optimization method according to necessary global optimality conditions which may escape and improve some kkt points finally we design a global optimization method for problem qpop by combining the new strongly or e strongly local optimization method and an auxiliary function numerical examples show that our algorithms are efficient and stable

classical and strong convexity of sublevel sets and application to attainable sets of nonlinear systems necessary and sufficient conditions for convexity and strong convexity respectively of connected sublevel sets that are defined by finitely many real valued c 1 1 maps are presented a novel characterization of strongly convex sets in terms of the so called local quadratic support is proved the results concerning strong convexity are used to derive sufficient conditions for attainable sets of continuous time nonlinear systems to be strongly convex an application of these conditions is a novel method to overapproximate attainable sets when strong convexity is present

np hardness of deciding convexity of quartic polynomials and related problems we show that unless p np there exists no polynomial time or even pseudo polynomial time algorithm that can decide whether a multivariate polynomial of degree four or higher even degree is globally convex this solves a problem that has been open since 1992 when n z shor asked for the complexity of deciding convexity for quartic polynomials we also prove that deciding strict convexity strong convexity quasiconvexity and pseudoconvexity of polynomials of even degree four or higher is strongly np hard by contrast we show that quasiconvexity and pseudoconvexity of odd degree polynomials can be decided in polynomial time

on convergence of krylov subspace approximations of time invariant self adjoint dynamical systems abstract we extend the rational krylov subspace algorithm from the computation of the action of the matrix exponential to the solution of stable dynamical systems a d dt u t b t u t 0 0 a d dt i 0 m a i d dt si i where m n a i a i r n n s 0 and u t b t r n b t 0 0 not assuming that evolution of b t is described by a low dimensional subspace of r n we show that the reduced equation is stable and derive an a priori error bound via rational approximation of the exponential on the boundary of the nonlinear numerical range of a we also describe a simple and easily computable external bound of this numerical range the obtained results are applied to the infinite order problem arising in the solution of the dispersive maxwell s system

on the complexity of detecting convexity over a box it has recently been shown that the problem of testing global convexity of polynomials of degree four is strongly np hard answering an open question of n z shor this result is minimal in the degree of the polynomial when global convexity is of concern in a number of applications however one is interested in testing convexity only over a compact region most commonly a box i e a hyper rectangle in this paper we show that this problem is also strongly np hard in fact for polynomials of degree as low as three this result is minimal in the degree of the polynomial and in some sense justifies why convexity detection in nonlinear optimization solvers is limited to quadratic functions or functions with special structure as a byproduct our proof shows that the problem of testing whether all matrices in an interval family are positive semidefinite is strongly np hard this problem which was previously shown to be weakly np hard by nemirovski is of independent interest in the theory of robust control

on the complexity of testing attainment of the optimal value in nonlinear optimization we prove that unless hbox p hbox np there exists no polynomial time or even pseudo polynomial time algorithm that can test whether the optimal value of a nonlinear optimization problem where the objective and constraints are given by low degree polynomials is attained if the degrees of these polynomials are fixed our results along with previously known frank wolfe type theorems imply that exactly one of two cases can occur either the optimal value is attained on every instance or it is strongly np hard to distinguish attainment from non attainment we also show that testing for some well known sufficient conditions for attainment of the optimal value such as coercivity of the objective function and closedness and boundedness of the feasible set is strongly np hard as a byproduct our proofs imply that testing the archimedean property of a quadratic module is strongly np hard a property that is of independent interest to the convergence of the lasserre hierarchy finally we give semidefinite programming sdp based sufficient conditions for attainment of the optimal value in particular a new characterization of coercive polynomials that lends itself to an sdp hierarchy

establishing convexity of polynomial lyapunov functions and their sublevel sets there has been renewed interest in the use of polynomial and homogeneous polynomial lyapunov functions for addressing stability and performance issues in systems and control theory however no condition is yet available to establish the convexity of these functions and the convexity of their sublevel sets properties that can be useful in many applications in this paper new conditions for establishing these convexity properties are proposed which can be handled through convex linear matrix inequalities optimizations by means of sum of squares relaxations

on the supermodular knapsack problem in this paper we introduce binary knapsack problems where the objective function is nonlinear and investigate their lagrangean and continuous relaxations some of our results generalize previously known theorems concerning linear and quadratic knapsack problems we investigate in particular the case in which the objective function is supermodular under this hypothesis although the problem remains np hard we show that its lagrangean dual and its continuous relaxation can be solved in polynomial time we also comment on the complexity of recognizing supermodular functions the particular case in which the knapsack constraint is of the cardinality type is also addressed and some properties of its optimal value as a function of the right hand side are derived

maximum entropy solution to ill posed inverse problems with approximately known operator we consider the linear inverse problem of reconstructing an unknown finite measure from a noisy observation of a generalized moment of defined as the integral of a continuous and bounded operator with respect to motivated by various applications we focus on the case where the operator is unknown instead only an approximation m to it is available an approximate maximum entropy solution to the inverse problem is introduced in the form of a minimizer of a convex functional subject to a sequence of convex constraints under several assumptions on the convex functional the convergence of the approximate solution is established

lagrange multipliers and optimality lagrange multipliers used to be viewed as auxiliary variables introduced in a problem of constrained minimization in order to write first order optimality conditions formally as a system of equations modern applications with their emphasis on numerical methods and more complicated side conditions than equations have demanded deeper understanding of the concept and how it fits into a larger theoretical picture a major line of research has been the nonsmooth geometry of one sided tangent and normal vectors to the set of points satisfying the given constraints another has been the game theoretic role of multiplier vectors as solutions to a dual problem interpretations as generalized derivatives of the optimal value with respect to problem parameters have also been explored lagrange multipliers are now being seen as arising from a general rule for the subdifferentiation of a nonsmooth objective function which allows black and white constraints to be replaced by penalty expressions this paper traces su

assessing misspecified asset pricing models with empirical likelihood estimators abstract hansen and jagannathan 1997 compare misspecified asset pricing models based on least square projections on a family of admissible stochastic discount factors we extend their fundamental contribution by considering minimum discrepancy projections where misspecification is measured by a family of convex functions that take into account higher moments of asset returns the minimum discrepancy problems are solved on dual spaces producing a family of estimators that captures the least square problem as a particular case we derive the asymptotic distributions of the estimators for the cressie read family of discrepancies and illustrate their use with an assessment of the consumption asset pricing model

maximum entropy reconstruction using derivative information part 2 computational results summary maximum entropy density estimation a technique for reconstructing an unknown density function on the basis of certain measurements has applications in various areas of applied physical sciences and engineering here we present numerical results for the maximum entropy inversion program based on a new class of information measures which are designed to control derivative values of the unknown densities

biquadratic optimization over unit spheres and semidefinite programming relaxations this paper studies the so called biquadratic optimization over unit spheres min x in mathbb r n y in mathbb r m sum 1 leq i k leq n 1 leq j l leq m b ijkl x i y j x k y l subject to x 1 y 1 we show that this problem is np hard and there is no polynomial time algorithm returning a positive relative approximation bound then we present various approximation methods based on semidefinite programming sdp relaxations our theoretical results are as follows for general biquadratic forms we develop a frac 1 2 max m n 2 approximation algorithm under a slightly weaker approximation notion for biquadratic forms that are square free we give a relative approximation bound frac 1 nm when min n m is a constant we present two polynomial time approximation schemes ptass which are based on sum of squares sos relaxation hierarchy and grid sampling of the standard simplex for practical computational purposes we propose the first order sos relaxation a convex quadratic sdp relaxation and a simple minimum eigenvalue method and show their error bounds some illustrative numerical examples are also included

polyhedral approximation in mixed integer convex optimization generalizing both mixed integer linear optimization and convex optimization mixed integer convex optimization possesses broad modeling power but has seen relatively few advances in general purpose solvers in recent years in this paper we intend to provide a broadly accessible introduction to our recent work in developing algorithms and software for this problem class our approach is based on constructing polyhedral outer approximations of the convex constraints resulting in a global solution by solving a finite number of mixed integer linear and continuous convex subproblems the key advance we present is to strengthen the polyhedral approximations by constructing them in a higher dimensional space in order to automate this extended formulation we rely on the algebraic modeling technique of disciplined convex programming dcp and for generality and ease of implementation we use conic representations of the convex constraints although our framework requires a manual translation of existing models into dcp form after performing this transformation on the minlplib2 benchmark library we were able to solve a number of unsolved instances and on many other instances achieve superior performance compared with state of the art solvers like bonmin scip and artelys knitro

distinguishing separable and entangled states we show how to design families of operational criteria that distinguish entangled from separable quantum states the simplest of these tests corresponds to the well known peres horodecki positive partial transpose ppt criterion and the more complicated tests are strictly stronger the new criteria are tractable due to powerful computational and theoretical methods for the class of convex optimization problems known as semidefinite programs we successfully applied the results to many low dimensional states from the literature where the ppt test fails as a by product of the criteria we provide an explicit construction of the corresponding entanglement witnesses

the maximum entropy method on the mean applications to linear programming and superresolution in this paper we give two different results we propose new methods to solve classical optimization problems in linear programming we also obtain precise quantitative results for the superresolution phenomenon as observed earlier by practical searchers on specific algorithms the common background of our work is the generalized moment problem which is known to be connected with linear programming and superresolution we describe the maximum entropy method on the mean that provides solution to the problem and leads to computational criteria to decide the existence of solutions or not

on the lasserre hierarchy of semidefinite programming relaxations of convex polynomial optimization problems the lasserre hierarchy of semidefinite programming approximations to convex polynomial optimization problems is known to converge finitely under some assumptions j b lasserre convexity in semialgebraic geometry and polynomial optimization siam j optim 19 1995 2014 2009 we give a new proof of the finite convergence property that does not require the assumption that the hessian of the objective be positive definite on the entire feasible set but only at the optimal solution in addition we show that the number of steps needed for convergence depends on more than the input size of the problem in particular the size of the semidefinite program that gives the exact reformulation of the convex polynomial optimization problem may be exponential in the input size

discretizing distributions with exact moments error estimate and convergence analysis the maximum entropy principle is a powerful tool for solving underdetermined inverse problems this paper considers the problem of discretizing a continuous distribution which arises in various applied fields we obtain the approximating distribution by minimizing the kullback leibler information relative entropy of the unknown discrete distribution relative to an initial discretization based on a quadrature formula subject to some moment constraints we study the theoretical error bound and the convergence of this approximation method as the number of discrete points increases we prove that i the theoretical error bound of the approximate expectation of any bounded continuous function has at most the same order as the quadrature formula we start with and ii the approximate discrete distribution weakly converges to the given continuous distribution moreover we present some numerical examples that show the advantage of the method and apply to numerically solving an optimal portfolio problem

cut polytopes boolean quadratic polytopes and nonnegative quadratic pseudo boolean functions in this paper we present a class of valid inequalities as well as a class of facets for the cut polytope of the complete graph it is shown that many of the known classes of valid inequalities e g triangle hypermetric and cycle inequalities belong to this class by specializing some of the parameters large classes of facets can be defined it is shown that each of these classes contains at least n 3n 3 facets where n 10 denotes the number of vertices and that the corresponding separation problems are polynomially solvable these results are based on a one to one correspondence established between the set of valid inequalities facets for the cut polytope of kn 1 and the set of nonnegative extremal quadratic pseudo boolean functions in n variables

polynomial matrix inequality and semidefinite representation consider a convex set s x in cal d g x succeq 0 where g x is a symmetric matrix whose every entry is a polynomial or rational function d rn is a domain on which g x is defined and g x succeq 0 means g x is positive semidefinite the set s is called semidefinite representable if it equals the projection of a higher dimensional set that is defined by a linear matrix inequality lmi this paper studies sufficient conditions guaranteeing semidefinite representability of s we prove that s is semidefinite representable in the following cases i d rn g x is a matrix polynomial and matrix sos concave ii d is compact convex g x is a matrix polynomial and strictly matrix concave on d iii g x is a matrix rational function and q module matrix concave on d explicit constructions of semidefinite representations are given some examples are illustrated

boole bonferroni inequalities and linear programming we present a method to obtain sharp lower and upper bounds for the probability that at least one out of a number of events in an arbitrary probability space will occur the input data are some of the binomial moments of the occurrences such as the sum of the probabilities of the individual events or the sum of the joint probabilities of all pairs of events we develop a special very simple linear programming algorithm to obtain these bounds the method allows us to compute good bounds in an optimal way utilizing only the first few terms in the inclusion exclusion formula possible applications include obtaining bounds for the reliability of a stochastic system solving algorithmically some stochastic programming problems and approximating multivariate probabilities in statistics in a numerical example we approximate the probability that a gaussian process runs below a given level in a number of consecutive epochs

duality relationships for entropy like minimization problems this paper considers the minimization of a convex integral functional over the positive cone of an l p space subject to a finite number of linear equality constraints such problems arise in spectral estimation where the bjective function is often entropy like and in constrained approximation the lagrangian dual problem is finite dimensional and unconstrained under a quasi interior constraint qualification the primal and dual values are equal with dual attainment examples show the primal value may not be attained conditions are given that ensure that the primal optimal solution can be calculated directly from a dual optimum these conditions are satisfied in many examples

entropic quadrature for moment approximations of the boltzmann bgk equation abstract using the method of moments to approximate the solution of kinetic equations has become a standard technique especially in the context of solving the boltzmann equation to model rarefied gases one attractive example is the maximum entropy closure which however is computationally barely affordable in this paper we will combine the maximum entropy approach with the quadrature method of moments qmom two methods which have in some sense diametric properties thereby introducing the entropic quadrature eq closure eq calculates a sparse quadrature based reconstruction of the unknown velocity distribution like the qmom where the physical meaningful maximum entropy principle is employed as a criterion for selecting a quadrature among different quadratures fitting the given moments we will discuss the construction of the closure and give several numerical examples of its performance in one and two dimensions

convergence of best entropy estimates given a finite number of moments of an unknown density on a finite measure space the best entropy estimate that nonnegative density x with the given moments which minimizes the boltzmann shannon entropy i x x log x is considered a direct proof is given that i has the kadec property in l if yn converges weakly to 37 and i yn converges to i 37 then yn converges to 37 in norm as a corollary it is obtained that as the number of given moments increases the best entropy estimates converge in li norm to the best entropy estimate of the limiting problem which is simply in the determined case furthermore for classical moment problems on intervals with strictly positive and sufficiently smooth error bounds and uniform convergence are actually obtained

minimization of entropy functionals entropy i e convex integral functionals and extensions of these functionals are minimized on convex sets this paper is aimed at reducing as much as possible the assumptions on the constraint set primal attainment dual equalities dual attainment and characterizations of the minimizers are obtained with weak constraint qualifications these results improve several aspects of the literature on the subject

solution of the general moment problem via a one parameter imbedding this paper presents a computational theory for the general scalar moment problem the formalism is sufficiently general to encompass problems in sensor arrays with arbitrary geometry and dynamics and in nonuniform multidimensional sampling given a finite set of moments the theory provides a test for the existence of a positive measure which is consistent with such data at the same time the theory also provides a characterization of all such consistent positive measures it should be noted that classical results e g in the theory of the trigonometric moment problem hamburger stieljes nevanlinna pick interpolation etc are not applicable to the general setting sought herein where there is no natural shift operator in the space spanned by the integration kernels the centerpiece of the theory is a differential equation which depends on the given finite set of moments and on an arbitrary positive function spl psi which plays the role of a free parameter the differential equation has an exponentially attractive point of equilibrium if and only if there exists a consistent positive measure for each spl psi the fixed point determines a corresponding measure suitable choice of spl psi allows recovering any measure which is consistent with the data the fixed point of the differential equation corresponds to an extremum of an entropy like functional and the differential equation is constructed via an appropriate homotopy that follows changes in the lagrange multipliers from a convenient starting value to a value for the multipliers that corresponds to the given moments

maximum entropy reconstruction using derivative information part 1 fisher information and convex duality maximum entropy spectral density estimation is a technique for reconstructing an unknown density function from some known measurements by maximizing a given measure of entropy of the estimate here we present a variety of new entropy measures which attempt to control derivative values of the densities our models apply among others to the inference problem based on the averaged fisher information measure the duality theory we develop resembles models used in convex optimal control problems we present a variety of examples including relaxed moment matching with fisher information and best interpolation on a strip

a block qmr algorithm for non hermitian linear systems with multiple right hand sides abstract many applications require the solution of multiple linear systems that have the same coefficient matrix but differ in their right hand sides instead of applying an iterative method to each of these systems individually it is often more efficient to employ a block version of the method that generates iterates for all the systems simultaneously in this paper we propose a block version of freund and nachtigal s quasi minimal residual qmr method for the iterative solution of non hermitian linear systems the block qmr method uses a novel lanczos type process for multiple starting vectors which was recently developed by aliaga boley freund and hernandez to compute suitable basis vectors for the underlying block krylov subspaces we describe the basic block qmr method and also give important implementation details in particular we show how to incorporate deflation to drop converged linear systems and to delete linearly and almost linearly dependent vectors in the underlying block krylov sequences numerical results are reported that illustrate typical features of the block qmr method

probability bounds with cherry trees a third order upper bound is presented on the probability of the union of a finite number of events by means of graphs called cherry trees these are graphs that we construct recursively in such a way that every time we pick a new vertex connect it with two already existing vertices if the latters are always adjacent we call the cherry tree a t cherry tree a cherry tree has a weight that provides us with the upper bound on the union any hunter worsley bound can be majorized by a t cherry bound constructed by the use of the hunter worsley tree a cherry tree bound can be identified as a feasible solution to the dual of the boolean probability bounding problem a t cherry tree bound can be identified as the objective function value of the dual vector corresponding to a dual feasible basis in the boolean problem this enables us to improve on the bound algorithmically if we use the dual method of linear programming

continuous cubic formulations for cluster detection problems in networks the celebrated motzkin straus formulation for the maximum clique problem provides a nontrivial characterization of the clique number of a graph in terms of the maximum value of a nonconvex quadratic function over a standard simplex it was originally developed as a way of proving turan s theorem in graph theory but was later used to develop competitive algorithms for the maximum clique problem based on continuous optimization clique relaxations such as s defective clique and s plex emerged as attractive more practical alternatives to cliques in network based cluster detection models arising in numerous applications this paper establishes continuous cubic formulations for the maximum s defective clique problem and the maximum s plex problem by generalizing the motzkin straus formulation to the corresponding clique relaxations the formulations are used to extend turan s theorem and other known lower bounds on the clique number to the considered clique relaxations results of preliminary numerical experiments with the conopt solver demonstrate that the proposed formulations can be used to quickly compute high quality solutions for the maximum s defective clique problem and the maximum s plex problem the proposed formulations can also be used to generate interesting test instances for global optimization solvers

sharp bounds on probabilities using linear programming in a previous paper 1988 the author proposed methods to obtain sharp lower and upper bounds for probabilities that at least one out of n events occurs based on the knowledge of some of the binomial moments of the number of events which occur and linear programming formulations this paper presents further results concerning other and more general logical functions of events we give sharp lower and upper bounds for the probabilities that a exactly r events b at least r events occur using linear programming the basic facts are expressed by the dual feasible basis characterization theorems which are interpreted in terms of the vertices of the dual problems we mention some linear inequalities among the binomial moments generalize the theory for the case of nonconsecutive binomial moments and present numerical examples

a complete characterization of the gap between convexity and sos convexity our first contribution in this paper is to prove that three natural sum of squares sos based sufficient conditions for convexity of polynomials via the definition of convexity its first order characterization and its second order characterization are equivalent these three equivalent algebraic conditions henceforth referred to as sos convexity can be checked by semidefinite programming whereas deciding convexity is np hard if we denote the set of convex and sosconvex polynomials in n variables of degree d with cn d and cn d respectively then our main contribution is to prove that cn d cn d if and only if n 1 or d 2 or n d 2 4 we also present a complete characterization for forms homogeneous polynomials except for the case n d 3 4 which is joint work with g blekherman and is to be published elsewhere our result states that the set cn d of convex forms in n variables of degree d equals the set cn d of sos convex forms if and only if n 2 or d 2 or n d 3 4 to prove these results we present in particular explicit examples of polynomials in c2 6 c2 6 and c3 4 c3 4 and forms in c3 6 c3 6 and c4 4 c4 4 and a general procedure for constructing forms in cn d 2 cn d 2 from nonnegative but not sos forms in n variables and degree d although for disparate reasons the remarkable outcome is that convex polynomials resp forms are sos convex exactly in cases where nonnegative polynomials resp forms are sums of squares as characterized by hilbert

machine learning and portfolio optimization the portfolio optimization model has limited impact in practice because of estimation issues when applied to real data to address this we adapt two machine learning methods regularization and cross validation for portfolio optimization first we introduce performance based regularization pbr where the idea is to constrain the sample variances of the estimated portfolio risk and return which steers the solution toward one associated with less estimation error in the performance we consider pbr for both mean variance and mean conditional value at risk cvar problems for the mean variance problem pbr introduces a quartic polynomial constraint for which we make two convex approximations one based on rank 1 approximation and another based on a convex quadratic approximation the rank 1 approximation pbr adds a bias to the optimal allocation and the convex quadratic approximation pbr shrinks the sample covariance matrix for the mean cvar problem the pbr model is a combinatorial optimization problem but we prove its convex relaxation a quadratically constrained quadratic program is essentially tight we show that the pbr models can be cast as robust optimization problems with novel uncertainty sets and establish asymptotic optimality of both sample average approximation saa and pbr solutions and the corresponding efficient frontiers to calibrate the right hand sides of the pbr constraints we develop new performance based k fold cross validation algorithms using these algorithms we carry out an extensive empirical investigation of pbr against saa as well as l1 and l2 regularizations and the equally weighted portfolio we find that pbr dominates all other benchmarks for two out of three fama french data sets this paper was accepted by yinyu ye optimization

maximum entropy methods as the bridge between microscopic and macroscopic theory this paper is concerned with an investigation into a function of macroscopic variables known as the singular potential building on previous work by ball and majumdar the singular potential is a function of the admissible statistical averages of probability distributions on a state space defined so that it corresponds to the maximum possible entropy given known observed statistical averages although non classical entropy like objective functions will also be considered first the set of admissible moments must be established and under the conditions presented in this work the set is open bounded and convex allowing a description in terms of supporting hyperplanes which provides estimates on the development of singularities for related probability distributions under appropriate conditions it is shown that the singular potential is strictly convex as differentiable as the microscopic entropy and blows up uniformly as the macroscopic variable tends to the boundary of the set of admissible moments applications of the singular potential are then discussed and particular consideration will be given to certain free energy functionals typical in mean field theory demonstrating an equivalence between certain microscopic and macroscopic free energy functionals this allows statements about l 1 local minimisers of onsager s free energy to be obtained which cannot be given by two sided variations and overcomes the need to ensure local minimisers are bounded away from zero and infty before taking l infty variations the analysis also permits the definition of a dual order parameter for which onsager s free energy allows an explicit representation also the difficulties in approximating the singular potential by everywhere defined functions in particular by polynomial functions are addressed with examples demonstrating the failure of the taylor approximation to preserve relevant shape properties of the singular potential