model order selection of damped sinusoids in noise by predictive densities we develop a procedure for the order selection of damped sinusoidal models based on the maximum a posteriori map criterion the proposed method merges the concept of predictive densities with bayesian inference to arrive at a complex multidimensional integral whose solution is achieved by way of the efficient monte carlo importance sampling technique the importance function a multivariate cauchy probability density is employed to produce stratified samples over the hypersurfaces support region centrality location parameters for the cauchy are resolved by exploiting the special structure of the compressed likelihood function clf and applying the fast maximum likelihood fml procedure of umesh and tufts simulation results allow for a comparison between our method and the singular value decomposition svd based information theoretic criteria of reddy and biradar see ieee trans signal processing vol 41 no 9 p 2872 81 1993

3d human pose tracking priors using geodesic mixture models we present a novel approach for learning a finite mixture model on a riemannian manifold in which euclidean metrics are not applicable and one needs to resort to geodesic distances consistent with the manifold geometry for this purpose we draw inspiration on a variant of the expectation maximization algorithm that uses a minimum message length criterion to automatically estimate the optimal number of components from multivariate data lying on an euclidean space in order to use this approach on riemannian manifolds we propose a formulation in which each component is defined on a different tangent space thus avoiding the problems associated with the loss of accuracy produced when linearizing the manifold with a single tangent space our approach can be applied to any type of manifold for which it is possible to estimate its tangent space additionally we consider using shrinkage covariance estimation to improve the robustness of the method especially when dealing with very sparsely distributed samples we evaluate the approach on a number of situations going from data clustering on manifolds to combining pose and kinematics of articulated bodies for 3d human pose tracking in all cases we demonstrate remarkable improvement compared to several chosen baselines

short presburger arithmetic is hard we study the computational complexity of short sentences in presburger arithmetic short pa here by short we mean sentences with a bounded number of variables quantifiers inequalities and bo

a polynomial time algorithm for counting integral points in polyhedra when the dimension is fixed we prove that for any dimension d there exists a polynomial time algorithm for counting integral points in polyhedra in the d dimensional euclidean space previously such algorithms were known for dimensions d 1 2 3 and 4 only

intrinsic bayesian active contours for extraction of object boundaries in images we present a framework for incorporating prior information about high probability shapes in the process of contour extraction and object recognition in images here one studies shapes as elements of an infinite dimensional non linear quotient space and statistics of shapes are defined and computed intrinsically using differential geometry of this shape space prior models on shapes are constructed using probability distributions on tangent bundles of shape spaces similar to the past work on active contours where curves are driven by vector fields based on image gradients and roughness penalties we incorporate the prior shape knowledge in the form of vector fields on curves through experimental results we demonstrate the use of prior shape models in the estimation of object boundaries and their success in handling partial obscuration and missing data furthermore we describe the use of this framework in shape based object recognition or classification

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

algorithmic and modeling insights via volumetric comparison of polyhedral relaxations this is mostly a survey on some mathematical results concerning volumes of polytopes of interest in non convex optimization our motivation is in geometrically comparing relaxations in the context of mixed integer linear and nonlinear optimization with the goal of gaining algorithmic and modeling insights we consider relaxations of fixed charge formulations vertex packing polytopes boolean quadric polytopes and relaxations of graphs of monomials on box domains besides surveying the area we do give a few new results and we provide many directions for further work

a comparison of the sherali adams lov 341sz schrijver and lasserre relaxations for 0 1 programming sherali and adams sa90 lov asz and schrijver ls91 and recently lasserre las01b have proposed lift and project methods for constructing hierarchies of successive linear or semidefinite relaxations of a 0 1 polytope p subseteq or n converging to p in n steps lasserre s approach uses results about representations of positive polynomials as sums of squares and the dual theory of moments we present the three methods in a common elementary framework and show that the lasserre construction provides the tightest relaxations of p as an application this gives a direct simple proof for the convergence of the lasserre s hierarchy we describe applications to the stable set polytope and to the cut polytope

applying mdl to learn best model granularity abstract the minimum description length mdl principle is solidly based on a provably ideal method of inference using kolmogorov complexity we test how the theory behaves in practice on a general problem in model selection that of learning the best model granularity the performance of a model depends critically on the granularity for example the choice of precision of the parameters too high precision generally involves modeling of accidental noise and too low precision may lead to confusion of models that should be distinguished this precision is often determined ad hoc in mdl the best model is the one that most compresses a two part code of the data set this embodies occam s razor in two quite different experimental settings the theoretical value determined using mdl coincides with the best value found experimentally in the first experiment the task is to recognize isolated handwritten characters in one subject s handwriting irrespective of size and orientation based on a new modification of elastic matching using multiple prototypes per character the optimal prediction rate is predicted for the learned parameter length of sampling interval considered most likely by mdl which is shown to coincide with the best value found experimentally in the second experiment the task is to model a robot arm with two degrees of freedom using a three layer feed forward neural network where we need to determine the number of nodes in the hidden layer giving best modeling performance the optimal model the one that extrapolizes best on unseen examples is predicted for the number of nodes in the hidden layer considered most likely by mdl which again is found to coincide with the best value found experimentally

data driven version of neyman s smooth test of fit abstract neyman s smooth test for testing uniformity is a recognized goodness of fit procedure as stated by lariccia the test can be viewed as a compromise between omnibus test procedures with generally low power in all directions and procedures whose power is focused in the direction of a specific alternative the basic idea behind this test is to embed the null density into say a k dimensional exponential family and then to construct an asymptotically optimal test for the parametric testing problem the resulting procedure is neyman s test with k components the most difficult problem related with using this test is the choice of k recommendations in statistical literature are sometimes confusing some authors advocate a small number of components whereas others show that in some situations a larger number of components is profitable all existing suggestions concerning how to select k exploit in fact some preliminary knowledge about a possible alternative in this article a new data driven met

a riemannian bayesian framework for estimating diffusion tensor images diffusion tensor magnetic resonance imaging dt mri is a non invasive imaging technique allowing to estimate the molecular self diffusion tensors of water within surrounding tissue due to the low signal to noise ratio of magnetic resonance images reconstructed tensor images usually require some sort of regularization in a post processing step previous approaches are either suboptimal with respect to the reconstruction or regularization step this paper presents a bayesian approach for simultaneous reconstruction and regularization of dt mr images that allows to resolve the disadvantages of previous approaches to this end estimation theoretical concepts are generalized to tensor valued images that are considered as riemannian manifolds doing so allows us to derive a maximum a posteriori estimator of the tensor image that considers both the statistical characteristics of the rician noise occurring in mr images as well as the nonlinear structure of tensor valued images experiments on synthetic data as well as real dt mri data validate the advantage of considering both statistical as well as geometrical characteristics of dt mri

lagrangian coordination for enhancing the convergence of analytical target cascading doi 10 2514 1 15326 analytical target cascading is a hierarchical multilevel multidisciplinary design methodology in analytical target cascading top level design targets i e specifications are propagated to lower level design problems in a consistent and efficient manner in this paper a modified lagrangian dual formulation and coordination for analytical target cascadingaredevelopedtoenhanceaformulation andcoordinationproposedearlierintheliterature theproposed approach guarantees all the properties established earlier and additionally offers new significant advantages as established previously for the convex case the proposed analytical target cascading coordination converges to a global optimal solution with corresponding optimal lagrange multipliers in the dual space the lagrange multipliers can be viewed as the weights for deviations in analytical target cascading formulations thus the proposed coordination algorithm finds the optimal solution and the optimal weights for the deviation terms simultaneously theenhancementallowsfortargetcascadingbetweenlevels fortheuseofaugmentedlagrangianto improveconvergenceofthecoordinationalgorithm andforpreventionofunboundedness aguidelinetosetthestep size for subgradient optimization when solving the lagrangian dual problem is also proposed

estimation and inference by compact coding summary the systematic variation within a set of data as represented by a usual statistical model may be used to encode the data in a more compact form than would be possible if they were considered to be purely random the encoded form has two parts the first states the inferred estimates of the unknown parameters in the model the second states the data using an optimal code based on the data probability distribution implied by those parameter estimates choosing the model and the estimates that give the most compact coding leads to an interesting general inference procedure in its strict form it has great generality and several nice properties but is computationally infeasible an approximate form is developed and its relation to other methods is explored

sequentially decomposed programming model based decomposition is a powerful tool for breaking design problems into smaller subproblems establishing hierarchical structure and analyzing the interrelations in engineering design problems however the theoretical foundation for solving decomposed nonlinear optimization problems requires further work we show that theformulation of the coordination problem is critical in quickly identifying thecorrectactiveconstraints and that solving subproblems independently may hinder the local convergence of algorithms tailored to hierarchical coordination yet hierarchical decomposition algorithms can have excellent global convergence properties and can be expected to exhibit superior improvement in the rst few iterations when compared to the undecomposed case basedontheseinsights agenericsequentially decomposedprogramming sdp algorithmisoutlined sdphastwo phases far from the solution rst phase decomposition is used close to the solution second phase subproblems are not solved separately the generic sdp is applied to sequential quadratic programming sqp to de ne an sdp sqp implementation a global convergence proof and a simple example are given

computing the ehrhart quasi polynomial of a rational simplex we present a polynomial time algorithm to compute any fixed number of the highest coefficients of the ehrhart quasi polynomial of a rational simplex previously such algorithms were known for integer simplices and for rational polytopes of a fixed dimension the algorithm is based on the formula relating the kth coefficient of the ehrhart quasi polynomial of a rational polytope to volumes of sections of the polytope by affine lattice subspaces parallel to k dimensional faces of the polytope we discuss possible extensions and open questions

hierarchical overlapping coordination for large scale optimization by decomposition decomposition of large engineering design problems into smaller design subproblems enhances robustness and speed of numerical solution algorithms design subproblems can be solved in parallel using the optimization technique most suitable for the underlying subproblem this also reflects the typical multidisciplinary nature of system design problems and allows better interpretation of results hierarchical overlapping coordination hoc simultaneously uses two or more problem decompositions each of them associated with different partitions of the design variables and constraints coordination is achieved by the exchange of information between decompositions we present the hoc algorithm and a sufficient condition for global convergence of the algorithm to the solution of a convex optimization problem the convergence condition involves the rank of a matrix derived from the jacobian of the constraints computational results obtained by applying the hoc algorithm to problems of various sizes are also presented

integer programming with a fixed number of variables it is shown that the integer linear programming problem with a fixed number of variables is polynomially solvable the proof depends on methods from geometry of numbers

when is rounding allowed in integer nonlinear optimization abstract in this paper we consider nonlinear integer optimization problems nonlinear integer programming has mainly been studied for special classes such as convex and concave objective functions and polyhedral constraints in this paper we follow an other approach which is not based on convexity or concavity studying geometric properties of the level sets and the feasible region we identify cases in which an integer minimizer of a nonlinear program can be found by rounding up or down the coordinates of a solution to its continuous relaxation we call this property rounding property if it is satisfied it enables us for fixed dimension to solve an integer programming problem in the same time complexity as its continuous relaxation we also investigate the strong rounding property which allows rounding a solution to the continuous relaxation to the next integer solution and in turn yields that the integer version can be solved in the same time complexity as its continuous relaxation for arbitrary dimensions

voronoi regions of lattices second moments of polytopes and quantization if a point is picked at random inside a regular simplex octahedron 600 cell or other polytope what is its average squared distance from the centroid in n dimensional space what is the average squared distance of a random point from the closest point of the lattice a n or d n e n a n ast or d n ast the answers are given here together with a description of the voronoi or nearest neighbor regions of these lattices the results have applications to quantization and to the design of signals for the gaussian channel for example a quantizer based on the eight dimensional lattice e8 has a mean squared error per symbol of 0 0717 cdots when applied to uniformly distributed data compared with 0 08333 cdots for the best one dimensional quantizer

a method for autoregressive moving average estimation summary the problem considered is that of estimating an autoregressive moving average system including estimating the degrees of the autoregressive and moving average lag operators the basic method is that introduced by hannan rissanen 1982 however that method may sometimes overestimate the degrees and modifications are here introduced to correct this the problem is itself due to the use of a long autoregression of order clog t when t is large in the first stage of the process the effect of this is investigated and in particular its effect on the speed of convergence of the estimates

the selection of prior distributions by formal rules abstract subjectivism has become the dominant philosophical foundation for bayesian inference yet in practice most bayesian analyses are performed with so called noninformative priors that is priors constructed by some formal rule we review the plethora of techniques for constructing such priors and discuss some of the practical and philosophical issues that arise when they are used we give special emphasis to jeffreys s rules and discuss the evolution of his viewpoint about the interpretation of priors away from unique representation of ignorance toward the notion that they should be chosen by convention we conclude that the problems raised by the research on priors chosen by formal rules are serious and may not be dismissed lightly when sample sizes are small relative to the number of parameters being estimated it is dangerous to put faith in any default solution but when asymptotics take over jeffreys s rules and their variants remain reasonable choices we also provide an annotated b

universal modeling and coding the problems arising in the modeling and coding of strings for compression purposes are discussed the notion of an information source that simplifies and sharpens the traditional one is axiomatized and adaptive and nonadaptive models are defined with a measure of complexity assigned to the models a fundamental theorem is proved which states that models that use any kind of alphabet extension are inferior to the best models using no alphabet extensions at all a general class of so called first in first out fifo arithmetic codes is described which require no alphabet extension devices and which therefore can be used in conjunction with the best models because the coding parameters are the probabilities that define the model their design is easy and the application of the code is straightforward even with adaptively changing source models

invariant bayesian estimation on manifolds a frequent and well founded criticism of the maximum a posteriori map and minimum mean squared error mmse estimates of a continuous parameter taking values in a differentiable manifold is that they are not invariant to arbitrary reparameterizations of this paper clarifies the issues surrounding this problem by pointing out the difference between coordinate invariance which is a sine qua non for a mathematically well defined problem and diffeomorphism invariance which is a substantial issue and then provides a solution we first show that the presence of a metric structure on can be used to define coordinate invariant map and mmse estimates and we argue that this is the natural way to proceed we then discuss the choice of a metric structure on by imposing an invariance criterion natural within a bayesian framework we show that this choice is essentially unique it does not necessarily correspond to a choice of coordinates in cases of complete prior ignorance when jeffreys prior is used the invariant map estimate reduces to the maximum likelihood estimate the invariant map estimate coincides with the minimum message length mml estimate but no discretization or approximation is used in its derivation

global optimization with polynomials and the problem of moments we consider the problem of finding the unconstrained global minimum of a real valued polynomial p x mathbb r n to mathbb r as well as the global minimum of p x in a compact set k defined by polynomial inequalities it is shown that this problem reduces to solving an often finite sequence of convex linear matrix inequality lmi problems a notion of karush kuhn tucker polynomials is introduced in a global optimality condition some illustrative examples are provided

minimum description length induction bayesianism and kolmogorov complexity the relationship between the bayesian approach and the minimum description length approach is established we sharpen and clarify the general modeling principles minimum description length mdl and minimum message length mml abstracted as the ideal mdl principle and defined from bayes s rule by means of kolmogorov complexity the basic condition under which the ideal principle should be applied is encapsulated as the fundamental inequality which in broad terms states that the principle is valid when the data are random relative to every contemplated hypothesis and also these hypotheses are random relative to the universal prior the ideal principle states that the prior probability associated with the hypothesis should be given by the algorithmic universal probability and the sum of the log universal probability of the model plus the log of the probability of the data given the model should be minimized if we restrict the model class to finite sets then application of the ideal principle turns into kolmogorov s minimal sufficient statistic in general we show that data compression is almost always the best strategy both in model selection and prediction

local asymptotic coding and the minimum description length local asymptotic arguments imply that parameter selection via the minimum description length mdl resembles a traditional hypothesis test a common approximation for mdl estimates the cost of adding a parameter at about 1 2 log n bits for a model fit to n observations while accurate for parameters which are large on a standardized scale this approximation overstates the parameter cost near zero we find that encoding the parameter produces a shorter description length when the corresponding estimator is about two standard errors away from zero as in a traditional statistical hypothesis test

rational generating functions and integer programming games we explore the computational complexity of computing pure nash equilibria for a new class of strategic games called integer programming games with differences of piecewise linear convex functions as payoffs integer programming games are games where players action sets are integer points inside of polytopes using recent results from the study of short rational generating functions for encoding sets of integer points pioneered by alexander barvinok we present efficient algorithms for enumerating all pure nash equilibria and other computations of interest such as the pure price of anarchy and pure threat point when the dimension and number of convex linear pieces in the payoff functions are fixed sequential games where a leader is followed by competing followers a stackelberg nash setting are also considered

fptas for optimizing polynomials over the mixed integer points of polytopes in fixed dimension we show the existence of a fully polynomial time approximation scheme fptas for the problem of maximizing a non negative polynomial over mixed integer sets in convex polytopes when the number of variables is fixed moreover using a weaker notion of approximation we show the existence of a fully polynomial time approximation scheme for the problem of maximizing or minimizing an arbitrary polynomial over mixed integer sets in convex polytopes when the number of variables is fixed

a universal data compression system a universal data compression algorithm is described which is capable of compressing long strings generated by a finitely generated source with a near optimum per symbol length without prior knowledge of the source this class of sources may be viewed as a generalization of markov sources to random fields moreover the algorithm does not require a working storage much larger than that needed to describe the source generating parameters

asymptotic quantization error of continuous signals and the quantization dimension extensions of the limiting qnanfizafion error formula of bennet are proved these are of the form d s k n f n beta b where n is the number of output levels d s k n f is the s th moment of the metric distance between quantizer input and output beta b 0 k s beta is the signal space dimension and f is the signal distribution if a suitably well behaved k dimensional signal density f x exists b b s k int f rho x dx 1 rho rho k s k and b s k does not depend on f for k 1 s 2 this reduces to bennett s formula if f is the cantor distribution on 0 1 0 and this k equals the fractal dimension of the cantor set 12 13 random quantization optimal quantization in the presence of an output information constraint and quantization noise in high dimensional spaces are also investigated

minimizing cubic and homogeneous polynomials over integers in the plane we complete the complexity classification by degree of minimizing a polynomial over the integer points in a polyhedron in a real vector space of dimension two previous work shows that optimizing a quadratic polynomial over the integer points in a polyhedral region in a real vector space of dimension two can be done in polynomial time whereas optimizing a quartic polynomial in the same type of region is np hard we close the gap by showing that this problem can be solved in polynomial time for cubic polynomials furthermore we show that the problem of minimizing a homogeneous polynomial of any fixed degree over the integer points in a bounded polyhedron in a real vector space of dimension two is solvable in polynomial time we show that this holds for polynomials that can be translated into homogeneous polynomials even when the translation vector is unknown we demonstrate that such problems in the unbounded case can have smallest optimal solutions of exponential size in the size of the input thus req

information gain and a general measure of correlation summary given a parametric model of dependence between two random quantities x and y the notion of information gain can be used to define a measure of correlation this definition of correlation generalizes both the usual product moment correlation coeffi cient for the bivariate normal model and the multiple correlation coefficient in the standard linear regression model the use of this information based correlation in a descriptive statistical analysis is examined and several examples are given if the dependence between two random quantities x and y is modelled parametri cally then the concept of information gain can be used to define a measure of correlation this correlation coefficient can appear in two possible contexts depending on whether one models the joint distribution of x and y or just the conditional distribution of y given x an important motivating feature for this information based correlation coefficient is the fact that it generalizes both the usual product moment correlation coefficient for the bivariate normal model and the usual multiple correlation coefficient for the standard multiple regression model with normal errors our intuition is well developed for these usual correlation coefficients and hopefully our intuition will still be applicable for the information based correlation in more general modelling situations of parametric dependence further since our correlation coefficient is based on information gain we might hope to extend our intuition to interpret the information gain in any statistical modelling situation where we want to assess how much better a more complicated model is than a simpler model the concept of information gain for general statistical models is described in 2 this concept is then used to define an information based measure of correlation the joint case is covered in 3 and the conditional case in 4 estimation of the correlation coefficient is carried out by estimating the corresponding information gain see 5 7 the use of information gain for the purpose of model choice in a descriptive statistical analysis is discussed in 8 and a comparison between our approach and akaike s information criterion is given in 9 some examples of the use of this information based correlation are given in 10 and 11

maximum likelihood estimation of models for residual covariance in spatial regression we describe the maximum likelihood method for fitting the linear model when residuals are correlated and when the covariance among the residuals is determined by a parametric model containing unknown parameters observations are assumed to be gaussian we give conditions which ensure consistency and asymptotic normality of the estimators our main concern is with the analysis of spatial data and in this context we describe some simulation experiments to assess the small sample behaviour of estimators we also discuss an application of the spectral approximation to the likelihood for processes on a lattice

recursive estimation of mixed autoregressive moving average order summary the order p q of an autoregressive moving average sequence y t may be estimated by minimizing a criterion log 2 p q log t t with respect to p and q where 2 is the maximum likelihood estimate of the variance of the innovations t it is suggested that instead a2 be estimated from a series of regressions of y t on y t 1 y t 2 y t p 5 t 1 t q where the t are obtained by fitting a long autoregression to the data it is shown how the sequence of regressions may for p q be economically recursively calculated by embedding them in a sequence of bivariate autoregressions asymptotic properties of the procedure are established under very general conditions

wavelet modeling using finite mixtures of generalized gaussian distributions application to texture discrimination and retrieval this paper addresses statistical based texture modeling using wavelets we propose a new approach to represent the marginal distribution of the wavelet coefficients using finite mixtures of generalized gaussian mogg distributions the mogg captures a wide range of histogram shapes which provides better description and discrimination of texture than using single probability density functions pdf s as proposed by recent state of the art approaches moreover we propose a model similarity measure based on kullback leibler divergence kld approximation using monte carlo sampling methods through experiments on two popular texture data sets we show that our approach yields significant performance improvements for texture discrimination and retrieval as compared with recent methods of statistical based wavelet modeling

mle s bias pathology model updated mle and wallace s minimum message length method the inherent bias pathology of the maximum likelihood estimation method is confirmed for models with unknown parameters and when maximum likelihood estimate mle is function of mle to reduce s bias the likelihood equation to be solved for is updated using the model for the data y in it for various models with a shape parameter model updated mu mle mu reduces s bias for the pareto model mu reduces in addition s variance the results explain the difference that puzzled fisher between biased and the unbiased estimate he obtained for two models with the abandoned two stage procedure used in mumle s implementation mu is also obtained with the minimum message length method thus motivating the use of priors in frequentist inference

integer polynomial optimization in fixed dimension we classify according to their computational complexity integer optimization problems whose constraints and objective functions are polynomials with integer coefficients and the number of variables is fixed for the optimization of an integer polynomial over the lattice points of a convex polytope we show an algorithm to compute lower and upper bounds for the optimal value for polynomials that are nonnegative over the polytope these sequences of bounds lead to a fully polynomial time approximation scheme for the optimization problem