convex relaxations of 0 1 quadratic programming we consider three parametric relaxations of the 0 l quadratic programming problem these relaxations are to quadratic maximization over simple box constraints quadratic maximization over the sphere and the maximum eigenvalue of a bordered matrix when minimized over the parameter each of the relaxations provides an upper bound on the original discrete problem moreover these bounds are efficiently computable our main result is that surprisingly all three bounds are equal

bounding the gap between the mccormick relaxation and the convex hull for bilinear functions we investigate how well the graph of a bilinear function b 0 1 n rightarrow mathbb r b 0 1 n r can be approximated by its mccormick relaxation in particular we are interested in the smallest number c such that the difference between the concave upper bounding and convex lower bounding functions obtained from the mccormick relaxation approach is at most c times the difference between the concave and convex envelopes answering a question of luedtke namazifar and linderoth we show that this factor c cannot be bounded by a constant independent of n more precisely we show that for a random bilinear function b we have asymptotically almost surely c geqslant sqrt n 4 c n 4 on the other hand we prove that c leqslant 600 sqrt n c 600n which improves the linear upper bound proved by luedtke namazifar and linderoth in addition we present an alternative proof for a result of misener smadbeck and floudas characterizing functions b for which the mccormick relaxation is equal to the convex hull

a strong dual for conic mixed integer programs mixed integer conic programming is a generalization of mixed integer linear programming in this paper we present an extension of the duality theory for mixed integer linear programming see m guzelsoy and t k ralphs int j oper res taichung 4 2007 pp 118 137 g l nemhauser and l a wolsey integer and combinatorial optimization wiley interscience new york 1988 to the case of mixed integer conic programming in particular we construct a subadditive dual for mixed integer conic programming problems under a simple condition on the primal problem we show that strong duality holds

small and strong formulations for unions of convex sets from the cayley embedding there is often a significant trade off between formulation strength and size in mixed integer programming when modeling convex disjunctive constraints e g unions of convex sets adding auxiliary continuous variables can sometimes help resolve this trade off however standard formulations that use such auxiliary continuous variables can have a worse than expected computational effectiveness which is often attributed precisely to these auxiliary continuous variables for this reason there has been considerable interest in constructing strong formulations that do not use continuous auxiliary variables we introduce a technique to construct formulations without these detrimental continuous auxiliary variables to develop this technique we introduce a natural non polyhedral generalization of the cayley embedding of a family of polytopes and show it inherits many geometric properties of the original embedding we then show how the associated formulation technique can be used to construct small and strong formulation for a wide range of disjunctive constraints in particular we show it can recover and generalize all known strong formulations without continuous auxiliary variables

strong mixed integer programming formulations for trained neural networks we present strong mixed integer programming mip formulations for high dimensional piecewise linear functions that correspond to trained neural networks these formulations can be used for a number of important tasks such as verifying that an image classification network is robust to adversarial inputs or solving decision problems where the objective function is a machine learning model we present a generic framework which may be of independent interest that provides a way to construct sharp or ideal formulations for the maximum of d affine functions over arbitrary polyhedral input domains we apply this result to derive mip formulations for a number of the most popular nonlinear operations e g relu and max pooling that are strictly stronger than other approaches from the literature we corroborate this computationally showing that our formulations are able to offer substantial improvements in solve time on verification tasks for image classification networks

predictive updating methods with application to bayesian classification we propose algorithms based on random draws from predictive distributions of unknown quantities missing values for instance this procedure can either be iterative which is a special variation of the gibbs sampler or be sequential which is a variation of sequential imputation in the latter case one can update the posterior distribution with new observations easily the methods proposed have intuitive statistical implications and can be generalized to accommodate other bayesian like procedures we display some applications of the method in connection with the bayesian bootstrap classification hierarchical models and selection of variables in particular as an application of the method we present a unified treatment of switching regression models driven by a general binary process and we develop a bayesian testing procedure some simulations and a real example are used to illustrate the methods proposed

on mathematical programming with indicator constraints in this paper we review the relevant literature on mathematical optimization with logical implications i e where constraints can be either active or disabled depending on logical conditions to hold in the case of convex functions the theory of disjunctive programming allows one to formulate these logical implications as convex nonlinear programming problems in a space of variables lifted with respect to its original dimension we concentrate on the attempt of avoiding the issue of dealing with large nlps in particular we review some existing results that allow to work in the original space of variables for two relevant special cases where the disjunctions corresponding to the logical implications have two terms then we significantly extend these special cases in two different directions one involving more general convex sets and the other with disjunctions involving three terms computational experiments comparing disjunctive programming formulations in the original space of variables with straightforward bigm ones show that the former are computationally viable and promising

singular values of a real rectangular tensor real rectangular tensors arise from the strong ellipticity condition problem in solid mechanics and the entanglement problem in quantum physics in this paper we systematically study properties of singular values of a real rectangular tensor and give an algorithm to find the largest singular value of a nonnegative rectangular tensor numerical results show that the algorithm is efficient

on projection algorithms for solving convex feasibility problems due to their extraordinary utility and broad applicability in many areas of classical mathematics and modern physical sciences most notably computerized tomography algorithms for solving convex feasibility problems continue to receive great attention to unify generalize and review some of these algorithms a very broad and flexible framework is investigated several crucial new concepts which allow a systematic discussion of questions on behaviour in general hilbert spaces and on the quality of convergence are brought out numerous examples are given

estimating latent distributions in recurrent choice data this paper introduces a flexible class of stochastic mixture models for the analysis and interpretation of individual differences in recurrent choice and other types of count data these choice models are derived by specifying elements of the choice process at the individual level probability distributions are introduced to describe variations in the choice process among individuals and to obtain a representation of the aggregate choice behavior due to the explicit consideration of random effect sources the choice models are parsimonious and readily interpretable an easy to implement em algorithm is presented for parameter estimation two applications illustrate the proposed approach

ellipticity and deformations with discontinuous gradients in finite elastostatics loss of ellipticity of the equilibrium equations of finite elastostatics is closely related to the possible emergence of elastostatic shocks i e deformations with discontinuous gradients in certain situations where constitutive response functions are essentially one dimentional such as anti plane shear or bar theories strong ellipticity is closely related to convexity of the elastic potential and invertibility of certain constitutive response functions the present work addresses the analogous issues within the context of three dimensional elastostatics of compressible but not necessarily isotropic hyperelastic materials a certain direction dependent resolution of the deformation gradient is introduced and its existence and uniqueness for a given direction are established the elastic potential is expressed as a function of kinematic variables arising from this resolution strong ellipticity is shown to be equivalent to the positive definiteness of the hessian matrix of this function thus sufficing for its strict convexity the underlying variables are interpretable physically as simple shears and extensions their work conjugates define a traction response mapping it is shown that discontinuous deformation gradients are sustainable if and only if this mapping fails to be invertible this result is explicit in the sense that it characterizes the set of all possible piecewise homogeneous deformations given the elastic potential function

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

kernel based equiprobabilistic topographic map formation we introduce a new unsupervised competitive learning rule the kernel based maximum entropy learning rule kmer which performs equiprobabilistic topographic map formation in regular fixed topology lattices for use with nonparametric density estimation as well as nonparametric regression analysis the receptive fields of the formal neurons are overlapping radially symmetric kernels compatible with radial basis functions rbfs but unlike other learning schemes the radii of these kernels do not have to be chosen in an ad hoc manner the radii are adapted to the local input density together with the weight vectors that define the kernel centers so as to produce maps of which the neurons have an equal probability to be active equiprobabilistic maps both an online and a batch version of the learning rule are introduced which are applied to nonparametric density estimation and regression respectively the application envisaged is blind source separation bss from nonlinear noisy mixtures

m tensors and nonsingular m tensors abstract the m matrix is an important concept in matrix theory and has many applications recently this concept has been extended to higher order tensors 18 in this paper we establish some important properties of m tensors and nonsingular m tensors an m tensor is a z tensor we show that a z tensor is a nonsingular m tensor if and only if it is semi positive thus a nonsingular m tensor has all positive diagonal entries an m tensor regarding as the limit of a sequence of nonsingular m tensors has all nonnegative diagonal entries we introduce even order monotone tensors and present their spectral properties in matrix theory a z matrix is a nonsingular m matrix if and only if it is monotone this is no longer true in the case of higher order tensors we show that an even order monotone z tensor is an even order nonsingular m tensor but not vice versa an example of an even order nontrivial monotone z tensor is also given

the rate of convergence of a matrix power series abstract to estimate the truncation error of a matrix power series we need information about the magnitude of high powers of a matrix inequalities bearing on this question are surveyed and their use is exemplified by calculations of bounds for the truncation error of the geometric series in matrices

semi supervised cross entropy clustering with information bottleneck constraint abstract in this paper we propose a semi supervised clustering method cec ib that models data with a set of gaussian distributions and that retrieves clusters based on a partial labeling provided by the user partition level side information by combining the ideas from cross entropy clustering cec with those from the information bottleneck method ib our method trades between three conflicting goals the accuracy with which the data set is modeled the simplicity of the model and the consistency of the clustering with side information experiments demonstrate that cec ib performs similar as gaussian mixture models in a classical semi supervised scenario but is faster more robust to noisy labels automatically determines the optimal number of clusters and performs well when not all classes are present in the side information moreover in contrast to many other semi supervised models it can be successfully applied in discovering natural subgroups if the partition level side information is derived from the top levels of a hierarchical clustering

maximum likelihood estimation of latent interaction effects with the lms method in the context of structural equation modeling a general interaction model with multiple latent interaction effects is introduced a stochastic analysis represents the nonnormal distribution of the joint indicator vector as a finite mixture of normal distributions the latent moderated structural equations lms approach is a new method developed for the analysis of the general interaction model that utilizes the mixture distribution and provides a ml estimation of model parameters by adapting the em algorithm the finite sample properties and the robustness of lms are discussed finally the applicability of the new method is illustrated by an empirical example

code vector density in topographic mappings scalar case the author derives some new results that build on his earlier work 1989 of combining vector quantization vq theory and topographic mapping tm theory a vq model with a noisy transmission medium is used to model the processes that occur in tms which leads to the standard tm training algorithm albeit with a slight modification to the encoding process to emphasize this difference the model is called a topographic vector quantizer tvq in the continuum limit of the one dimensional scalar tvq it is found that the density of code vectors is proportional to p x sup a alpha 1 3 assuming that the transmission medium introduces additive noise with a zero mean symmetric monotically decreasing probability density this result is dramatically different from the result that is predicted when the standard tm training algorithm is used with a uniform symmetric neighborhood n n and it is noted that this difference arises entirely from using minimum distortion rather than nearest neighbor encoding

elasticity m tensors and the strong ellipticity condition abstract in this paper we establish two sufficient conditions for the strong ellipticity of any fourth order elasticity tensor and investigate a class of tensors satisfying the strong ellipticity condition the elasticity m tensor the first sufficient condition is that the strong ellipticity holds if the unfolding matrix of this fourth order elasticity tensor can be modified into a positive definite one by preserving the summations of some corresponding entries second an alternating projection algorithm is proposed to verify whether an elasticity tensor satisfies the first condition or not besides the elasticity m tensor is defined with respect to the m eigenvalues of elasticity tensors we prove that any nonsingular elasticity m tensor satisfies the strong ellipticity condition by employing a perron frobenius type theorem for m spectral radii of nonnegative elasticity tensors other equivalent definitions of nonsingular elasticity m tensors are also established

strong formulations for quadratic optimization with m matrices and indicator variables we study quadratic optimization with indicator variables and an m matrix i e a psd matrix with non positive off diagonal entries which arises directly in image segmentation and portfolio optimization with transaction costs as well as a substructure of general quadratic optimization problems we prove under mild assumptions that the minimization problem is solvable in polynomial time by showing its equivalence to a submodular minimization problem to strengthen the formulation we decompose the quadratic function into a sum of simple quadratic functions with at most two indicator variables each and provide the convex hull descriptions of these sets we also describe strong conic quadratic valid inequalities preliminary computational experiments indicate that the proposed inequalities can substantially improve the strength of the continuous relaxations with respect to the standard perspective reformulation

application of the em algorithm for the multitarget multisensor tracking problem an important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies a number of approaches have been proposed for this multitarget multisensor tracking problem ranging from reasonably simple though ad hoc schemes to fairly complex but theoretically optimum approaches in this paper we describe an iterative procedure for time recursive multitarget multisensor tracking based on use of the expectation maximization em algorithm more specifically we pose the multitarget multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data whereas the target associated sensor outputs constitute the complete data target updates at each time use an em based approach that calculates the maximum a posteriori map estimate of the target states under the assumption of appropriate motion models based on the outputs of disparate sensors the approach uses a markov random field mrf model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration the advantage of this em based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities we provide selected results illustrating the performance complexity characteristics of this em based approach compared with competing schemes

probabilistic self organizing maps for continuous data the original self organizing feature map did not define any probability distribution on the input space however the advantages of introducing probabilistic methodologies into self organizing map models were soon evident this has led to a wide range of proposals which reflect the current emergence of probabilistic approaches to computational intelligence the underlying estimation theories behind them derive from two main lines of thought the expectation maximization methodology and stochastic approximation methods here we present a comprehensive view of the state of the art with a unifying perspective of the involved theoretical frameworks in particular we examine the most commonly used continuous probability distributions self organization mechanisms and learning schemes special emphasis is given to the connections among them and their relative advantages depending on the characteristics of the problem at hand furthermore we evaluate their performance in two typical applications of self organizing maps classification and visualization

cardinality constrained linear quadratic optimal control as control implementation often incurs not only a variable cost associated with the magnitude or energy of the control but also a setup cost we consider a discrete time linear quadratic lq optimal control problem with a limited number of control implementations termed in this technical note the cardinality constrained linear quadratic optimal control cclq we first derive a semi analytical feedback policy for cclq problems using dynamic programming dp due to the exponential growth of the complexity in calculating the action regions however dp procedure is only efficient for cclq problems with a scalar state space recognizing this fact we develop then two lower bounding schemes and integrate them into a branch and bound bnb solution framework to offer an efficient algorithm in solving general cclq problems adopting the devised solution algorithm for cclq problems we can solve efficiently the linear quadratic optimal control problem with setup costs

cutting planes for optimization of convex functions over nonconvex sets we derive linear inequality characterizations for sets of the form mathop rm conv x q in mathbb r d times mathbb r q ge q x x in mathbb r d mathop rm int p where q is convex and differentiable and p subset mathbb r d we show that in several cases our characterization leads to polynomial time separation algorithms that operate in the original space of variables in particular when q is a positive definite quadratic and p is a polyhedron or an ellipsoid

feature selection based on the approximation of class densities by finite mixtures of special type abstract a new method of feature selection based on the approximation of class conditional densities by a mixture of parameterized densities of a special type suitable especially for multimodal data is presented no search procedure is needed when using the proposed method its performance is tested both on real simulated data

two term disjunctions on the second order cone balas introduced disjunctive cuts in the 1970s for mixed integer linear programs several recent papers have attempted to extend this work to mixed integer conic programs in this paper we study the structure of the convex hull of a two term disjunction applied to the second order cone and develop a methodology to derive closed form expressions for convex inequalities describing the resulting convex hull our approach is based on first characterizing the structure of undominated valid linear inequalities for the disjunction and then using conic duality to derive a family of convex possibly nonlinear valid inequalities that correspond to these linear inequalities we identify and study the cases where these valid inequalities can equivalently be expressed in conic quadratic form and where a single inequality from this family is sufficient to describe the convex hull in particular our results on two term disjunctions on the second order cone generalize related results on split cuts by modaresi kilinc and vielma and by andersen and jensen

model based clustering by probabilistic self organizing maps in this paper we consider the learning process of a probabilistic self organizing map pbsom as a model based data clustering procedure that preserves the topological relationships between data clusters in a neural network based on this concept we develop a coupling likelihood mixture model for the pbsom that extends the reference vectors in kohonen s self organizing map som to multivariate gaussian distributions we also derive three expectation maximization em type algorithms called the socem soem and sodaem algorithms for learning the model pbsom based on the maximum likelihood criterion socem is derived by using the classification em cem algorithm to maximize the classification likelihood soem is derived by using the em algorithm to maximize the mixture likelihood and sodaem is a deterministic annealing da variant of socem and soem moreover by shrinking the neighborhood size socem and soem can be interpreted respectively as da variants of the cem and em algorithms for gaussian model based clustering the experimental results show that the proposed pbsom learning algorithms achieve comparable data clustering performance to that of the deterministic annealing em daem approach while maintaining the topology preserving property

anderson acceleration for fixed point iterations this paper concerns an acceleration method for fixed point iterations that originated in work of d g anderson j assoc comput mach 12 1965 pp 547 560 which we accordingly call anderson acceleration here this method has enjoyed considerable success and wide usage in electronic structure computations where it is known as anderson mixing however it seems to have been untried or underexploited in many other important applications moreover while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities this method has received relatively little attention from these communities over the years a recent paper by h fang and y saad numer linear algebra appl 16 2009 pp 197 221 has clarified a remarkable relationship of anderson acceleration to quasi newton secant updating methods and extended it to define a broader anderson family of acceleration methods in this paper our goals are to shed additional light on anderson acceleration and to draw further attention to its usefulness as a general tool we first show that on linear problems anderson acceleration without truncation is essentially equivalent in a certain sense to the generalized minimal residual gmres method we also show that the type 1 variant in the fang saad anderson family is similarly essentially equivalent to the arnoldi full orthogonalization method we then discuss practical considerations for implementing anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications

assessing a mixture model for clustering with the integrated completed likelihood we propose an assessing method of mixture model in a cluster analysis setting with integrated completed likelihood for this purpose the observed data are assigned to unknown clusters using a maximum a posteriori operator then the integrated completed likelihood icl is approximated using the bayesian information criterion bic numerical experiments on simulated and real data of the resulting icl criterion show that it performs well both for choosing a mixture model and a relevant number of clusters in particular icl appears to be more robust than bic to violation of some of the mixture model assumptions and it can select a number of dusters leading to a sensible partitioning of the data

finite mixture varying coefficient models for analyzing longitudinal heterogenous data this paper aims to develop a mixture model to study heterogeneous longitudinal data on the treatment effect of heroin use from a california civil addict program each component of the mixture is characterized by a varying coefficient mixed effect model we use the bayesian p splines approach to approximate the varying coefficient functions we develop markov chain monte carlo algorithms to estimate the smooth functions unknown parameters and latent variables in the model we use modified deviance information criterion to determine the number of components in the mixture a simulation study demonstrates that the modified deviance information criterion selects the correct number of components and the estimation of unknown quantities is accurate we apply the proposed model to the heroin treatment study furthermore we identify heterogeneous longitudinal patterns copyright 2011 john wiley sons ltd

growing hierarchical probabilistic self organizing graphs since the introduction of the growing hierarchical self organizing map much work has been done on self organizing neural models with a dynamic structure these models allow adjusting the layers of the model to the features of the input dataset here we propose a new self organizing model which is based on a probabilistic mixture of multivariate gaussian components the learning rule is derived from the stochastic approximation framework and a probabilistic criterion is used to control the growth of the model moreover the model is able to adapt to the topology of each layer so that a hierarchy of dynamic graphs is built this overcomes the limitations of the self organizing maps with a fixed topology and gives rise to a faithful visualization method for high dimensional data

robust convex optimization we study convex optimization problems for which the data is not specified exactly and it is only known to belong to a given uncertainty set u yet the constraints must hold for all possible values of the data from u the ensuing optimization problem is called robust optimization in this paper we lay the foundation of robust convex optimization in the main part of the paper we show that if u is an ellipsoidal uncertainty set then for some of the most important generic convex optimization problems linear programming quadratically constrained programming semidefinite programming and others the corresponding robust convex program is either exactly or approximately a tractable problem which lends itself to efficientalgorithms such as polynomial time interior point methods

on convex relaxations for quadratically constrained quadratic programming we consider convex relaxations for the problem of minimizing a possibly nonconvex quadratic objective subject to linear and possibly nonconvex quadratic constraints let mathcal f denote the feasible region for the linear constraints we first show that replacing the quadratic objective and constraint functions with their convex lower envelopes on mathcal f is dominated by an alternative methodology based on convexifying the range of the quadratic form genfrac 0 0pt 1 x genfrac 0 0pt 1 x t for x in mathcal f we next show that the use of alpha bb underestimators as computable estimates of convex lower envelopes is dominated by a relaxation of the convex hull of the quadratic form that imposes semidefiniteness and linear constraints on diagonal terms finally we show that the use of a large class of d c difference of convex underestimators is dominated by a relaxation that combines semidefiniteness with rlt constraints

on minimal valid inequalities for mixed integer conic programs we study disjunctive conic sets involving a general regular closed convex full dimensional and pointed cone x1d4a6 such as the nonnegative orthant the lorentz cone or the positive semidefinite cone in a unified framework we introduce x1d4a6 minimal inequalities and show that under mild assumptions these inequalities together with the trivial cone implied inequalities are sufficient to describe the convex hull we focus on the properties of x1d4a6 minimal inequalities by establishing algebraic necessary conditions for an inequality to be x1d4a6 minimal this characterization leads to a broader algebraically defined class of x1d4a6 sublinear inequalities we demonstrate a close connection between x1d4a6 sublinear inequalities and the support functions of convex sets with a particular structure this connection results in practical ways of verifying x1d4a6 sublinearity and or x1d4a6 minimality of inequalities our study generalizes some of the results from the mixed integer linear case it is well known that the minimal inequalities for

intersection cuts for nonlinear integer programming convexification techniques for structured sets we study the generalization of split k branch split and intersection cuts from mixed integer linear programming to the realm of mixed integer nonlinear programming constructing such cuts requires calculating the convex hull of the difference between a convex set and an open set with a simple geometric structure we introduce two techniques to give precise characterizations of such convex hulls and use them to construct split k branch split and intersection cuts for several classes of non polyhedral sets in particular we give simple formulas for split cuts for essentially all convex sets described by a single conic quadratic inequality we also give simple formulas for k branch split cuts and some general intersection cuts for a wide variety of convex quadratic sets

fast approximate energy minimization via graph cuts many tasks in computer vision involve assigning a label such as disparity to every pixel a common constraint is that the labels should vary smoothly almost everywhere while preserving sharp discontinuities that may exist e g at object boundaries these tasks are naturally stated in terms of energy minimization the authors consider a wide class of energies with various smoothness constraints global minimization of these energy functions is np hard even in the simplest discontinuity preserving case therefore our focus is on efficient approximation algorithms we present two algorithms based on graph cuts that efficiently find a local minimum with respect to two types of large moves namely expansion moves and swap moves these moves can simultaneously change the labels of arbitrarily large sets of pixels in contrast many standard algorithms including simulated annealing use small moves where only one pixel changes its label at a time our expansion algorithm finds a labeling within a known factor of the global minimum while our swap algorithm handles more general energy functions both of these algorithms allow important cases of discontinuity preserving energies we experimentally demonstrate the effectiveness of our approach for image restoration stereo and motion on real data with ground truth we achieve 98 percent accuracy

quadratic convex reformulations for semicontinuous quadratic programming we consider in this paper a class of semicontinuous quadratic programming problems which arises in many real world applications such as production planning portfolio selection and subset selection in regression we build upon the idea of the quadratic convex reformulation approach i e adding to the original objective function an additional convex term that is zero at all points of the feasible region exploiting the structure of semicontinuous variables we derive the most general set of quadratic functions that can be added we also incorporate the state of the art perspective reformulation into our new framework within this framework the set of optimal reformulations with the tightest continuous relaxation can be found by solving a semidefinite programming problem we also explore the relationship among different reformulations within our framework and simplify the search for optimal ones in certain cases with the help of second order cone programming among the proposed set of optimal reformula

how to generate ordered maps by maximizing the mutual information between input and output signals a learning rule that performs gradient ascent in the average mutual information between input and an output signal is derived for a system having feedforward and lateral interactions several processes emerge as components of this learning rule hebb like modification and cooperation and competition among processing nodes topographic map formation is demonstrated using the learning rule an analytic expression relating the average mutual information to the response properties of nodes and their geometric arrangement is derived in certain cases this yields a relation between the local map magnification factor and the probability distribution in the input space the results provide new links between unsupervised learning and information theoretic optimization in a system whose properties are biologically motivated

mixed integer second order cone programming formulations for variable selection in linear regression abstract this study concerns a method of selecting the best subset of explanatory variables in a multiple linear regression model goodness of fit measures for example adjusted r2 aic and bic are generally used to evaluate a subset regression model although variable selection with regard to these measures is usually performed with a stepwise regression method it does not always provide the best subset of explanatory variables in this paper we propose mixed integer second order cone programming formulations for selecting the best subset of variables with respect to adjusted r2 aic and bic computational experiments show that in terms of these measures the proposed formulations yield better solutions than those provided by common stepwise regression methods

differential log likelihood for evaluating and learning gaussian mixtures we introduce a new unbiased metric for assessing the quality of density estimation based on gaussian mixtures called differential log likelihood as an application we determine the optimal smoothness and the optimal number of kernels in gaussian mixtures furthermore we suggest a learning strategy for gaussian mixture density estimation and compare its performance with log likelihood maximization for a wide range of real world data sets

lifting for conic mixed integer programming lifting is a procedure for deriving valid inequalities for mixed integer sets from valid inequalities for suitable restrictions of those sets lifting has been shown to be very effective in developing strong valid inequalities for linear integer programming and it has been successfully used to solve such problems with branch and cut algorithms here we generalize the theory of lifting to conic integer programming i e integer programs with conic constraints we show how to derive conic valid inequalities for a conic integer program from conic inequalities valid for its lower dimensional restrictions in order to simplify the computations we also discuss sequence independent lifting for conic integer programs when the cones are restricted to nonnegative orthants conic lifting reduces to the lifting for linear integer programming as one may expect

parameter estimation and tissue segmentation from multispectral mr images a statistical method is developed to classify tissue types and to segment the corresponding tissue regions from relaxation time t sub 1 t sub 2 and proton density p sub d weighted magnetic resonance images the method assumes that the distribution of image intensities associated with each tissue type can be expressed as a multivariate likelihood function of three weighted signal intensity values t sub 1 t sub 2 p sub d at each location within that tissue regions the method further assumes that the underlying tissue regions are piecewise contiguous and can be characterized by a markov random field prior in classifying the tissue types the method models the likelihood of realizing the images as a finite multivariate mixture function the class parameters associated with the tissue types i e the weighted intensity means variances and correlation coefficients of the multivariate function as well as the number of voxels within regions of the tissue types of are estimated by maximum likelihood the estimation fits the class parameters to the image data via the expectation maximization algorithm the number of classes associated with the tissue types is determined by the information criterion of minimum description length the method segments the tissue regions given the estimated class parameters by maximum a posteriori probability the prior is constructed by the tissue region membership of the first and second order neighborhood the method is tested by a few sets of t sub 1 t sub 2 and p sub d weighted images of the brain acquired with a 1 5 tesla whole body scanner the number of classes and the associated class parameters are automatically estimated the regions of different brain tissues are satisfactorily segmented

a conic integer programming approach to stochastic joint location inventory problems we study several joint facility location and inventory management problems with stochastic retailer demand in particular we consider cases with uncapacitated facilities capacitated facilities correlated retailer demand stochastic lead times and multicommodities we show how to formulate these problems as conic quadratic mixed integer problems valid inequalities including extended polymatroid and extended cover cuts are added to strengthen the formulations and improve the computational results compared to the existing modeling and solution methods the new conic integer programming approach not only provides a more general modeling framework but also leads to fast solution times in general

semiparametric maximum likelihood for measurement error model regression this paper presents an em algorithm for semiparametric likelihood analysis of linear generalized linear and nonlinear regression models with measurement errors in explanatory variables a structural model is used in which probability distributions are specified for a the response and b the measurement error a distribution is also assumed for the true explanatory variable but is left unspecified and is estimated by nonparametric maximum likelihood for various types of extra information about the measurement error distribution the proposed algorithm makes use of available routines that would be appropriate for likelihood analysis of a and b if the true x were available simulations suggest that the semiparametric maximum likelihood estimator retains a high degree of efficiency relative to the structural maximum likelihood estimator based on correct distributional assumptions and can outperform maximum likelihood based on an incorrect distributional assumption the approach is illustrated on three examples with a variety of structures and types of extra information about the measurement error distribution

a hybrid sem algorithm for high dimensional unsupervised learning using a finite generalized dirichlet mixture this paper applies a robust statistical scheme to the problem of unsupervised learning of high dimensional data we develop analyze and apply a new finite mixture model based on a generalization of the dirichlet distribution the generalized dirichlet distribution has a more general covariance structure than the dirichlet distribution and offers high flexibility and ease of use for the approximation of both symmetric and asymmetric distributions we show that the mathematical properties of this distribution allow high dimensional modeling without requiring dimensionality reduction and thus without a loss of information this makes the generalized dirichlet distribution more practical and useful we propose a hybrid stochastic expectation maximization algorithm hsem to estimate the parameters of the generalized dirichlet mixture the algorithm is called stochastic because it contains a step in which the data elements are assigned randomly to components in order to avoid convergence to a saddle point the adjective hybrid is justified by the introduction of a newton raphson step moreover the hsem algorithm autonomously selects the number of components by the introduction of an agglomerative term the performance of our method is tested by the classification of several pattern recognition data sets the generalized dirichlet mixture is also applied to the problems of image restoration image object recognition and texture image database summarization for efficient retrieval for the texture image summarization problem results are reported for the vistex texture image database from the mit media lab

sar amplitude probability density function estimation based on a generalized gaussian model in the context of remotely sensed data analysis an important problem is the development of accurate models for the statistics of the pixel intensities focusing on synthetic aperture radar sar data this modeling process turns out to be a crucial task for instance for classification or for denoising purposes in this paper an innovative parametric estimation methodology for sar amplitude data is proposed that adopts a generalized gaussian gg model for the complex sar backscattered signal a closed form expression for the corresponding amplitude probability density function pdf is derived and a specific parameter estimation algorithm is developed in order to deal with the proposed model specifically the recently proposed method of log cumulants molc is applied which stems from the adoption of the mellin transform instead of the usual fourier transform in the computation of characteristic functions and from the corresponding generalization of the concepts of moment and cumulant for the developed gg based amplitude model the resulting molc estimates turn out to be numerically feasible and are also analytically proved to be consistent the proposed parametric approach was validated by using several real ers 1 xsar e sar and nasa jpl airborne sar images and the experimental results prove that the method models the amplitude pdf better than several previously proposed parametric models for backscattering phenomena

cuts for mixed 0 1 conic programming in this we paper we study techniques for generating valid convex constraints for mixed 0 1 conic programs we show that many of the techniques developed for generating linear cuts for mixed 0 1 linear programs such as the gomory cuts the lift and project cuts and cuts from other hierarchies of tighter relaxations extend in a straightforward manner to mixed 0 1 conic programs we also show that simple extensions of these techniques lead to methods for generating convex quadratic cuts gomory cuts for mixed 0 1 conic programs have interesting implications for comparing the semidefinite programming and the linear programming relaxations of combinatorial optimization problems e g we show that all the subtour elimination inequalities for the traveling salesman problem are rank 1 gomory cuts with respect to a single semidefinite constraint we also include results from our preliminary computational experiments with these cuts

self organizing mixture networks for probability density estimation a self organizing mixture network somn is derived for learning arbitrary density functions the network minimizes the kullback leibler information metric by means of stochastic approximation methods the density functions are modeled as mixtures of parametric distributions a mixture needs not to be homogenous i e it can have different density profiles the first layer of the network is similar to kohonen s self organizing map som but with the parameters of the component densities as the learning weights the winning mechanism is based on maximum posterior probability and updating of the weights is limited to a small neighborhood around the winner the second layer accumulates the responses of these local nodes weighted by the learned mixing parameters the network possesses a simple structure and computational form yet yields fast and robust convergence the network has a generalization ability due to the relative entropy criterion used applications to density profile estimation and pattern classification are presented the somn can also provide an insight to the role of neighborhood function used in the som

computational study of a family of mixed integer quadratic programming problems we present computational experience with a branch and cut algorithm to solve quadratic programming problems where there is an upper bound on the number of positive variables such problems arise in financial applications the algorithm solves the largest real life problems in a few minutes of run time

positive definiteness of paired symmetric tensors and elasticity tensors abstract in this paper we consider higher order paired symmetric tensors and strongly paired symmetric tensors elasticity tensors and higher order elasticity tensors in solid mechanics are strongly paired symmetric tensors a strongly paired symmetric tensor is said to be positive definite if the homogeneous polynomial defined by it is positive definite positive definiteness of elasticity and higher order elasticity tensors is strong ellipticity in solid mechanics which plays an important role in nonlinear elasticity theory we mainly investigate positive definiteness of fourth order three dimensional and sixth order three dimensional strongly paired symmetric tensors we first show that the concerned strongly paired symmetric tensor is positive definite if and only if its smallest m eigenvalue is positive second we propose several necessary and sufficient conditions under which the concerned strongly paired symmetric tensor is positive definite third we study the conditions under which the homogeneous polynomial defined by a fourth order three dimensional or sixth order three dimensional strongly paired symmetric tensor can be written as a sum of squares of polynomials and further propose several necessary and or sufficient conditions to judge whether the concerned strongly paired symmetric tensors are positive definite or not fourth by using semidefinite relaxation we propose a sequential semidefinite programming method to compute the smallest m eigenvalue of a fourth order three dimensional strongly paired symmetric tensor by which we can check positive definiteness of the concerned tensor the preliminary numerical results confirm our theoretical findings

some results on the strength of relaxations of multilinear functions we study approaches for obtaining convex relaxations of global optimization problems containing multilinear functions specifically we compare the concave and convex envelopes of these functions with the relaxations that are obtained with a standard relaxation approach due to mccormick the standard approach reformulates the problem to contain only bilinear terms and then relaxes each term independently we show that for a multilinear function having a single product term this approach yields the convex and concave envelopes if the bounds on all variables are symmetric around zero we then review and extend some results on conditions when the concave envelope of a multilinear function can be written as a sum of concave envelopes of its individual terms finally for bilinear functions we prove that the difference between the concave upper bounding and convex lower bounding functions obtained from the mccormick relaxation approach is always within a constant of the difference between the concave and convex envelopes these results along with numerical examples we provide give insight into how to construct strong relaxations of multilinear functions

a bayesian approach to finite mixture models in bioassay via data augmentation and gibbs sampling and its application to insecticide resistance after continued treatment with an insecticide within the population of the susceptible insects resistant strains will occur it is important to know whether there are any resistant strains what the proportions are and what the median lethal doses are for the insecticide lwin and martin 1989 biometrics 45 721 732 propose a probit mixture model and use the em algorithm to obtain the maximum likelihood estimates for the parameters this approach has difficulties in estimating the confidence intervals and in testing the number of components we propose a bayesian approach to obtaining the credible intervals for the location and scale of the tolerances in each component and for the mixture proportions by using data augmentation and gibbs sampler we use bayes factor for model selection and determining the number of components we illustrate the method with data published in lwin and martin 1989

edgeworth expanded gaussian mixture density modeling instead of increasing the order of the edgeworth expansion of a single gaussian kernel we suggest using mixtures of edgeworth expanded gaussian kernels of moderate order we introduce a simple closed form solution for estimating the kernel parameters based on weighted moment matching furthermore we formulate the extension to the multivariate case which is not always feasible with algebraic density approximation procedures

convex programming for disjunctive convex optimization given a finite number of closed convex sets whose algebraic representation is known we study the problem of finding the minimum of a convex function on the closure of the convex hull of the union of those sets we derive an algebraic characterization of the feasible region in a higher dimensional space and propose a solution procedure akin to the interior point approach for convex programming

perspective reformulations of mixed integer nonlinear programs with indicator variables we study mixed integer nonlinear programs minlp s that are driven by a collection of indicator variables where each indicator variable controls a subset of the decision variables an indicator variable when it is turned off forces some of the decision variables to assume fixed values and when it is turned on forces them to belong to a convex set many practical minlps contain integer variables of this type we first study a mixed integer set defined by a single separable quadratic constraint and a collection of variable upper and lower bound constraints and a convex hull description of this set is derived we then extend this result to produce an explicit characterization of the convex hull of the union of a point and a bounded convex set defined by analytic functions further we show that for many classes of problems the convex hull can be expressed via conic quadratic constraints and thus relaxations can be solved via second order cone programming our work is closely related with the earlier work of ceria and soares math program 86 595 614 1999 as well as recent work by frangioni and gentile math program 106 225 236 2006 and akturk et al oper res lett 37 187 191 2009 finally we apply our results to develop tight formulations of mixed integer nonlinear programs in which the nonlinear functions are separable and convex and in which indicator variables play an important role in particular we present computational results for three applications quadratic facility location network design with congestion and portfolio optimization with buy in thresholds that show the power of the reformulation technique

on copositive matrices and strong ellipticity for isotropic elastic materials on etablit les conditions necessaires et suffisantes pour une forte ellipticite des equations d un materiau elastique non lineaire isotrope compressible a l equilibre on utilise un theoreme de representation de matrices copositives du a cottle habetler et lemke 1970

decompositions of semidefinite matrices and the perspective reformulation of nonseparable quadratic programs we study the problem of decomposing the hessian matrix of a mixed integer convex quadratic program micqp into the sum of positive semidefinite 2 2 matrices solving this problem enables the use

a branch and cut method for 0 1 mixed convex programming we generalize the disjunctive approach of balas ceria and cornuejols 2 and devevlop a branch and cut method for solving 0 1 convex programming problems we show that cuts can be generated by solving a single convex program we show how to construct regions similar to those of sherali and adams 20 and lovasz and schrijver 12 for the convex case finally we give some preliminary computational results for our method

maximum likelihood topographic map formation we introduce a new unsupervised learning algorithm for kernel based topographic map formation of heteroscedastic gaussian mixtures that allows for a unified account of distortion error vector quantization log likelihood and kullback leibler divergence

maximizing a class of utility functions over the vertices of a polytope given a polytope x a monotone concave univariate function g and two vectors c and d we study the discrete optimization problem of finding a vertex of x that maximizes the utility function c x g d x this problem has numerous applications in combinatorial optimization with a probabilistic objective including estimation of project duration with stochastic times in reliability models in multinomial logit models and in robust optimization we show that the problem is x1d4a9 x1d4ab hard for any strictly concave function g even for simple polytopes such as the uniform matroid assignment and path polytopes and propose a 1 2 approximation algorithm for it we discuss improvements for special cases where g is the square root log utility negative exponential utility and multinomial logit probability function in particular for the square root function the approximation ratio is 4 5 we also propose a 1 25 approximation algorithm for a class of minimization problems in which the maximization of the utility func

joint entropy maximization in kernel based topographic maps a new learning algorithm for kernel based topographic map formation is introduced the kernel parameters are adjusted individually so as to maximize the joint entropy of the kernel outputs this is done by maximizing the differential entropies of the individual kernel outputs given that the map s output redundancy due to the kernel overlap needs to be minimized the latter is achieved by minimizing the mutual information between the kernel outputs as a kernel the radial incomplete gamma distribution is taken since for a gaussian input density the differential entropy of the kernel output will be maximal since the theoretically optimal joint entropy performance can be derived for the case of nonoverlapping gaussian mixture densities a new clustering algorithm is suggested that uses this optimum as its null distribution finally it is shown that the learning algorithm is similar to one that performs stochastic gradient descent on the kullback leibler divergence for a heteroskedastic gaussian mixture density model

a faster strongly polynomial time algorithm for submodular function minimization we consider the problem of minimizing a submodular function f defined on a set v with n elements we give a combinatorial algorithm that runs in o n 5eo n 6 time where eo is the time to evaluate f s for some s subseteq v this improves the previous best strongly polynomial running time by more than a factor of n we also extend our result to ring families

lifting inequalities a framework for generating strong cuts for nonlinear programs in this paper we introduce the first generic lifting techniques for deriving strong globally valid cuts for nonlinear programs the theory is geometric and provides insights into lifting based cut generation procedures yielding short proofs of earlier results in mixed integer programming using convex extensions we obtain conditions that allow for sequence independent lifting in nonlinear settings paving a way for efficient cut generation procedures for nonlinear programs this sequence independent lifting framework also subsumes the superadditive lifting theory that has been used to generate many general purpose strong cuts for integer programs we specialize our lifting results to derive facet defining inequalities for mixed integer bilinear knapsack sets finally we demonstrate the strength of nonlinear lifting by showing that these inequalities cannot be obtained using a single round of traditional integer programming cut generation techniques applied on a tight reformulation of the problem

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

statistical approach to segmentation of single channel cerebral mr images a statistical model is presented that represents the distributions of major tissue classes in single channel magnetic resonance mr cerebral images using the model cerebral images are segmented into gray matter white matter and cerebrospinal fluid csf the model accounts for random noise magnetic field inhomogeneities and biological variations of the tissues intensity measurements are modeled by a finite gaussian mixture smoothness and piecewise contiguous nature of the tissue regions are modeled by a three dimensional 3 d markov random field mrf a segmentation algorithm based on the statistical model approximately finds the maximum a posteriori map estimation of the segmentation and estimates the model parameters from the image data the proposed scheme for segmentation is based on the iterative conditional modes icm algorithm in which measurement model parameters are estimated using local information at each site and the prior model parameters are estimated using the segmentation after each cycle of iterations application of the algorithm to a sample of clinical mr brain scans comparisons of the algorithm with other statistical methods and a validation study with a phantom are presented the algorithm constitutes a significant step toward a complete data driven unsupervised approach to segmentation of mr images in the presence of the random noise and intensity inhomogeneities

conic mixed integer rounding cuts a conic integer program is an integer programming problem with conic constraints many problems in finance engineering statistical learning and probabilistic optimization are modeled using conic constraints here we study mixed integer sets defined by second order conic constraints we introduce general purpose cuts for conic mixed integer programming based on polyhedral conic substructures of second order conic sets these cuts can be readily incorporated in branch and bound algorithms that solve either second order conic programming or linear programming relaxations of conic integer programs at the nodes of the branch and bound tree central to our approach is a reformulation of the second order conic constraints with polyhedral second order conic constraints in a higher dimensional space in this representation the cuts we develop are linear even though they are nonlinear in the original space of variables this feature leads to a computationally efficient implementation of nonlinear cuts for conic mixed integer programming the reformulation also allows the use of polyhedral methods for conic integer programming we report computational results on solving unstructured second order conic mixed integer problems as well as mean variance capital budgeting problems and least squares estimation problems with binary inputs our computational experiments show that conic mixed integer rounding cuts are very effective in reducing the integrality gap of continuous relaxations of conic mixed integer programs and hence improving their solvability

an incremental nested partition method for data clustering clustering methods are a powerful tool for discovering patterns in a given data set through an organization of data into subsets of objects that share common features motivated by the independent use of some different partitions criteria and the theoretical and empirical analysis of some of its properties in this paper we introduce an incremental nested partition method which combines these partitions criteria for finding the inner structure of static and dynamic datasets for this we proved that there are relationships of nesting between partitions obtained respectively from these partition criteria and besides that the sensitivity when a new object arrives to the dataset is rigorously studied our algorithm exploits all of these mathematical properties for obtaining the hierarchy of clusterings moreover we realize a theoretical and experimental comparative study of our method with classical hierarchical clustering methods such as single link and complete link and other more recently introduced methods the experimental results over databases of uci repository and the afp and tdt2 news collections show the usefulness and capability of our method to reveal different levels of information hidden in datasets

perspective cuts for a class of convex 0 1 mixed integer programs we show that the convex envelope of the objective function of mixed integer programming problems with a specific structure is the perspective function of the continuous part of the objective function using a characterization of the subdifferential of the perspective function we derive perspective cuts a family of valid inequalities for the problem perspective cuts can be shown to belong to the general family of disjunctive cuts but they do not require the solution of a potentially costly nonlinear programming problem to be separated using perspective cuts substantially improves the performance of branch cut approaches for at least two models that either naturally or after a proper reformulation have the required structure the unit commitment problem in electrical power production and the mean variance problem in portfolio optimization