on the combinatorics of locally repairable codes via matroid theory this paper provides a link between matroid theory and locally repairable codes lrcs that are either linear or more generally almost affine using this link new results on both lrcs and matroid theory are derived the parameters n k d r delta of lrcs are generalized to matroids and the matroid analog of the generalized singleton bound by gopalan et al for linear lrcs is given for matroids it is shown that the given bound is not tight for certain classes of parameters implying a nonexistence result for the corresponding locally repairable almost affine codes that are coined perfect in this paper constructions of classes of matroids with a large span of the parameters n k d r delta and the corresponding local repair sets are given using these matroid constructions new lrcs are constructed with prescribed parameters the existence results on linear lrcs and the nonexistence results on almost affine lrcs given in this paper strengthen the nonexistence and existence results on perfect linear lrcs given by song et al

predictive updating methods with application to bayesian classification we propose algorithms based on random draws from predictive distributions of unknown quantities missing values for instance this procedure can either be iterative which is a special variation of the gibbs sampler or be sequential which is a variation of sequential imputation in the latter case one can update the posterior distribution with new observations easily the methods proposed have intuitive statistical implications and can be generalized to accommodate other bayesian like procedures we display some applications of the method in connection with the bayesian bootstrap classification hierarchical models and selection of variables in particular as an application of the method we present a unified treatment of switching regression models driven by a general binary process and we develop a bayesian testing procedure some simulations and a real example are used to illustrate the methods proposed

information theoretic matching of two point sets this paper describes the theoretic roadmap of least relative entropy matching of two point sets the novel feature is to align two point sets without needing to establish explicit point correspondences the recovery of transformational geometry is achieved using a mixture of principal axes registrations whose parameters are estimated by minimizing the relative entropy between the two point distributions and using the expectation maximization algorithm we give evidence of the optimality of the method and we then evaluate the algorithm s performance in both rigid and nonrigid image registration cases

estimating latent distributions in recurrent choice data this paper introduces a flexible class of stochastic mixture models for the analysis and interpretation of individual differences in recurrent choice and other types of count data these choice models are derived by specifying elements of the choice process at the individual level probability distributions are introduced to describe variations in the choice process among individuals and to obtain a representation of the aggregate choice behavior due to the explicit consideration of random effect sources the choice models are parsimonious and readily interpretable an easy to implement em algorithm is presented for parameter estimation two applications illustrate the proposed approach

maximum correntropy criterion for robust face recognition in this paper we present a sparse correntropy framework for computing robust sparse representations of face images for recognition compared with the state of the art l1norm based sparse representation classifier src which assumes that noise also has a sparse representation our sparse algorithm is developed based on the maximum correntropy criterion which is much more insensitive to outliers in order to develop a more tractable and practical approach we in particular impose nonnegativity constraint on the variables in the maximum correntropy criterion and develop a half quadratic optimization technique to approximately maximize the objective function in an alternating way so that the complex optimization problem is reduced to learning a sparse representation through a weighted linear least squares problem with nonnegativity constraint at each iteration our extensive experiments demonstrate that the proposed method is more robust and efficient in dealing with the occlusion and corruption problems in face recognition as compared to the related state of the art methods in particular it shows that the proposed method can improve both recognition accuracy and receiver operator characteristic roc curves while the computational cost is much lower than the src algorithms

adaptive methods for linear programming decoding detectability of failures of linear programming lp decoding and the potential for improvement by adding new constraints motivate the use of an adaptive approach in selecting the constraints for the underlying lp problem in this paper we make a first step in studying this method and show that by starting from a simple lp problem and adaptively adding the necessary constraints the complexity of lp decoding can be significantly reduced in particular we observe that with adaptive lp decoding the sizes of the lp problems that need to be solved become practically independent of the density of the parity check matrix we further show that adaptively adding extra constraints such as constraints based on redundant parity checks can provide large gains in the performance

measure fields for function approximation the computation of a piecewise smooth function that approximates a finite set of data points may be decomposed into two decoupled tasks 1 the computation of the locally smooth models and hence the segmentation of the data into classes that consist of the sets of points best approximated by each model 2 the computation of the normalized discriminant functions for each induced class which maybe interpreted as relative probabilities the approximating function may then be computed as the optimal estimator with respect to this measure field for the first step we propose a scheme that involves both robust regression and spatial localization using gaussian windows the discriminant functions are obtained fitting gaussian mixture models for the data distribution inside each class we give an efficient procedure for effecting both computations and for the determination of the optimal number of components examples of the application of this scheme to image filtering surface reconstruction and time series prediction are presented

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

semi supervised cross entropy clustering with information bottleneck constraint abstract in this paper we propose a semi supervised clustering method cec ib that models data with a set of gaussian distributions and that retrieves clusters based on a partial labeling provided by the user partition level side information by combining the ideas from cross entropy clustering cec with those from the information bottleneck method ib our method trades between three conflicting goals the accuracy with which the data set is modeled the simplicity of the model and the consistency of the clustering with side information experiments demonstrate that cec ib performs similar as gaussian mixture models in a classical semi supervised scenario but is faster more robust to noisy labels automatically determines the optimal number of clusters and performs well when not all classes are present in the side information moreover in contrast to many other semi supervised models it can be successfully applied in discovering natural subgroups if the partition level side information is derived from the top levels of a hierarchical clustering

a matroid framework for noncoherent random network communications models for noncoherent error control in random linear network coding rlnc and store and forward saf have been recently proposed in this paper we model different types of random network communications as the transmission of flats of matroids this novel framework encompasses rlnc and saf and allows us to introduce a novel protocol referred to as random affine network coding ranc based on affine combinations of packets although the models previously proposed for rlnc and saf only consider error control using our framework we first evaluate and compare the performance of different network protocols in the error free case we define and determine the rate average delay and throughput of such protocols and we also investigate the possibilities of partial decoding before the entire message is received we thus show that ranc outperforms rlnc in terms of data rate and throughput thanks to a more efficient encoding of messages into packets second we model the possible alterations of a message by the network as an operator channel which generalizes the channels proposed for rlnc and saf error control is thus reduced to a coding theoretic problem on flats of a matroid where two distinct metrics can be used for error correction we study the maximum cardinality of codes on flats in general and codes for error correction in ranc in particular we finally design a class of nearly optimal codes for ranc based on rank metric codes for which we propose a low complexity decoding algorithm the gain of ranc over rlnc is thus preserved with no additional cost in terms of complexity

maximum likelihood estimation of latent interaction effects with the lms method in the context of structural equation modeling a general interaction model with multiple latent interaction effects is introduced a stochastic analysis represents the nonnormal distribution of the joint indicator vector as a finite mixture of normal distributions the latent moderated structural equations lms approach is a new method developed for the analysis of the general interaction model that utilizes the mixture distribution and provides a ml estimation of model parameters by adapting the em algorithm the finite sample properties and the robustness of lms are discussed finally the applicability of the new method is illustrated by an empirical example

cluster number selection for a small set of samples using the bayesian ying yang model one major problem in cluster analysis is the determination of the number of clusters in this paper we describe both theoretical and experimental results in determining the cluster number for a small set of samples using the bayesian kullback ying yang byy model selection criterion under the second order approximation we derive a new equation for estimating the smoothing parameter in the cost function finally we propose a gradient descent smoothing parameter estimation approach that avoids complicated integration procedure and gives the same optimal result

application of the em algorithm for the multitarget multisensor tracking problem an important problem in surveillance and reconnaissance systems is the tracking of multiple moving targets in cluttered noise environments using outputs from a number of sensors possessing wide variations in individual characteristics and accuracies a number of approaches have been proposed for this multitarget multisensor tracking problem ranging from reasonably simple though ad hoc schemes to fairly complex but theoretically optimum approaches in this paper we describe an iterative procedure for time recursive multitarget multisensor tracking based on use of the expectation maximization em algorithm more specifically we pose the multitarget multisensor tracking problem as an incomplete data problem with the observable sensor outputs representing the incomplete data whereas the target associated sensor outputs constitute the complete data target updates at each time use an em based approach that calculates the maximum a posteriori map estimate of the target states under the assumption of appropriate motion models based on the outputs of disparate sensors the approach uses a markov random field mrf model of the associations between observations and targets and allows for estimation of joint association probabilities without explicit enumeration the advantage of this em based approach is that it provides a computationally efficient means for approaching the performance offered by theoretically optimum approaches that use explicit enumeration of the joint association probabilities we provide selected results illustrating the performance complexity characteristics of this em based approach compared with competing schemes

hierarchical mixtures of experts and the em algorithm we present a tree structured architecture for supervised learning the statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models glim s learning is treated as a maximum likelihood problem in particular we present an expectation maximization em algorithm for adjusting the parameters of the architecture we also develop an on line learning algorithm in which the parameters are updated incrementally comparative simulation results are presented in the robot dynamics domain

feature selection based on the approximation of class densities by finite mixtures of special type abstract a new method of feature selection based on the approximation of class conditional densities by a mixture of parameterized densities of a special type suitable especially for multimodal data is presented no search procedure is needed when using the proposed method its performance is tested both on real simulated data

impact of redundant checks on the lp decoding thresholds of ldpc codes feldman et al 11 asked whether the performance of the linear programming lp decoder can be improved by adding redundant parity checks to tighten the lp relaxation we prove in this paper that for low density parity check codes even if we include all redundant parity checks asymptotically there is no gain in the lp decoder threshold on the binary symmetric channel under certain conditions on the base tanner graph first we show that if the tanner graph has bounded check degree and satisfies a condition which we call asymptotic strength then including high degree redundant parity checks in the lp does not significantly improve the threshold of the lp decoder in the following sense for each constant delta 0 there is a constant k 0 such that the threshold of the lp decoder containing all redundant checks of degree at most k improves by at most delta upon adding to the lp all redundant checks of degree larger than k we conclude that if the graph satisfies an additional condition which we call rigidity then including all redundant checks does not improve the threshold of the base lp we call the graph asymptotically strong if the lp decoder corrects a constant fraction of errors even if the log likelihood ratios of the correct variables are arbitrarily small by building on a construction due feldman et al 9 and its recent improvement by viderman 24 we show that asymptotic strength follows from sufficiently large variable to check expansion we also give a geometric interpretation of asymptotic strength in terms pseudocodewords we call the graph rigid if the minimum weight of a sum of check nodes involving a cycle tends to infinity as the block length tends to infinity under the assumptions that the graph girth is logarithmic and the minimum check degree is at least 3 rigidity is equivalent to the nondegeneracy property that adding at least logarithmically many checks does not give a constant weight check we argue that nondegeneracy is a typical property of random check regular tanner graphs

comments on application of the conditional population mixture model to image segmentation in the above correspondence1 a maximum likelihood method is proposed for estimating class memberships and underlying statistical parameters within the context of distribution mixtures in the present comment it is pointed out that biases are incurred in parameter estimation that the class memberships and parameters are conceptually different and therefore that the so called standard mixture likelihood is to be preferred also in the correspondence 1 akaike s information criterion aic is used to choose the number of classes in the mixture here a brief theoretical caveat is issued

sequential algorithms for parameter estimation based on the kullback leibler information measure methods of stochastic approximation are used to convert iterative algorithms for maximizing the kullback leibler information measure into sequential algorithms special attention is given to the case of incomplete data and several algorithms are presented to deal with situations of this kind the application of these algorithms to the identification of finite impulse response systems is considered

anderson acceleration for fixed point iterations this paper concerns an acceleration method for fixed point iterations that originated in work of d g anderson j assoc comput mach 12 1965 pp 547 560 which we accordingly call anderson acceleration here this method has enjoyed considerable success and wide usage in electronic structure computations where it is known as anderson mixing however it seems to have been untried or underexploited in many other important applications moreover while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities this method has received relatively little attention from these communities over the years a recent paper by h fang and y saad numer linear algebra appl 16 2009 pp 197 221 has clarified a remarkable relationship of anderson acceleration to quasi newton secant updating methods and extended it to define a broader anderson family of acceleration methods in this paper our goals are to shed additional light on anderson acceleration and to draw further attention to its usefulness as a general tool we first show that on linear problems anderson acceleration without truncation is essentially equivalent in a certain sense to the generalized minimal residual gmres method we also show that the type 1 variant in the fang saad anderson family is similarly essentially equivalent to the arnoldi full orthogonalization method we then discuss practical considerations for implementing anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications

assessing a mixture model for clustering with the integrated completed likelihood we propose an assessing method of mixture model in a cluster analysis setting with integrated completed likelihood for this purpose the observed data are assigned to unknown clusters using a maximum a posteriori operator then the integrated completed likelihood icl is approximated using the bayesian information criterion bic numerical experiments on simulated and real data of the resulting icl criterion show that it performs well both for choosing a mixture model and a relevant number of clusters in particular icl appears to be more robust than bic to violation of some of the mixture model assumptions and it can select a number of dusters leading to a sensible partitioning of the data

on minimal tree realizations of linear codes a tree decomposition of the coordinates of a code is a mapping from the coordinate set to the set of vertices of a tree a tree decomposition can be extended to a tree realization i e a cycle free realization of the code on the underlying tree by specifying a state space at each edge of the tree and a local constraint code at each vertex of the tree the constraint complexity of a tree realization is the maximum dimension of any of its local constraint codes a measure of the complexity of maximum likelihood ml decoding for a code is its treewidth which is the least constraint complexity of any of its tree realizations it is known that among all tree realizations of a linear code that extends a given tree decomposition there exists a unique minimal realization that minimizes the state space dimension at each vertex of the underlying tree in this paper we give two new constructions of these minimal realizations as a by product of the first construction a generalization of the state merging procedure for trellis realizations we obtain the fact that the minimal tree realization also minimizes the local constraint code dimension at each vertex of the underlying tree the second construction relies on certain code decomposition techniques that we develop we further observe that the treewidth of a code is related to a measure of graph complexity also called treewidth we exploit this connection to resolve a conjecture of forney s regarding the gap between the minimum trellis constraint complexity and the treewidth of a code we present a family of codes for which this gap can be arbitrarily large

finite mixture varying coefficient models for analyzing longitudinal heterogenous data this paper aims to develop a mixture model to study heterogeneous longitudinal data on the treatment effect of heroin use from a california civil addict program each component of the mixture is characterized by a varying coefficient mixed effect model we use the bayesian p splines approach to approximate the varying coefficient functions we develop markov chain monte carlo algorithms to estimate the smooth functions unknown parameters and latent variables in the model we use modified deviance information criterion to determine the number of components in the mixture a simulation study demonstrates that the modified deviance information criterion selects the correct number of components and the estimation of unknown quantities is accurate we apply the proposed model to the heroin treatment study furthermore we identify heterogeneous longitudinal patterns copyright 2011 john wiley sons ltd

networks matroids and non shannon information inequalities we define a class of networks called matroidal networks which includes as special cases all scalar linearly solvable networks and in particular solvable multicast networks we then present a method for constructing matroidal networks from known matroids we specifically construct networks that play an important role in proving results in the literature such as the insufficiency of linear network coding and the unachievability of network coding capacity we also construct a new network from the vamos matroid which we call the vamos network and use it to prove that shannon type information inequalities are in general not sufficient for computing network coding capacities to accomplish this we obtain a capacity upper bound for the vamos network using a non shannon type information inequality discovered in 1998 by zhang and yeung and then show that it is smaller than any such bound derived from shannon type information inequalities this is the first application of a non shannon type inequality to network coding we also compute the exact routing capacity and linear coding capacity of the vamos network finally using a variation of the vamos network we prove that shannon type information inequalities are insufficient even for computing network coding capacities of multiple unicast networks

using linear programming to decode binary linear codes a new method is given for performing approximate maximum likelihood ml decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel the decoding algorithm is based on a linear programming lp relaxation that is defined by a factor graph or parity check representation of the code the resulting lp decoder generalizes our previous work on turbo like codes a precise combinatorial characterization of when the lp decoder succeeds is provided based on pseudocodewords associated with the factor graph our definition of a pseudocodeword unifies other such notions known for iterative algorithms including stopping sets irreducible closed walks trellis cycles deviation sets and graph covers the fractional distance d sub frac of a code is introduced which is a lower bound on the classical distance it is shown that the efficient lp decoder will correct up to spl lceil d sub frac 2 spl rceil 1 errors and that there are codes with d sub frac spl omega n sup 1 spl epsi an efficient algorithm to compute the fractional distance is presented experimental evidence shows a similar performance on low density parity check ldpc codes between lp decoding and the min sum and sum product algorithms methods for tightening the lp relaxation to improve performance are also provided

on the intractability of permuting a block code to minimize trellis complexity an important problem in the theory and application of block code trellises is to find a coordinate permutation of a given code to minimize the trellis complexity we show that the problem of finding a coordinate permutation that minimizes the number of vertices at a given depth in the minimal trellis for a binary linear block code is np complete

the 2f modules for nearly simple groups abstract we complete the classification of irreducible 2f modules for groups of lie type acting in the natural characteristic by dealing with the three open cases from r m guralnick g malle classification of 2f modules ii in c ho p sin p tiep a turull eds finite groups 2003 proceedings of the gainesville conference on finite groups march 6 12 2003 de gruyter berlin 2004 pp 117 183 we also finish the classification of such modules for almost quasi simple groups and show that in all cases there is an offender with cubic action

classification based adaptive filtering for multiframe blind image restoration in this paper the blind restoration of a scene is investigated when multiple degraded blurred and noisy acquisitions are available an adaptive filtering technique is proposed where the distorted images are filtered classified and then fused based upon the classification decisions finite normal density mixture fnm models are used to model the filtered outputs at each iteration for simplicity fixed number of gaussian components classes is initially considered for each degraded frame and the selection of the optimal number of classes is performed according to the global relative entropy criterion however there exist cases where dynamically varying fnm models should be considered where the optimal number of classes is selected according to the akaike information criterion the iterative application of classification and fusion followed by optimal adaptive filtering converges to a global enhanced representation of the original scene in only a few iterations the proposed restoration method does not require knowledge of the point spread function support size or exact alignment of the acquired frames simulation results on synthetic and real data using both fixed and dynamically varying fnm models demonstrate its efficiency under both noisy and noise free conditions

on convergence properties of the em algorithm for gaussian mixtures we build up the mathematical connection between the expectation maximization em algorithm and gradient based approaches for maximum likelihood learning of finite gaussian mixtures we show that the em step in parameter space is obtained from the gradient via a projection matrix p and we provide an explicit expression for the matrix we then analyze the convergence of em in terms of special properties of p and provide new results analyzing the effect that p has on the likelihood surface based on these mathematical results we present a comparative discussion of the advantages and disadvantages of em and other algorithms for the learning of gaussian mixture models

application of the conditional population mixture model to image segmentation the problem of image segmentation is considered in the context of a mixture of probability distributions the segments fall into classes a probability distribution is associated with each class of segment parametric families of distributions are considered a set of parameter values being associated with each class with each observation is associated an unobservable label indicating from which class the observation arose segmentation algorithms are obtained by applying a method of iterated maximum likelihood to the resulting likelihood function a numerical example is given choice of the number of classes using akaike s information criterion aic for model identification is illustrated

on the complexity of some coding problems corresp it is shown that the problem of finding a codeword with least weight and whose weight is not a multiple of k in a binary linear code belongs to the class of nondeterministic polynomial np hard problems for any k geq 2 some other related problems are shown to belong to the same class these results were motivated by a conjecture due to berlekamp mceliece and van tilborg that the problem of finding the hamming distance of a code is np hard

coset codes ii binary lattices and related codes for pt i see ibid vol 34 no 5 p 1123 51 1988 the family of barnes wall lattices including d sub 4 and e sub 8 of lengths n 2 sup n and their principal sublattices which are useful in constructing coset codes are generated by iteration of a simple construction called the squaring construction the closely related reed muller codes are generated by the same construction the principal properties of these codes and lattices are consequences of the general properties of iterated squaring constructions which also exhibit the interrelationships between codes and lattices of different lengths an extension called the cubing construction generates good codes and lattices of lengths n 3 2 sup n including the golay code and leech lattice with the use of special bases for 8 space another related construction generates the nordstrom robinson code and an analogous 16 dimensional nonlattice packing these constructions are represented by trellis diagrams that display their structure and interrelationships and that lead to efficient maximum likelihood decoding algorithms

dimension length profiles and trellis complexity of linear block codes this semi tutorial paper discusses the connections between the dimension length profile dlp of a linear code which is essentially the same as its generalized hamming weight hierarchy and the complexity of its minimal trellis diagram these connections are close and deep dlp duality is closely related to trellis duality the dlp of a code gives tight bounds on its state and branch complexity profiles under any coordinate ordering these bounds can often be met a maximum distance separable mds code is characterized by a certain extremal dlp from which the main properties of mds codes are easily derived the simplicity and generality of these interrelationships are emphasized

face recognition a convolutional neural network approach we present a hybrid neural network for human face recognition which compares favourably with other methods the system combines local image sampling a self organizing map som neural network and a convolutional neural network the som provides a quantization of the image samples into a topological space where inputs that are nearby in the original space are also nearby in the output space thereby providing dimensionality reduction and invariance to minor changes in the image sample and the convolutional neural network provides partial invariance to translation rotation scale and deformation the convolutional network extracts successively larger features in a hierarchical set of layers we present results using the karhunen loeve transform in place of the som and a multilayer perceptron mlp in place of the convolutional network for comparison we use a database of 400 images of 40 individuals which contains quite a high degree of variability in expression pose and facial details we analyze the computational complexity and discuss how new classes could be added to the trained recognizer

on the pseudocodeword redundancy of binary linear codes for a binary linear code the pseudocodeword redundancy with respect to the additive white gaussian noise channel the binary symmetric channel or the max fractional weight is defined to be the smallest number of rows in a parity check matrix such that the corresponding minimum pseudoweight is equal to the minimum hamming distance of the code it is shown that most codes do not have a finite pseudocodeword redundancy also upper bounds on the pseudocodeword redundancy for some families of codes including codes based on designs are provided the pseudocodeword redundancies for all codes of small length at most 9 are computed furthermore comprehensive results are provided on the cases of cyclic codes of length at most 250 for which the eigenvalue bound of vontobel and koetter is sharp

optimal locally repairable codes and connections to matroid theory petabyte scale distributed storage systems are currently transitioning to erasure codes to achieve higher storage efficiency classical codes such as reed solomon rs are highly sub optimal for distributed environments due to their high overhead during single failure events locally repairable codes lrcs form a new family of codes that are repair efficient in particular lrcs minimize the number of nodes participating in single node repairs fundamental bounds and methods for explicitly constructing lrcs suitable for deployment in distributed storage clusters are not fully understood and currently form an active area of research in this paper we present an explicit lrc that is simple to construct and is optimal for a specific set of coding parameters our construction is based on grouping rs symbols and then adding extra simple parities that allow for small repair locality for the analysis of the optimality of the code we derive a new result on the matroid represented by the code s generator matrix

classification of 2f modules i abstract we classify the 2f modules for nearly simple groups excluding the case of modules for groups of lie type in their defining characteristic we also show that for all such modules there exists an offender with cubic action these modules play a crucial role in the current revisions of the classification of the finite simple groups

semiparametric maximum likelihood for measurement error model regression this paper presents an em algorithm for semiparametric likelihood analysis of linear generalized linear and nonlinear regression models with measurement errors in explanatory variables a structural model is used in which probability distributions are specified for a the response and b the measurement error a distribution is also assumed for the true explanatory variable but is left unspecified and is estimated by nonparametric maximum likelihood for various types of extra information about the measurement error distribution the proposed algorithm makes use of available routines that would be appropriate for likelihood analysis of a and b if the true x were available simulations suggest that the semiparametric maximum likelihood estimator retains a high degree of efficiency relative to the structural maximum likelihood estimator based on correct distributional assumptions and can outperform maximum likelihood based on an incorrect distributional assumption the approach is illustrated on three examples with a variety of structures and types of extra information about the measurement error distribution

a hybrid sem algorithm for high dimensional unsupervised learning using a finite generalized dirichlet mixture this paper applies a robust statistical scheme to the problem of unsupervised learning of high dimensional data we develop analyze and apply a new finite mixture model based on a generalization of the dirichlet distribution the generalized dirichlet distribution has a more general covariance structure than the dirichlet distribution and offers high flexibility and ease of use for the approximation of both symmetric and asymmetric distributions we show that the mathematical properties of this distribution allow high dimensional modeling without requiring dimensionality reduction and thus without a loss of information this makes the generalized dirichlet distribution more practical and useful we propose a hybrid stochastic expectation maximization algorithm hsem to estimate the parameters of the generalized dirichlet mixture the algorithm is called stochastic because it contains a step in which the data elements are assigned randomly to components in order to avoid convergence to a saddle point the adjective hybrid is justified by the introduction of a newton raphson step moreover the hsem algorithm autonomously selects the number of components by the introduction of an agglomerative term the performance of our method is tested by the classification of several pattern recognition data sets the generalized dirichlet mixture is also applied to the problems of image restoration image object recognition and texture image database summarization for efficient retrieval for the texture image summarization problem results are reported for the vistex texture image database from the mit media lab

sar amplitude probability density function estimation based on a generalized gaussian model in the context of remotely sensed data analysis an important problem is the development of accurate models for the statistics of the pixel intensities focusing on synthetic aperture radar sar data this modeling process turns out to be a crucial task for instance for classification or for denoising purposes in this paper an innovative parametric estimation methodology for sar amplitude data is proposed that adopts a generalized gaussian gg model for the complex sar backscattered signal a closed form expression for the corresponding amplitude probability density function pdf is derived and a specific parameter estimation algorithm is developed in order to deal with the proposed model specifically the recently proposed method of log cumulants molc is applied which stems from the adoption of the mellin transform instead of the usual fourier transform in the computation of characteristic functions and from the corresponding generalization of the concepts of moment and cumulant for the developed gg based amplitude model the resulting molc estimates turn out to be numerically feasible and are also analytically proved to be consistent the proposed parametric approach was validated by using several real ers 1 xsar e sar and nasa jpl airborne sar images and the experimental results prove that the method models the amplitude pdf better than several previously proposed parametric models for backscattering phenomena

2f modules abelian sets of roots and 2 ranks abstract we complete the classification of the irreducible 2f modules for quasi simple groups by dealing with the two open cases for sporadic groups from r m guralnick g malle classification of 2f modules i j algebra 257 2002 348 372 the techniques developed involve abelian sets of roots within root systems and enable us to determine the 2 ranks of the monster the baby monster and the double cover of the latter

parameter estimation and tissue segmentation from multispectral mr images a statistical method is developed to classify tissue types and to segment the corresponding tissue regions from relaxation time t sub 1 t sub 2 and proton density p sub d weighted magnetic resonance images the method assumes that the distribution of image intensities associated with each tissue type can be expressed as a multivariate likelihood function of three weighted signal intensity values t sub 1 t sub 2 p sub d at each location within that tissue regions the method further assumes that the underlying tissue regions are piecewise contiguous and can be characterized by a markov random field prior in classifying the tissue types the method models the likelihood of realizing the images as a finite multivariate mixture function the class parameters associated with the tissue types i e the weighted intensity means variances and correlation coefficients of the multivariate function as well as the number of voxels within regions of the tissue types of are estimated by maximum likelihood the estimation fits the class parameters to the image data via the expectation maximization algorithm the number of classes associated with the tissue types is determined by the information criterion of minimum description length the method segments the tissue regions given the estimated class parameters by maximum a posteriori probability the prior is constructed by the tissue region membership of the first and second order neighborhood the method is tested by a few sets of t sub 1 t sub 2 and p sub d weighted images of the brain acquired with a 1 5 tesla whole body scanner the number of classes and the associated class parameters are automatically estimated the regions of different brain tissues are satisfactorily segmented

codes on graphs constraint complexity of cycle free realizations of linear codes cycle free graphical realizations of linear codes generalize trellis realizations given a linear code c and a cycle free graph topology there exists a well defined minimal realization for c on that graph in which each constraint is a linear code with a well defined length and dimension the constraint complexity of the realization is defined as maximum dimension of any constraint code there exists a graph that minimizes constraint complexity in which all internal nodes have degree 3 and all interface nodes have degree 2 and which moreover can be put in the form of a tree structured trellis realization the constraint complexity of a general cycle free graph realization can be less than that of any conventional trellis realization but not by very much such realizations can yield reductions in decoding complexity even when they do not reduce constraint complexity

the dynamics of group codes state spaces trellis diagrams and canonical encoders a group code c over a group g is a set of sequences of group elements that itself forms a group under a component wise group operation a group code has a well defined state space sigma sub k at each time k each code sequence passes through a well defined state sequence the set of all state sequences is also a group code the state code of c the state code defines an essentially unique minimal realization of c the trellis diagram of c is defined by the state code of c and by labels associated with each state transition the set of all label sequences forms a group code the label code of c which is isomorphic to the state code of c if c is complete and strongly controllable then a minimal encoder in controller canonical feedbackfree form may be constructed from certain sets of shortest possible code sequences called granules the size of the state space sigma sub k is equal to the size of the state space of this canonical encoder which is given by a decomposition of the input groups of c at each time k if c is time invariant and nu controllable then mod sigma sub k mod pi sub 1

a bayesian approach to finite mixture models in bioassay via data augmentation and gibbs sampling and its application to insecticide resistance after continued treatment with an insecticide within the population of the susceptible insects resistant strains will occur it is important to know whether there are any resistant strains what the proportions are and what the median lethal doses are for the insecticide lwin and martin 1989 biometrics 45 721 732 propose a probit mixture model and use the em algorithm to obtain the maximum likelihood estimates for the parameters this approach has difficulties in estimating the confidence intervals and in testing the number of components we propose a bayesian approach to obtaining the credible intervals for the location and scale of the tolerances in each component and for the mixture proportions by using data augmentation and gibbs sampler we use bayes factor for model selection and determining the number of components we illustrate the method with data published in lwin and martin 1989

edgeworth expanded gaussian mixture density modeling instead of increasing the order of the edgeworth expansion of a single gaussian kernel we suggest using mixtures of edgeworth expanded gaussian kernels of moderate order we introduce a simple closed form solution for estimating the kernel parameters based on weighted moment matching furthermore we formulate the extension to the multivariate case which is not always feasible with algebraic density approximation procedures

adaptive mixture density estimation abstract a recursive nonparametric method is developed for performing density estimation derived from mixture models kernel estimation and stochastic approximation the asymptotic performance of the method dubbed adaptive mixtures priebe and marchette pattern recognition24 1197 1209 1991 for its data driven development of a mixture model approximation to the true density is investigated using the method of sieves simulations are included indicating convergence properties for some simple examples

on the existence of asymptotically good linear codes in minor closed classes let c c 1 c 2 be a sequence of codes such that each c i is a linear n i k i d i code over some fixed finite field f where n i is the length of the code words k i is the dimension and d i is the minimum distance we say that c is asymptotically good if for some e 0 and for all i 0 we have n i i and min k i n i d i n i e sequences of asymptotically good codes exist we prove that if c is a class of gf p n linear codes where p is prime and n 1 closed under puncturing and shortening and if c contains an asymptotically good sequence then c must contain all gf p linear codes our proof relies on a powerful new result from matroid structure theory

the extraction and complexity limits of graphical models for linear codes two broad classes of graphical modeling problems for codes can be identified in the literature constructive and extractive problems the former class of problems concern the construction of a graphical model in order to define a new code the latter class of problems concern the extraction of a graphical model for a fixed given code the design of a new low density parity check code for some given criteria e g target block length and code rate is an example of a constructive problem the determination of a graphical model for a classical linear block code that implies a decoding algorithm with desired performance and complexity characteristics is an example of an extractive problem this work focuses on extractive graphical model problems and aims to lay out some of the foundations of the theory of such problems for linear codes the primary focus of this work is a study of the space of all graphical models for a fixed given code the tradeoff between cyclic topology and complexity in this space is characterized via the introduction of a new bound the forest inducing cut set bound fi csb the proposed bound provides a more precise characterization of this tradeoff than that which can be obtained using existing tools e g the csb and can be viewed as a generalization of the square root bound for tail biting trellises to graphical models with arbitrary cyclic topologies searching the space of graphical models for a given code is then enabled by introducing a set of basic graphical model transformation operations that are shown to span this space finally heuristics for extracting novel graphical models for linear block codes using these transformations are investigated

maximum likelihood topographic map formation we introduce a new unsupervised learning algorithm for kernel based topographic map formation of heteroscedastic gaussian mixtures that allows for a unified account of distortion error vector quantization log likelihood and kullback leibler divergence

mathematical programming decoding of binary linear codes theory and algorithms mathematical programming is a branch of applied mathematics and has recently been used to derive new decoding approaches challenging established but often heuristic algorithms based on iterative message passing concepts from mathematical programming used in the context of decoding include linear integer and nonlinear programming network flows notions of duality as well as matroid and polyhedral theory this paper reviews and categorizes decoding methods based on mathematical programming approaches for binary linear codes over binary input memoryless symmetric channels

quantification of mr brain images by mixture density and partial volume modeling the problem of automatic quantification of brain tissue by utilizing single valued single echo magnetic resonance imaging mri brain scans is addressed it is shown that this problem can be solved without classification or segmentation a method that may be particularly useful in quantifying white matter lesions where the range of values associated with the lesions and the white matter may heavily overlap the general technique utilizes a statistical model of the noise and partial volume effect together with a finite mixture density description of the tissues the quantification is then formulated as a minimization problem of high order with up to six separate densities as part of the mixture this problem is solved by tree annealing with and without partial volume utilized the results compared and the sensitivity of the tree annealing algorithm to various parameters is exhibited the actual quantification is performed by two methods a classification based method called bayes quantification and parameter estimation results from each method are presented for synthetic and actual data

the intractability of computing the minimum distance of a code it is shown that the problem of computing the minimum distance of a binary linear code is np hard and the corresponding decision problem is np complete this result constitutes a proof of the conjecture of berlekamp mceliece and van tilborg 1978 extensions and applications of this result to other problems in coding theory are discussed

dense error correction via ell 1 minimization this paper studies the problem of recovering a sparse signal x n from highly corrupted linear measurements y ax e m where e is an unknown error vector whose nonzero entries may be unbounded motivated by an observation from face recognition in computer vision this paper proves that for highly correlated and possibly overcomplete dictionaries a any sufficiently sparse signal x can be recovered by solving an l1 minimization problem min x 1 e 1 subject to y ax e more precisely if the fraction of the support of the error e is bounded away from one and the support of a is a very small fraction of the dimension m then as m becomes large the above l1 minimization succeeds for all signals x and almost all sign and support patterns of e this result suggests that accurate recovery of sparse signals is possible and computationally feasible even with nearly 100 of the observations corrupted the proof relies on a careful characterization of the faces of a convex polytope spanned together by the standard crosspolytope and a set of independent identically distributed i i d gaussian vectors with nonzero mean and small variance dubbed the cross and bouquet cab model simulations and experiments corroborate the findings and suggest extensions to the result

f stability in finite groups let g be a finite group s syl p g and s be the set subgroups containing s for m s and v 1 z op m the paper discusses the action of m on v apart from other results it is shown that for groups of parabolic characteristic p either s is contained in a unique maximal p local subgroup or there exists a maximal p local subgroup in m s such that v is a nearly quadratic 2f module for m

statistical approach to segmentation of single channel cerebral mr images a statistical model is presented that represents the distributions of major tissue classes in single channel magnetic resonance mr cerebral images using the model cerebral images are segmented into gray matter white matter and cerebrospinal fluid csf the model accounts for random noise magnetic field inhomogeneities and biological variations of the tissues intensity measurements are modeled by a finite gaussian mixture smoothness and piecewise contiguous nature of the tissue regions are modeled by a three dimensional 3 d markov random field mrf a segmentation algorithm based on the statistical model approximately finds the maximum a posteriori map estimation of the segmentation and estimates the model parameters from the image data the proposed scheme for segmentation is based on the iterative conditional modes icm algorithm in which measurement model parameters are estimated using local information at each site and the prior model parameters are estimated using the segmentation after each cycle of iterations application of the algorithm to a sample of clinical mr brain scans comparisons of the algorithm with other statistical methods and a validation study with a phantom are presented the algorithm constitutes a significant step toward a complete data driven unsupervised approach to segmentation of mr images in the presence of the random noise and intensity inhomogeneities

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

a generalisation of a theorem of wielandt abstract in 1974 helmut wielandt proved that in a finite group g a subgroup a is subnormal if and only if it is subnormal in a g for all g g in this paper we prove that the subnormality of an odd order nilpotent subgroup a of g is already guaranteed by a seemingly weaker condition a is subnormal in g if for every conjugacy class c of g there exists c c for which a is subnormal in a c we also prove the following property of finite non abelian simple groups if a is a subgroup of odd prime order p in a finite almost simple group g then there exists a cyclic p subgroup of f g which does not normalise any non trivial p subgroup of g that is generated by conjugates of a

data mapping by probabilistic modular networks and information theoretic criteria the quantitative mapping of a database that represents a finite set of classified and or unclassified data points may be decomposed into three distinctive learning tasks 1 detection of the structure of each class model with locally mixture clusters 2 estimation of the data distributions for each induced cluster inside each class and 3 classification of the data into classes that realizes the data memberships the mapping function accomplished by the probabilistic modular networks may then be constructed as the optimal estimator with respect to information theory and each of the three tasks can be interpreted as an independent objective in real world applications we adapt a model fitting scheme that determines both the number and kernel of local clusters using information theoretic criteria the class distribution functions are then obtained by learning generalized gaussian mixtures where a soft classification of the data is performed by an efficient incremental algorithm further classification of the data is treated as a hard bayesian detection problem in particular the decision boundaries between the classes are fine tuned by a reinforce or antireinforce supervised learning scheme examples of the application of this framework to medical image quantification automated face recognition and featured database analysis are presented as well

a decomposition theory for binary linear codes the decomposition theory of matroids initiated by paul seymour in the 1980s has had an enormous impact on research in matroid theory this theory when applied to matrices over the binary field yields a powerful decomposition theory for binary linear codes in this paper we give an overview of this code decomposition theory and discuss some of its implications in the context of the recently discovered formulation of maximum likelihood ml decoding of a binary linear code over a binary input discrete memoryless channel as a linear programming problem we translate matroid theoretic results of grotschel and truemper from the combinatorial optimization literature to give examples of nontrivial families of codes for which the ml decoding problem can be solved in time polynomial in the length of the code one such family is that consisting of codes for which the codeword polytope is identical to the koetter vontobel fundamental polytope derived from the entire dual code cperp however we also show that such families of codes are not good in a coding theoretic sense either their dimension or their minimum distance must grow sublinearly with code length as a consequence we have that decoding by linear programming when applied to good codes cannot avoid failing occasionally due to the presence of pseudocode words

graphical codes revisited the set of all even subgraphs of a connected graph g on p vertices with q edges forms a binary linear code c c sub e g with parameters q q p 1 g where g is the girth of g such codes were studied systematically by bredeson and hakimi 1967 and hakimi and bredeson 1968 who were concerned with the problems of augmenting c to a larger q k g code and of efficiently decoding such augmented graphical codes we give a new approach to these problems by requiring the augmented codes to be graphical on one hand we present two construction methods which turn out to contain the methods proposed by hakimi and bredeson as special cases as we show this not only gives a better understanding of their construction it also results in augmenting codes of larger dimension we look at the case of 1 error correcting graphical codes in some detail in particular we show how to obtain the extended hamming codes as purely graphical codes by our approach on the other hand we follow a suggestion of ntafos and hakimi 1981 and use techniques from combinatorial optimization to give decoding procedures for graphical codes which turn out to be considerably more efficient than the approach via majority logic decoding proposed by bredeson and hakimi we also consider the decoding problem for the even graphical code based on the complete graph k sub 2n in more detail we discuss an efficient hardware implementation of an encoding decoding scheme for these codes and show that things may be arranged in such a way that one can also correct all adjacent double errors finally we discuss nonlinear graphical codes

an almost linear time algorithm for graph realization given a 0 1 matrix m the graph realization problem for m is to find a tree t such that the columns of m are incidence vectors of paths in t or to show that no such t exists an algorithm is presented for this problem the time complexity of which is very nearly linear in the number of ones in m

on pseudocodewords and decision regions of linear programming decoding of hdpc codes in this paper we explore the decision regions of linear programming lp decoding we compare the decision regions of an lp decoder a belief propagation bp decoder and the optimal maximum likelihood ml decoder we study the effect of minimal weight pseudocodewords on lp decoding we present global optimization as a method for finding the minimal pseudoweight of a given code as well as the number of minimal weight generators we present a complete pseudoweight distribution for the 24 12 8 extended golay code and provide justifications of why the pseudoweight distribution alone cannot be used for obtaining a tight upper bound on the error probability

an incremental nested partition method for data clustering clustering methods are a powerful tool for discovering patterns in a given data set through an organization of data into subsets of objects that share common features motivated by the independent use of some different partitions criteria and the theoretical and empirical analysis of some of its properties in this paper we introduce an incremental nested partition method which combines these partitions criteria for finding the inner structure of static and dynamic datasets for this we proved that there are relationships of nesting between partitions obtained respectively from these partition criteria and besides that the sensitivity when a new object arrives to the dataset is rigorously studied our algorithm exploits all of these mathematical properties for obtaining the hierarchy of clusterings moreover we realize a theoretical and experimental comparative study of our method with classical hierarchical clustering methods such as single link and complete link and other more recently introduced methods the experimental results over databases of uci repository and the afp and tdt2 news collections show the usefulness and capability of our method to reveal different levels of information hidden in datasets