a fast algorithm for robust mixtures in the presence of measurement errors in experimental and observational sciences detecting atypical peculiar data from large sets of measurements has the potential of highlighting candidates of interesting new types of objects that deserve more detailed domain specific followup study however measurement data is nearly never free of measurement errors these errors can generate false outliers that are not truly interesting although many approaches exist for finding outliers they have no means to tell to what extent the peculiarity is not simply due to measurement errors to address this issue we have developed a model based approach to infer genuine outliers from multivariate data sets when measurement error information is available this is based on a probabilistic mixture of hierarchical density models in which parameter estimation is made feasible by a tree structured variational expectation maximization algorithm here we further develop an algorithmic enhancement to address the scalability of this approach in order to make it applicable to large data sets via a k dimensional tree based partitioning of the variational posterior assignments this creates a non trivial tradeoff between a more detailed noise model to enhance the detection accuracy and the coarsened posterior representation to obtain computational speedup hence we conduct extensive experimental validation to study the accuracy speed tradeoffs achievable in a variety of data conditions we find that at low to moderate error levels a speedup factor that is at least linear in the number of data points can be achieved without significantly sacrificing the detection accuracy the benefits of including measurement error information into the modeling is evident in all situations and the gain roughly recovers the loss incurred by the speedup procedure in large error conditions we analyze and discuss in detail the characteristics of our algorithm based on results obtained on appropriately designed synthetic data experiments and we also demonstrate its working in a real application example

growth type invariants for ℤ d subshifts of finite type and arithmetical classes of real numbers we discuss some numerical invariants of multidimensional shifts of finite type sfts which are associated with the growth rates of the number of admissible finite configurations extending an unpublished example of tsirelson a strange two dimensional symbolic system 1992 we show that growth complexities of the form exp n o 1 are possible for non integer s in terminology of de carvalho port math 54 1 19 40 1997 such subshifts have entropy dimension the class of possible s are identified in terms of arithmetical classes of real numbers of weihrauch and zheng math log q 47 1 51 65 2001

a uniform type inverse spectrum theorem for entrywise nonnegative matrices abstract influenced by earlier results of fiedler m fiedler eigenvalues of nonnegative symmetric matrices linear algebra appl 9 1974 119 142 wuven guo wuven eigenvalues of nonnegative matrices linear algebra appl 266 1997 261 270 established interesting theorems and posed several natural questions in the inverse spectrum problem for entrywise nonnegative matrices we answer some of these of a uniform character obtaining the sharpest possible results

algebraic shift equivalence and primitive matrices motivated by symbolic dynamics we study the problem given a unital subring s of the reals when is a matrix a algebraically shift equivalent over s to a primitive matrix we conjecture that simple necessary conditions on the nonzero spectrum of a are sufficient and establish the conjecture in many cases if s is the integers we give some lower bounds on sizes of realizing primitive matrices for dedekind domains we prove that algebraic shift equivalence implies algebraic strong shift equivalence

a semi lagrangian vlasov solver in tensor train format in this article we derive a semi lagrangian scheme for the solution of the vlasov equation represented as a low parametric tensor grid based methods for the vlasov equation have been shown to give accurate results but their use has mostly been limited to simulations in two dimensional phase space due to extensive memory requirements in higher dimensions compression of the solution via high order singular value decomposition can help in reducing the storage requirements and the tensor train tt format provides efficient basic linear algebra routines for low rank representations of tensors in this paper we develop interpolation formulas for a semi lagrangian solver in tt format in order to efficiently implement the method we propose a compression of the matrix representing the interpolation step and an efficient implementation of the hadamard product we show numerical simulations for standard test cases in two four and six dimensional phase space depending on the test case the memory requirem

universality and complexity in cellular automata abstract cellular automata are discrete dynamical systems with simple construction but complex self organizing behaviour evidence is presented that all one dimensional cellular automata fall into four distinct universality classes characterizations of the structures generated in these classes are discussed three classes exhibit behaviour analogous to limit points limit cycles and chaotic attractors the fourth class is probably capable of universal computation so that properties of its infinite time behaviour are undecidable

a spatially constrained generative model and an em algorithm for image segmentation in this paper we present a novel spatially constrained generative model and an expectation maximization em algorithm for model based image segmentation the generative model assumes that the unobserved class labels of neighboring pixels in the image are generated by prior distributions with similar parameters where similarity is defined by entropic quantities relating to the neighboring priors in order to estimate model parameters from observations we derive a spatially constrained em algorithm that iteratively maximizes a lower bound on the data log likelihood where the penalty term is data dependent our algorithm is very easy to implement and is similar to the standard em algorithm for gaussian mixtures with the main difference that the labels posteriors are smoothed over pixels between each e and m step by a standard image filter experiments on synthetic and real images show that our algorithm achieves competitive segmentation results compared to other markov based methods and is in general faster

unsupervised learning of gaussian mixtures based on variational component splitting in this paper we present an incremental method for model selection and learning of gaussian mixtures based on the recently proposed variational bayes approach the method adds components to the mixture using a bayesian splitting test procedure a component is split into two components and then variational update equations are applied only to the parameters of the two components as a result either both components are retained in the model or one of them is found to be redundant and is eliminated from the model in our approach the model selection problem is treated locally in a region of the data space so we can set more informative priors based on the local data distribution a modified bayesian mixture model is presented to implement this approach along with a learning algorithm that iteratively applies a splitting test on each mixture component experimental results and comparisons with two other techniques testify for the adequacy of the proposed approach

the mean field theory in em procedures for markov random fields in many signal processing and pattern recognition applications the hidden data are modeled as markov processes and the main difficulty of using the maximisation em algorithm for these applications is the calculation of the conditional expectations of the hidden markov processes it is shown how the mean field theory from statistical mechanics can be used to calculate the conditional expectations for these problems efficiently the efficacy of the mean field theory approach is demonstrated on parameter estimation for one dimensional mixture data and two dimensional unsupervised stochastic model based image segmentation experimental results indicate that in the 1 d case the mean field theory approach provides results comparable to those obtained by baum s 1987 algorithm which is known to be optimal in the 2 d case where baum s algorithm can no longer be used the mean field theory provides good parameter estimates and image segmentation for both synthetic and real world images

the classification of some generalised bunce deddens algebras abstract we use k theory to prove an isomorphism theorem for a large class of generalised bunce deddens algebras constructed by kribs and solel from a directed graph e and a sequence of positive integers in particular we compute the torsion free component of the k 0 group for a class of generalised bunce deddens algebras to show that supernatural numbers are a complete invariant for this class

a low rank method for two dimensional time dependent radiation transport calculations abstract the low rank approximation is a complexity reduction technique to approximate a tensor or a matrix with a reduced rank which has been applied to the simulation of high dimensional problems to reduce the memory required and computational cost in this work a dynamical low rank approximation method is developed for the time dependent radiation transport equation in 1 d and 2 d cartesian geometries using a finite volume discretization in space and a spherical harmonics basis in angle we construct a system that evolves on a low rank manifold via an operator splitting approach numerical results on five test problems demonstrate that the low rank solution requires less memory and computational time than solving the full rank equations with the same accuracy it is furthermore shown that the low rank algorithm can obtain high fidelity results by increasing the number of basis functions while keeping the rank fixed

hierarchical mixtures of experts and the em algorithm we present a tree structured architecture for supervised learning the statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models glim s learning is treated as a maximum likelihood problem in particular we present an expectation maximization em algorithm for adjusting the parameters of the architecture we also develop an on line learning algorithm in which the parameters are updated incrementally comparative simulation results are presented in the robot dynamics domain

on line em algorithm for the normalized gaussian network a normalized gaussian network ngnet moody darken 1989 is a network of local linear regression units the model softly partitions the input space by normalized gaussian functions and each local unit linearly approximates the output within the partition in this article we propose a new on line em algorithm for the ngnet which is derived from the batch em algorithm xu jordan hinton 1995 by introducing a discount factor we show that the on line em algorithm is equivalent to the batch em algorithm if a specific scheduling of the discount factor is employed in addition we show that the on line em algorithm can be considered as a stochastic approximation method to find the maximum likelihood estimator a new regularization method is proposed in order to deal with a singular input distribution in order to manage dynamic environments where the input output distribution of data changes over time unit manipulation mechanisms such as unit production unit deletion and unit division are also introduced based on probabilistic interpretation experimental results show that our approach is suitable for function approximation problems in dynamic environments we also apply our on line em algorithm to robot dynamics problems and compare our algorithm with the mixtures of experts family

unsupervised learning of finite mixture models this paper proposes an unsupervised algorithm for learning a finite mixture model from multivariate data the adjective unsupervised is justified by two properties of the algorithm 1 it is capable of selecting the number of components and 2 unlike the standard expectation maximization em algorithm it does not require careful initialization the proposed method also avoids another drawback of em for mixture fitting the possibility of convergence toward a singular estimate at the boundary of the parameter space the novelty of our approach is that we do not use a model selection criterion to choose one among a set of preestimated candidate models instead we seamlessly integrate estimation and model selection in a single algorithm our technique can be applied to any type of parametric mixture model for which it is possible to write an em algorithm in this paper we illustrate it with experiments involving gaussian mixtures these experiments testify for the good performance of our approach

elgenvalues of nonnegative matrices abstract in a partial continuation of work by fiedler some spectral properties of symmetric nonnegative matrices are extended to general nonnegative matrices

power series with positive coefficients arising from the characteristic polynomials of positive matrices let a be an n times n entrywise positive matrix and let f t det i t a we prove the surprising result that there always exists a positive integer n such that the formal power series expansion of 1 f t 1 n around t 0 has positive coefficients

the spectra of nonnegative integer matrices via formal power series an old problem in matrix theory is to determine the n tuples of complex numbers which can occur as the spectrum of a matrix with nonnegative entries see bp94 chapter 4 or min88 chapter vii authors have studied the case where the ntuple is comprised of real numbers bor95 cia68 fri78 kel71 per53 sal72 sou83 sul49 the case where the matrices under consideration are symmetric fie74 jll96 and the general problem joh81 lm99 ll79 rea94 rea96 wuw97 various necessary conditions and sufficient conditions have been provided but a complete characterization is known for real n tuples only for n 4 kel71 sul49 and for complex n tuples only for n 3 ll79 motivated by symbolic dynamics boyle and handelman refocused attention on the nonzero part of the spectrum by making the following spectral conjecture bh91 bh93 see also boy93 8 and lm95 chapter 11 below a matrix a is primitive if all entries of a are nonnegative and for some n all entries of a are strictly positive also

asymptotic diffusion limit of cell temperature discretisation schemes for thermal radiation transport abstract this paper attempts to unify the asymptotic diffusion limit analysis of thermal radiation transport schemes for a linear discontinuous representation of the material temperature reconstructed from cell centred temperature unknowns in a process known as source tilting the asymptotic limits of both monte carlo continuous in space and deterministic approaches based on linear discontinuous finite elements for solving the transport equation are investigated in slab geometry the resulting discrete diffusion equations are found to have nonphysical terms that are proportional to any cell edge discontinuity in the temperature representation based on this analysis it is possible to design accurate schemes for representing the material temperature for coupling thermal radiation transport codes to a cell centred representation of internal energy favoured by ale arbitrary lagrange eulerian hydrodynamics schemes

a characterization of trace zero nonnegative 5 5 matrices let 1 2 3 4 5 be a list of complex numbers with 1 2 3 4 5 0 necessary and sufficient conditions for the existence of an entry wise nonnegative 5 5 matrix a with spectrum are presented

on convergence properties of the em algorithm for gaussian mixtures we build up the mathematical connection between the expectation maximization em algorithm and gradient based approaches for maximum likelihood learning of finite gaussian mixtures we show that the em step in parameter space is obtained from the gradient via a projection matrix p and we provide an explicit expression for the matrix we then analyze the convergence of em in terms of special properties of p and provide new results analyzing the effect that p has on the likelihood surface based on these mathematical results we present a comparative discussion of the advantages and disadvantages of em and other algorithms for the learning of gaussian mixture models

realizable lists on a class of nonnegative matrices abstract a square matrix of order n with n 2 is called permutative matrix when all its rows are permutations of the first row in this paper recalling spectral results for partitioned into 2 by 2 symmetric blocks matrices sufficient conditions on a given complex list to be the list of the eigenvalues of a nonnegative permutative matrix are given in particular we study niep and pniep when some complex elements in the lists under consideration have non zero imaginary part realizability regions for nonnegative permutative matrices are obtained a guo s realizability preserving perturbations result is obtained

a distributed classification estimation algorithm for sensor networks in this paper we address the problem of simultaneous classification and estimation of hidden parameters in a sensor network with communications constraints in particular we consider a network of noisy sensors which measure a common scalar unknown parameter we assume that a fraction of the nodes represent faulty sensors whose measurements are poorly reliable the goal for each node is to simultaneously identify its class faulty or nonfaulty and estimate the common parameter we propose a novel cooperative iterative algorithm which copes with the communication constraints imposed by the network and shows remarkable performance our main result is a rigorous proof of the convergence of the algorithm under a fixed communication graph and a characterization of the limit behavior as the network size goes to infinity in particular we prove that in the limit when the number of sensors goes to infinity the common unknown parameter is estimated with arbitrary small error while the classification error

on the nonnegative eigenvalue problem abstract 1 1 n is the spectrum of a stochastic matrix of order n 1 if and only if there exists some real matrix b of order n with spectrum 1 n and some n simplex s r n containing the origin such that bs s this result appears implicitly in ciarlet s work and it provides us with a geometrical tool to obtain sufficient conditions for the nonnegative eigenvalue problem we employ it here to generalize the sufficient conditions given by kellogg

strong shift equivalence and the generalized spectral conjecture for nonnegative matrices abstract given matrices a and b shift equivalent over a dense subring r of r with a primitive we show that b is strong shift equivalent over r to a primitive matrix this result shows that the weak form of the generalized spectral conjecture for primitive matrices implies the strong form the foundation of this work is the recent result that for any ring r the group nk 1 r of algebraic k theory classifies the refinement of shift equivalence by strong shift equivalence for matrices over r

on the dynamics and recursive properties of multidimensional symbolic systems we study the sub dynamics of multidimensional shifts of finite type and sofic shifts and the action of cellular automata on their limit sets such a subaction is always an effective dynamical system i e it is isomorphic to a subshift over the cantor set the complement of which can be written as the union of a recursive sequence of basic sets our main result is that to varying degrees this recursive theoretic condition is also sufficient we show that the class of expansive subactions of multidimensional sofic shifts is the same as the class of expansive effective systems and that a general effective system can be realized modulo a small extension as the subaction of a shift of finite type or as the action of a cellular automaton on its limit set after removing a dynamically trivial set as applications we characterize in terms of their computational properties the numbers which can occur as the entropy of cellular automata and construct sfts and cas with various interesting properties

the real nonnegative inverse eigenvalue problem is np hard abstract a list of complex numbers is realizable if it is the spectrum of a nonnegative matrix in 1949 suleimanova posed the nonnegative inverse eigenvalue problem niep the problem of determining which lists of complex numbers are realizable the version for reals of the niep rniep asks for realizable lists of real numbers in the literature there are many sufficient conditions or criteria for lists of real numbers to be realizable we will present an unified account of these criteria then we will see that the decision problem associated to the rniep is np hard and we will study the complexity for the decision problems associated to known criteria

em procedures using mean field like approximations for markov model based image segmentation this paper deals with markov random field model based image segmentation this involves parameter estimation in hidden markov models for which one of the most widely used procedures is the em algorithm in practice difficult ies arise due to the dependence structure in the models and approximations are required to make the algorithm tractable we propose a class of algorithms in which the idea is to deal with systems of independent variables this corresponds to approximations of the pixels interactions similar to the mean field approximation it follows algorithms that have the advantage of taking the markovian structure into account while preserving the good features of em in addition this class that includes new and already known procedures is presented in a unified framework showing that apparently distant algorithms come from similar approximation principles we illustrate the algorithms performance on synthetic and real images these experiments point out the ability of our procedures to take the spatial information into account our algorithms often show significant improvement when comparing with the em algorithm applied with no account of the spatial structure and with the icm algorithm based on maximization of the pseudo likelihood and commonly used in image segmentation

a tutorial on particle filters for online nonlinear non gaussian bayesian tracking increasingly for many application areas it is becoming important to include elements of nonlinearity and non gaussianity in order to model accurately the underlying dynamics of a physical system moreover it is typically crucial to process data on line as it arrives both from the point of view of storage costs as well as for rapid adaptation to changing signal characteristics in this paper we review both optimal and suboptimal bayesian algorithms for nonlinear non gaussian tracking problems with a focus on particle filters particle filters are sequential monte carlo methods based on point mass or particle representations of probability densities which can be applied to any state space model and which generalize the traditional kalman filtering methods several variants of the particle filter such as sir asir and rpf are introduced within a generic framework of the sequential importance sampling sis algorithm these are discussed and compared with the standard ekf through an illustrative example

small polynomial matrix presentations of nonnegative matrices we investigate the use of polynomial matrices to give efficient presentations of nonnegative matrices exhibiting prescribed spectral and algebraic behavior

consensus seeking in multiagent systems under dynamically changing interaction topologies this note considers the problem of information consensus among multiple agents in the presence of limited and unreliable information exchange with dynamically changing interaction topologies both discrete and continuous update schemes are proposed for information consensus this note shows that information consensus under dynamically changing interaction topologies can be achieved asymptotically if the union of the directed interaction graphs have a spanning tree frequently enough as the system evolves

survey paper structured low rank approximation and its applications fitting data by a bounded complexity linear model is equivalent to low rank approximation of a matrix constructed from the data the data matrix being hankel structured is equivalent to the existence of a linear time invariant system that fits the data and the rank constraint is related to a bound on the model complexity in the special case of fitting by a static model the data matrix and its low rank approximation are unstructured we outline applications in system theory approximate realization model reduction output error and errors in variables identification signal processing harmonic retrieval sum of damped exponentials and finite impulse response modeling and computer algebra approximate common divisor algorithms based on heuristics and local optimization methods are presented generalizations of the low rank approximation problem result from different approximation criteria e g weighted norm and constraints on the data matrix e g nonnegativity related problems are rank minimization and structured pseudospectra

on positive matrices abstract we relate several results on positive matrices due to soittola 1976 handelman 1981 1987 and lind 1984 giving conditions for the existence of some positive matrix to represent a linear map the result of soittola concerns finite automata and those of handelman and lind concern symbolic dynamics

on solutions to the pn equations for thermal radiative transfer we present results for the spherical harmonics p n method for solving problems of time dependent thermal radiative transport we prove a theorem that demonstrates that in the streaming limit the spatially and temporally continuous p n equations will allow negative energy densities for any finite order of n we also develop an implicit numerical method for solving the p n equations to explore the impact of the theorem the numerical method uses a high resolution riemann solver to produce an upwinded discretization we employ a quasi linear approach to integrate the nonlinearites added to make the scheme non oscillatory we use the backward euler method for time integration and treat the material interaction terms fully nonlinearly reflecting boundary conditions for the p n equations are presented and we show how to implement this boundary condition using ghost cells the implicit method was able to produce robust results to thermal transport problems in one and two dimensions the numerical method is used to analyze the accuracy of various p n expansion orders on several problems in two dimensional problems the numerical p n solutions contained negative radiation energy densities as predicted by our theorem the numerical results showed that the material temperature also became negative a result outside the scope of the theorem our numerical method can handle these negative values but they would cause problems in a radiation hydrodynamics calculation

online learning of hierarchical pitman yor process mixture of generalized dirichlet distributions with feature selection in this paper a novel statistical generative model based on hierarchical pitman yor process and generalized dirichlet distributions gds is presented the proposed model allows us to perform joint clustering and feature selection thanks to the interesting properties of the gd distribution we develop an online variational inference algorithm formulated in terms of the minimization of a kullback leibler divergence of our resulting model that tackles the problem of learning from high dimensional examples this variational bayes formulation allows simultaneously estimating the parameters determining the model s complexity and selecting the appropriate relevant features for the clustering structure moreover the proposed online learning algorithm allows data instances to be processed in a sequential manner which is critical for large scale and real time applications experiments conducted using challenging applications namely scene recognition and video segmentation where our approach is viewed as an unsupervised technique for visual learning in high dimensional spaces showed that the proposed approach is suitable and promising

robust and accurate filtered spherical harmonics expansions for radiative transfer we present a novel application of filters to the spherical harmonics p n expansion for radiative transfer problems in the high energy density regime the filter we use is based on non oscillatory spherical splines and a filter strength chosen to i preserve the equilibrium diffusion limit and ii vanish as the expansion order tends to infinity our implementation is based on modified equations that are derived by applying the filter after every time step in a simple first order time integration scheme the method is readily applied to existing codes that solve the p n equations numerical results demonstrate that the solution to the filtered p n equations are i more robust and less oscillatory than standard p n solutions and ii more accurate than discrete ordinates solutions of comparable order in particular the filtered p 7 solution demonstrates comparable accuracy to an implicit monte carlo solution for a benchmark hohlraum problem in 2d cartesian geometry

distributed em algorithm for gaussian mixtures in sensor networks this paper presents a distributed expectation maximization em algorithm over sensor networks in the e step of this algorithm each sensor node independently calculates local sufficient statistics by using local observations a consensus filter is used to diffuse local sufficient statistics to neighbors and estimate global sufficient statistics in each node by using this consensus filter each node can gradually diffuse its local information over the entire network and asymptotically the estimate of global sufficient statistics is obtained in the m step of this algorithm each sensor node uses the estimated global sufficient statistics to update model parameters of the gaussian mixtures which can maximize the log likelihood in the same way as in the standard em algorithm because the consensus filter only requires that each node communicate with its neighbors the distributed em algorithm is scalable and robust it is also shown that the distributed em algorithm is a stochastic approximation to the standard em algorithm thus it converges to a local maximum of the log likelihood several simulations of sensor networks are given to verify the proposed algorithm

a characterization of the entropies of multidimensional shifts of finite type we show that the values of entropies of multidimensional shifts of finite type sfts are characterized by a certain computation theoretic property a real number h 0 is the entropy of such an sft if and only if it is right recursively enumerable i e there is a computable sequence of rational numbers converging to h from above the same characterization holds for the entropies of sofic shifts on the other hand the entropy of strongly irreducible sfts is computable

cooperative information maximization with gaussian activation functions for self organizing maps in this paper we propose a new information theoretic method to produce explicit self organizing maps soms competition is realized by maximizing mutual information between input patterns and competitive units competitive unit outputs are computed by the gaussian function of distance between input patterns and competitive units a property of this gaussian function is that as distance becomes smaller a neuron tends to fire strongly cooperation processes are realized by taking into account the firing rates of neighboring neurons we applied our method to uniform distribution learning chemical compound classification and road classification experimental results confirmed that cooperation processes could significantly increase information content in input patterns when cooperative operations are not effective in increasing information mutual information as well as entropy maximization is used to increase information experimental results showed that entropy maximization could be used to increase information and to obtain clearer soms because competitive units are forced to be equally used on average

time dependent simplified pn approximation to the equations of radiative transfer the steady state simplified pn approximation to the radiative transport equation has been successfully applied to many problems involving radiation this paper presents the derivation of time dependent simplified pn spn equations up to n 3 via two different approaches first we use an asymptotic analysis similar to the asymptotic derivation of the steady state spn equations second we use an approach similar to the original derivation of the steady state spn equations and we show that both approaches lead to similar results special focus is put on the well posedness of the equations and the question whether it can be guaranteed that the solution satisfies the correct physical bounds several numerical test cases are shown including an analytical benchmark due to su and olson b su g l olson an analytical benchmark for non equilibrium radiative transfer in an isotropically scattering medium ann nucl energy 24 1997 1035 1055