gtm the generative topographic mapping latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent or hidden variables a familiar example is factor analysis which is based on a linear transformations between the latent space and the data space in this paper we introduce a form of non linear latent variable model called the generative topographic mapping for which the parameters of the model can be determined using the em algorithm gtm provides a principled alternative to the widely used self organizing map som of kohonen 1982 and overcomes most of the significant limitations of the som we demonstrate the performance of the gtm algorithm on a toy problem and on simulated data from flow diagnostics for a multi phase oil pipeline

function approximation with spiked random networks examines the function approximation properties of the random neural network model or gnn the output of the gnn can be computed from the firing probabilities of selected neurons we consider a feedforward bipolar gnn bgnn model which has both positive and negative neurons in the output layer and prove that the bgnn is a universal function approximator specifically for any f spl isin c 0 1 sup s and any spl epsiv 0 we show that there exists a feedforward bgnn which approximates f uniformly with error less than spl epsiv we also show that after some appropriate clamping operation on its output the feedforward gnn is also a universal function approximator

universal approximation using radial basis function networks there have been several recent studies concerning feedforward networks and the problem of approximating arbitrary functionals of a finite number of real variables some of these studies deal with cases in which the hidden layer nonlinearity is not a sigmoid this was motivated by successful applications of feedforward networks with nonsigmoidal hidden layer units this paper reports on a related study of radial basis function rbf networks and it is proved that rbf networks having one hidden layer are capable of universal approximation here the emphasis is on the case of typical rbf networks and the results show that a certain class of rbf networks with the same smoothing factor in each kernel node is broad enough for universal approximation

scaling limits of the schelling model the schelling model of segregation introduced by schelling in 1969 as a model for residential segregation in cities describes how populations of multiple types self organize to form homogeneous clusters of one type in this model vertices in an n dimensional lattice are initially assigned types randomly as time evolves the type at a vertex v has a tendency to be replaced with the most common type within distance w of v we present the first mathematical description of the dynamical scaling limit of this model as w tends to infinity and the lattice is correspondingly rescaled we do this by deriving an integro differential equation for the limiting schelling dynamics and proving almost sure existence and uniqueness of the solutions when the initial conditions are described by white noise the evolving fields are in some sense very rough but we are able to make rigorous sense of the evolution in a key lemma we show that for certain gaussian fields h the supremum of the occupation density of h phi at zero taken over all 1 lipschitz functions phi is almost surely finite thereby extending a result of bass and burdzy in the one dimensional case we also describe the scaling limit of the limiting clusters obtained at time infinity thereby resolving a conjecture of brandt immorlica kamath and kleinberg

probabilistic non linear principal component analysis with gaussian process latent variable models summarising a high dimensional data set with a low dimensional embedding is a standard approach for exploring its structure in this paper we provide an overview of some existing techniques for discovering such embeddings we then introduce a novel probabilistic interpretation of principal component analysis pca that we term dual probabilistic pca dppca the dppca model has the additional advantage that the linear mappings from the embedded space can easily be non linearised through gaussian processes we refer to this model as a gaussian process latent variable model gp lvm through analysis of the gp lvm objective function we relate the model to popular spectral techniques such as kernel pca and multidimensional scaling we then review a practical algorithm for gp lvms in the context of large data sets and develop it to also handle discrete valued data and missing attributes we demonstrate the model on a range of real world and artificially generated data sets

random neural networks with negative and positive signals and product form solution we introduce a new class of random neural networks in which signals are either negative or positive a positive signal arriving at a neuron increases its total signal count or potential by one a negative signal reduces it by one if the potential is positive and has no effect if it is zero when its potential is positive a neuron fires sending positive or negative signals at random intervals to neurons or to the outside positive signals represent excitatory signals and negative signals represent inhibition we show that this model with exponential signal emission intervals poisson external signal arrivals and markovian signal movements between neurons has a product form leading to simple analytical expressions for the system state

a uniform central limit theorem for set indexed partial sum processes with finite variance on obtient des resultats de convergence faible qui ameliorent ceux de pyke 1983 et de bass et pyke 1984

tipping points in 1 dimensional schelling models with switching agents schelling s spacial proximity model was an early agent based model illustrating how ethnic segregation can emerge unwanted from the actions of citizens acting according to individual local preferences here a 1 dimensional unperturbed variant is studied under switching agent dynamics interpretable as being open in that agents may enter and exit the model following the authors work barmpalias et al focs 2014 and that of brandt et al proceedings of the 44th acm symposium on theory of computing stoc 2012 2012 rigorous asymptotic results are established the dynamic allows either type to take over almost everywhere tipping points are identified between the regions of takeover and staticity in a generalization of the models considered in 1 and 3 the model s parameters comprise the initial proportions of the two types along with independent values of the tolerance for each type this model comprises a 1 dimensional spin 1 model with spin dependent external field as well as providing an example of cascading behaviour within a network

unperturbed schelling segregation in two or three dimensions schelling s models of segregation first described in 1969 am econ rev 59 488 493 1969 are among the best known models of self organising behaviour their original purpose was to identify mechanisms of urban racial segregation but his models form part of a family which arises in statistical mechanics neural networks social science and beyond where populations of agents interact on networks despite extensive study unperturbed schelling models have largely resisted rigorous analysis prior results generally focusing on variants in which noise is introduced into the dynamics the resulting system being amenable to standard techniques from statistical mechanics or stochastic evolutionary game theory young in individual strategy and social structure an evolutionary theory of institutions princeton university press princeton 1998 a series of recent papers brandt et al in proceedings of the 44th annual acm symposium on theory of computing stoc 2012 2012 barmpalias et al in 55th annual ieee symposium on foundations of computer science philadelphia 2014 j stat phys 158 806 852 2015 has seen the first rigorous analyses of 1 dimensional unperturbed schelling models in an asymptotic framework largely unknown in statistical mechanics here we provide the first such analysis of 2 and 3 dimensional unperturbed models establishing most of the phase diagram and answering a challenge from brandt et al in proceedings of the 44th annual acm symposium on theory of computing stoc 2012 2012

bayesian variable selection in linear regression abstract this article is concerned with the selection of subsets of predictor variables in a linear regression model for the prediction of a dependent variable it is based on a bayesian approach intended to be as objective as possible a probability distribution is first assigned to the dependent variable through the specification of a family of prior distributions for the unknown parameters in the regression model the method is not fully bayesian however because the ultimate choice of prior distribution from this family is affected by the data it is assumed that the predictors represent distinct observables the corresponding regression coefficients are assigned independent prior distributions for each regression coefficient subject to deletion from the model the prior distribution is a mixture of a point mass at 0 and a diffuse uniform distribution elsewhere that is a spike and slab distribution the random error component is assigned a normal distribution with mean 0 and standard deviation

stability of the random neural network model in a recent paper 1 we have introduced a new neural network model called the random network in which negative or positive signals circulate modelling inhibitory and excitatory signals they are summed at the input of each neuron and constitute its signal potential the state of each neuron in this model is its signal potential while the network state is the vector of signal potentials at each neuron if its potential is positive a neuron fires and sends out signals to the other neurons of the network or to the outside world as it does so its signal potential is depleted we have shown that in the markovian case this model has product form i e the steady state probability distribution of its potential vector is the product of the marginal probabilities of the potential at each neuron the signal flow equations of the network which describe the rate at which positive or negative signals arrive to each neuron are non linear so that their existence and uniqueness is not easily established except for the case of feedforward or backpropagation networks 1 we examine two sub classes of networks balanced and damped networks and obtain stability conditions in each case a hardware implementation of these networks is also suggested

self organized segregation on the grid we consider an agent based model with exponentially distributed waiting times in which two types of agents interact locally over a graph and based on this interaction and on the value of a common intolerance threshold tau decide whether to change their types this is equivalent to a zero temperature ising model with glauber dynamics an asynchronous cellular automaton with extended moore neighborhoods or a schelling model of self organized segregation in an open system and has applications in the analysis of social and biological networks and spin glasses systems some rigorous results were recently obtained in the theoretical computer science literature and this work provides several extensions we enlarge the intolerance interval leading to the expected formation of large segregated regions of agents of a single type from the known size epsilon 0 to size approx 0 134 namely we show that for 0 433 tau 1 2 and by symmetry 1 2 tau 0 567 the expected size of the largest segregated region containing an arbitrary agent is exponential in the size of the neighborhood we further extend the interval leading to expected large segregated regions to size approx 0 312 considering almost segregated regions namely regions where the ratio of the number of agents of one type and the number of agents of the other type vanishes quickly as the size of the neighborhood grows in this case we show that for 0 344 tau le 0 433 and by symmetry for 0 567 le tau 0 656 the expected size of the largest almost segregated region containing an arbitrary agent is exponential in the size of the neighborhood this behavior is reminiscent of supercritical percolation where small clusters of empty sites can be observed within any sufficiently large region of the occupied percolation cluster the exponential bounds that we provide also imply that complete segregation where agents of a single type cover the whole grid does not occur with high probability for p 1 2 and the range of intolerance considered

wavelet neural networks for function learning a wavelet based neural network is described the structure of this network is similar to that of the radial basis function rbf network except that in the present paper the radial basis functions are replaced by orthonormal scaling functions that are not necessarily radial symmetric the efficacy of this type of network in function learning and estimation is demonstrated through theoretical analysis and experimental results in particular it has been shown that the wavelet network has universal and l sup 2 approximation properties and is a consistent function estimator convergence rates associated with these properties are obtained for certain function classes where the rates avoid the curse of dimensionality in the experiments the wavelet network performed well and compared favorably to the mlp and rbf networks

a unifying probabilistic perspective for spectral dimensionality reduction insights and new models we introduce a new perspective on spectral dimensionality reduction which views these methods as gaussian markov random fields grfs our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding the resulting model which we call maximum entropy unfolding meu is a nonlinear generalization of principal component analysis we relate the model to laplacian eigenmaps and isomap we show that parameter fitting in the locally linear embedding lle is approximate maximum likelihood meu we introduce a variant of lle that performs maximum likelihood exactly acyclic lle alle we show that meu and alle are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set finally the maximum likelihood perspective allows us to introduce a new approach to dimensionality reduction based on l1 regularization of the gaussian random field via the graphical lasso

rational function neural network in this paper we observe that a particular class of rational function rf approximations may be viewed as feedforward networks like the radial basis function rbf network the training of the rf network may be performed using a linear adaptive filtering algorithm we illustrate the application of the rf network by considering two nonlinear signal processing problems the first problem concerns the one step prediction of a time series consisting of a pair of complex sinusoid in the presence of colored non gaussian noise simulated data were used for this problem in the second problem we use the rf network to build a nonlinear dynamic model of sea clutter radar backscattering from a sea surface here real life data were used for the study

on the period two property of the majority operator in infinite graphs a self mapping m x x of a nonempty set x has the period two property p2p if m2x x holds for every af periodic point x e x let x be the set of all 0 1 labelings x v 0 1 of the set of vertices f of a locally finite connected graph g for x e x let mx x label v e v by the majority bit that x applies to its neighbors retaining v s x label in case of a tie we show that m has the p2p if there is a finite bound on the degrees in g and log b 0 where b is the number of v 6 v at a distance at most n from a fixed vertex vq v

mamdani fuzzy system universal approximator to a class of random processes the issue of fuzzy systems as universal approximators has drawn significant attention but all results obtained are restricted to deterministic input output i o relationships it should be noted that in practice many i o systems including fuzzy systems operate in the environment which is essentially stochastic in this paper the mamdani fuzzy systems are generalized as stochastic systems by proving the mamdani systems as universal approximators with l sup 2 norm the approximation capability of the stochastic mamdani systems to a class of random processes is systematically analyzed in the mean square sense such stochastic fuzzy systems are capable of approximating the prescribed random processes with arbitrary accuracy further an efficient learning algorithm for the stochastic mamdani systems is developed finally a simulation example is employed to demonstrate our results

robust filtering and smoothing with gaussian processes we propose a principled algorithm for robust bayesian filtering and smoothing in nonlinear stochastic dynamic systems when both the transition function and the measurement function are described by non parametric gaussian process gp models gps are gaining increasing importance in signal processing machine learning robotics and control for representing unknown system functions by posterior probability distributions this modern way of system identification is more robust than finding point estimates of a parametric function representation our principled filtering smoothing approach for gp dynamic systems is based on analytic moment matching in the context of the forward backward algorithm our numerical evaluations demonstrate the robustness of the proposed approach in situations where other state of the art gaussian filters and smoothers can fail

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

bayesian inference for the uncertainty distribution of computer model outputs we consider a problem of inference for the output of a computationally expensive computer model we suppose that the model is to be used in a context where the values of one or more inputs are uncertain so that the input configuration is a random variable we require to make inference about the induced distribution of the output this distribution is called the uncertainty distribution and the general problem is known to users of computer models as uncertainty analysis to be specific we develop bayesian inference for the distribution and density functions of the model output modelling the output as a function of its inputs as a gaussian process we derive expressions for the posterior mean and variance of the distribution and density functions based on data comprising observed outputs at a sample of input configurations we show that direct computation of these expressions may encounter numerical difficulties we develop an alternative approach based on simulating approximate realisations from the posterior distribution of the output function two examples are given to illustrate our methods

probabilistic principal component analysis principal component analysis pca is a ubiquitous technique for data analysis and processing but one which is not based upon a probability model in this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model closely related to factor analysis we consider the properties of the associated likelihood function giving an em algorithm for estimating the principal subspace iteratively and discuss the advantages conveyed by the definition of a probability density function for pca

variational inference for latent variables and uncertain inputs in gaussian processes the gaussian process latent variable model gp lvm provides a flexible approach for non linear dimensionality reduction that has been widely applied however the current approach for training gp lvms is based on maximum likelihood where the latent projection variables are maximised over rather than integrated out in this paper we present a bayesian method for training gp lvms by introducing a non standard variational inference framework that allows to approximately integrate out the latent variables and subsequently train a gp lvm by maximising an analytic lower bound on the exact marginal likelihood we apply this method for learning a gp lvm from i i d observations and for learning non linear dynamical systems where the observations are temporally correlated we show that a benefit of the variational bayesian procedure is its robustness to overfitting and its ability to automatically select the dimensionality of the non linear latent space the resulting framework is generic flexible and easy to extend for other purposes such as gaussian process regression with uncertain or partially missing inputs we demonstrate our method on synthetic data and standard machine learning benchmarks as well as challenging real world datasets including high resolution video data

sparse on line gaussian processes we develop an approach for sparse representations of gaussian process gp models which are bayesian types of kernel machines in order to overcome their limitations for large data sets the method is based on a combination of a bayesian on line algorithm together with a sequential construction of a relevant subsample of the data that fully specifies the prediction of the gp model by using an appealing parameterization and projection techniques in a reproducing kernel hilbert space recursions for the effective parameters and a sparse gaussian approximation of the posterior process are obtained this allows for both a propagation of predictions and bayesian error measures the significance and robustness of our approach are demonstrated on a variety of experiments

coexistence in threshold voter models the threshold voter models considered in this paper are special cases of the nonlinear voter models which were introduced recently by cox and durrett they are spin systems on z d with transition rates begin equation c x eta begin cases 1 text if there is a y text with x y leq n text and eta x neq eta y 0 text otherwise end cases end equation this system is known to cluster if n d 1 and to coexist if n geq 4 in one dimension and if n is reasonably large in other dimensions cox and durrett conjectured that it coexists in all cases except n d 1 in this paper we prove this conjecture the proof is based on comparisons with threshold contact processes the hard part of the proof consists of showing that the second nearest neighbor threshold contact process in one dimension with parameter 1 survives the proof of this result is modeled after the proof by holley and liggett that the critical value of the basic contact process in one dimension is at most 2 by comparison with that proof however the fact that the interaction is not of nearest neighbor type presents substantial additional difficulties in fact part of the proof is computer aided

a unifying view of sparse approximate gaussian process regression we provide a new unifying view including all existing proper probabilistic sparse approximations for gaussian process regression our approach relies on expressing the effective prior which the methods are using this allows new insights to be gained and highlights the relationship between existing methods it also allows for a clear theoretically justified ranking of the closeness of the known approximations to the corresponding full gps finally we point directly to designs of new better sparse approximations combining the best of the existing strategies within attractive computational constraints

the variational gaussian approximation revisited the variational approximation of posterior distributions by multivariate gaussians has been much less popular in the machine learning community compared to the corresponding approximation by factorizing distributions this is for a good reason the gaussian approximation is in general plagued by an number of variational parameters to be optimized being the number of random variables in this letter we discuss the relationship between the laplace and the variational approximation and we show that for models with gaussian priors and factorizing likelihoods the number of variational parameters is actually the approach is applied to gaussian process regression with nongaussian likelihoods

can fuzzy neural nets approximate continuous fuzzy functions abstract we first argue that regular fuzzy neural nets based on standard fuzzy arithmetic and the extension principle cannot be universal approximators this negative result is surprising since regular neural nets are universal approximators we then argue that hybrid fuzzy neural nets not necessarily based on standard fuzzy arithmetic can be universal approximators