serial correlation in neural spike trains experimental evidence stochastic modeling and single neuron variability the activity of spiking neurons is frequently described by renewal point process models that assume the statistical independence and identical distribution of the intervals between action potentials however the assumption of independent intervals must be questioned for many different types of neurons we review experimental studies that reported the feature of a negative serial correlation of neighboring intervals commonly observed in neurons in the sensory periphery as well as in central neurons notably in the mammalian cortex in our experiments we observed the same short lived negative serial dependence of intervals in the spontaneous activity of mushroom body extrinsic neurons in the honeybee to model serial interval correlations of arbitrary lags we suggest a family of autoregressive point processes its marginal interval distribution is described by the generalized gamma model which includes as special cases the log normal and gamma distributions which have been widely used to characterize regular spiking neurons in numeric simulations we investigated how serial correlation affects the variance of the neural spike count we show that the experimentally confirmed negative correlation reduces single neuron variability as quantified by the fano factor by up to 50 which favors the transmission of a rate code we argue that the feature of a negative serial correlation is likely to be common to the class of spike frequency adapting neurons and that it might have been largely overlooked in extracellular single unit recordings due to spike sorting errors

nonparametric modeling of neural point processes via stochastic gradient boosting regression statistical nonparametric modeling tools that enable the discovery and approximation of functional forms e g tuning functions relating neural spiking activity to relevant covariates are desirable tools in neuroscience in this article we show how stochastic gradient boosting regression can be successfully extended to the modeling of spiking activity data while preserving their point process nature thus providing a robust nonparametric modeling tool we formulate stochastic gradient boosting in terms of approximating the conditional intensity function of a point process in discrete time and use the standard likelihood of the process to derive the loss function for the approximation problem to illustrate the approach we apply the algorithm to the modeling of primary motor and parietal spiking activity as a function of spiking history and kinematics during a two dimensional reaching task model selection goodness of fit via the time rescaling theorem model interpretation via partial dependence plots ranking of covariates according to their relative importance and prediction of peri event time histograms are illustrated and discussed additionally we use the tenfold cross validated log likelihood of the modeled neural processes 67 cells to compare the performance of gradient boosting regression to two alternative approaches standard generalized linear models glms and bayesian p splines with markov chain monte carlo mcmc sampling in our data set gradient boosting outperformed both bayesian p splines in approximately 90 of the cells and glms 100 because of its good performance and computational efficiency we propose stochastic gradient boosting regression as an off the shelf nonparametric tool for initial analyses of large neural data sets e g more than 50 cells more than 105 samples per cell with corresponding multidimensional covariate spaces e g more than four covariates in the cases where a functional form might be amenable to a more compact representation gradient boosting might also lead to the discovery of simpler parametric models

a continuous entropy rate estimator for spike trains using a k means based context tree entropy rate quantifies the change of information of a stochastic process cover thomas 2006 for decades the temporal dynamics of spike trains generated by neurons has been studied as a stochastic process barbieri quirk frank wilson brown 2001 brown frank tang quirk wilson 1998 kass ventura 2001 metzner koch wessel gabbiani 1998 zhang ginzburg mcnaughton sejnowski 1998 we propose here to estimate the entropy rate of a spike train from an inhomogeneous hidden markov model of the spike intervals the model is constructed by building a context tree structure to lay out the conditional probabilities of various subsequences of the spike train for each state in the markov chain we assume a gamma distribution over the spike intervals although any appropriate distribution may be employed as circumstances dictate the entropy and confidence intervals for the entropy are calculated from bootstrapping samples taken from a large raw data sequence the estimator was first tested on synthetic data generated by multiple order markov chains and it always converged to the theoretical shannon entropy rate except in the case of a sixth order model where the calculations were terminated before convergence was reached we also applied the method to experimental data and compare its performance with that of several other methods of entropy estimation

estimating functions of probability distributions from a finite set of samples this paper addresses the problem of estimating a function of a probability distribution from a finite set of samples of that distribution a bayesian analysis of this problem is presented the optimal properties of the bayes estimators are discussed and as an example of the formalism closed form expressions for the bayes estimators for the moments of the shannon entropy function are derived then numerical results are presented that compare the bayes estimator to the frequency counts estimator for the shannon entropy we also present the closed form estimators all derived elsewhere for the mutual information mathrm ensuremath chi 2 covariance and some other statistics c 1995 the american physical society

an entropy satisfying conservative method for the fokker planck equation of the finitely extensible nonlinear elastic dumbbell model in this paper we propose an entropy satisfying conservative method to solve the fokker planck equation of the finitely extensible nonlinear elastic dumbbell model for polymers subject to homogeneous fluids both semidiscrete and fully discrete schemes satisfy all three desired properties i mass conservation ii positivity preserving and iii entropy satisfying in the sense that these schemes satisfy discrete entropy inequalities for both the physical entropy and the quadratic entropy these ensure that the computed solution is a probability density and the schemes are entropy stable and preserve the equilibrium solutions we also prove convergence of the numerical solution to the equilibrium solution as time becomes large zero flux at boundary is naturally incorporated and boundary behavior is resolved sharply both one and two dimensional numerical results are provided to demonstrate the good qualities of the scheme and the effects of some canonical homogeneous flows

mean field approximations for coupled populations of generalized linear model spiking neurons with markov refractoriness there has recently been a great deal of interest in inferring network connectivity from the spike trains in populations of neurons one class of useful models that can be fit easily to spiking data is based on generalized linear point process models from statistics once the parameters for these models are fit the analyst is left with a nonlinear spiking network model with delays which in general may be very difficult to understand analytically here we develop mean field methods for approximating the stimulus driven firing rates in both the time varying and steady state cases auto and cross correlations and stimulus dependent filtering properties of these networks these approximations are valid when the contributions of individual network coupling terms are small and hence the total input to a neuron is approximately gaussian these approximations lead to deterministic ordinary differential equations that are much easier to solve and analyze than direct monte carlo simulation of the network activity these approximations also provide an analytical way to evaluate the linear input output filter of neurons and how the filters are modulated by network interactions and some stimulus feature finally in the case of strong refractory effects the mean field approximations in the generalized linear model become inaccurate therefore we introduce a model that captures strong refractoriness retains all of the easy fitting properties of the standard generalized linear model and leads to much more accurate approximations of mean firing rates and cross correlations that retain fine temporal behaviors

some theoretical results on neural spike train probability models this article contains two main theoretical results on neural spike train models using the counting or point process on the real line as a model for the spike train the first part of this article considers template matching of multiple spike trains p values for the occurrences of a given template or pattern in a set of spike trains are computed using a general scoring system by identifying the pattern with an experimental stimulus multiple spike trains can be deciphered to provide useful information the second part of the article assumes that the counting process has a conditional intensity function that is a product of a free firing rate function s which depends only on the stimulus and a recovery function r which depends only on the time since the last spike if s and r belong to a q smooth class of functions it is proved that sieve maximum likelihood estimators for s and r achieve the optimal convergence rate except for a logarithmic factor under l 1 loss

a comparison of descriptive models of a single spike train by information geometric measure in examining spike trains different models are used to describe their structure the different models often seem quite similar but because they are cast in different formalisms it is often difficult to compare their predictions here we use the information geometric measure an orthogonal coordinate representation of point processes to express different models of stochastic point processes in a common coordinate system within such a framework it becomes straightforward to visualize higher order correlations of different models and thereby assess the differences between models we apply the information geometric measure to compare two similar but not identical models of neuronal spike trains the inhomogeneous markov and the mixture of poisson models it is shown that they differ in the second and higher order interaction terms in the mixture of poisson model the second and higher order interactions are of comparable magnitude within each order whereas in the inhomogeneous markov model they have alternating signs over different orders this provides guidance about what measurements would effectively separate the two models as newer models are proposed they also can be compared to these models using information geometry

network amplification of local fluctuations causes high spike rate variability fractal firing patterns and oscillatory local field potentials we investigate a model for neural activity in a two dimensional sheet of leaky integrate and fire neurons with feedback connectivity consisting of local excitation and surround inhibition each neuron receives stochastic input from an external source independent in space and time as recently suggested by softky and koch 1992 1993 independent stochastic input alone cannot explain the high interspike interval variability exhibited by cortical neurons in behaving monkeys we show that high variability can be obtained due to the amplification of correlated fluctuations in a recurrent network furthermore the cross correlation functions have a dual structure with a sharp peak on top of a much broader hill this is due to the inhibitory and excitatory feedback connections which cause hotspots of neural activity to form within the network these localized patterns of excitation appear as clusters or stripes that coalesce disintegrate or fluctuate in size while simultaneously moving in a random walk constrained by the interaction with other clusters the synaptic current impinging upon a single neuron shows large fluctuations at many time scales leading to a large coefficient of variation cv for the interspike interval statistics the power spectrum associated with single units shows a 1 f decay for small frequencies and is flat at higher frequencies while the power spectrum of the spiking activity averaged over many cells equivalent to the local field potential shows no 1 f decay but a prominent peak around 40 hz in agreement with data recorded from cat and monkey cortex gray et al 1990 eckhorn et al 1993 firing rates exhibit self similarity between 20 and 800 msec resulting in 1 f like noise consistent with the fractal nature of neural spike trains teich 1992

preserving energy resp dissipation in numerical pdes using the average vector field method we give a systematic method for discretizing hamiltonian partial differential equations pdes with constant symplectic structure while preserving their energy exactly the same method applied to pdes with constant dissipative structure also preserves the correct monotonic decrease of energy the method is illustrated by many examples in the hamiltonian case these include the sine gordon korteweg de vries nonlinear schrodinger linear time dependent schrodinger and maxwell equations in the dissipative case the examples are the allen cahn cahn hilliard ginzburg landau and heat equations

difference schemes for solving the generalized nonlinear schrodinger equation this paper studies finite difference schemes for solving the generalized nonlinear schrodinger gnls equationiut uxx q u 2 u f x t u a new linearlized crank nicolson type scheme is presented by applying an extrapolation technique to the real coefficient of the nonlinear term in the gnls equation several schemes including crank nicolson type schemes hopscotch type schemes split step fourier scheme and pseudospectral scheme are adopted for solving three model problems of gnls equation which arise from many physical problems withq s s2 q s ln 1 s andq s 4s 1 s respectively the numerical results demonstrate that the linearized crank nicolson scheme is efficient and robust

quantization of memoryless and gauss markov sources over binary markov channels joint source channel coding for stationary memoryless and gauss markov sources and binary markov channels is considered the channel is an additive noise channel where the noise process is an mth order markov chain two joint source channel coding schemes are considered the first is a channel optimized vector quantizer optimized for both source and channel the second scheme consists of a scalar quantizer and a maximum a posteriori detector in this scheme it is assumed that the scalar quantizer output has residual redundancy that can be exploited by the maximum a posteriori detector to combat the correlated channel noise these two schemes are then compared against two schemes which use channel interleaving numerical results show that the proposed schemes outperform the interleaving schemes for very noisy channels with high noise correlation gains of 4 5 db in signal to noise ratio are possible

runge kutta approximation of quasi linear parabolic equations we study the convergence properties of implicit runge kutta meth ods applied to time discretization of parabolic equations with time or solution dependent operator error bounds are derived in the energy norm the con vergence analysis uses two different approaches the first technically simpler approach relies on energy estimates and requires algebraic stability of the runge kutta method the second one is based on estimates for linear time invariant equations and uses fourier and perturbation techniques it applies to a 9 stable runge kutta methods and yields the precise temporal order of conver gence this order is noninteger in general and depends on the type of boundary conditions

a new point symmetry based fuzzy genetic clustering technique for automatic evolution of clusters in this paper a fuzzy point symmetry based genetic clustering technique fuzzy vgaps is proposed which can automatically determine the number of clusters present in a data set as well as a good fuzzy partitioning of the data the clusters can be of any size shape or convexity as long as they possess the property of symmetry here the membership values of points to different clusters are computed using the newly proposed point symmetry based distance a variable number of cluster centers are encoded in the chromosomes a new fuzzy symmetry based cluster validity index fsym index is first proposed here and thereafter it is utilized to measure the fitness of the chromosomes the proposed index can detect non convex as well as convex non hyperspherical partitioning with variable number of clusters it is mathematically justified via its relationship to a well defined hard cluster validity function the dunn s index for which the condition of uniqueness has already been established the results of the fuzzy vgaps are compared with those obtained by seven other algorithms including both fuzzy and crisp methods on four artificial and four real life data sets some real life applications of fuzzy vgaps to automatically cluster the gene expression data as well as segmenting the magnetic resonance brain image with multiple sclerosis lesions are also demonstrated

examples of the construction of integral equations in equilibrium statistical mechanics from invariance principles abstract the integral equation which identically relates the gradient of the density r and the ornstein zernike direction correlation function c r r is derived from a simple but fundamental invariance principle this generalization of the heuristic translational invariance argument used in the past establishes that this integral equation is valid even in the presence of intermolecular forces which are not translationally invariant the same methodology can be used to generate all the integral equations in the hierarchy of relations between r and higher order direct correlation functions it is also observed that as one can proceed up the hierarchy by simple functional differentiation it is also possible to proceed down the hierarchy in particular the generation of the first member of the hierarchy from the second provides a derivation of this relation which closely parallels the derivation of the first member of the born green yvon hierarchy the hierarchy of integral equations based upon rotational invariance is similarly developed

an extension of multiple correspondence analysis for identifying heterogeneous subgroups of respondents an extension of multiple correspondence analysis is proposed that takes into account cluster level heterogeneity in respondents preferences choices the method involves combining multiple correspondence analysis and k means in a unified framework the former is used for uncovering a low dimensional space of multivariate categorical variables while the latter is used for identifying relatively homogeneous clusters of respondents the proposed method offers an integrated graphical display that provides information on cluster based structures inherent in multivariate categorical data as well as the interdependencies among the data an empirical application is presented which demonstrates the usefulness of the proposed method and how it compares to several extant approaches

discrete time rescaling theorem determining goodness of fit for discrete time statistical models of neural spiking one approach for understanding the encoding of information by spike trains is to fit statistical models and then test their goodness of fit the time rescaling theorem provides a goodness of fit test consistent with the point process nature of spike trains the interspike intervals isis are rescaled as a function of the model s spike probability to be independent and exponentially distributed if the model is accurate a kolmogorov smirnov ks test between the rescaled isis and the exponential distribution is then used to check goodness of fit this rescaling relies on assumptions of continuously defined time and instantaneous events however spikes have finite width and statistical models of spike trains almost always discretize time into bins here we demonstrate that finite temporal resolution of discrete time models prevents their rescaled isis from being exponentially distributed poor goodness of fit may be erroneously indicated even if the model is exactly correct we present two adaptations of the time rescaling theorem to discrete time models in the first we propose that instead of assuming the rescaled times to be exponential the reference distribution be estimated through direct simulation by the fitted model in the second we prove a discrete time version of the time rescaling theorem that analytically corrects for the effects of finite resolution this allows us to define a rescaled time that is exponentially distributed even at arbitrary temporal discretizations we demonstrate the efficacy of both techniques by fitting generalized linear models to both simulated spike trains and spike trains recorded experimentally in monkey v1 cortex both techniques give nearly identical results reducing the false positive rate of the ks test and greatly increasing the reliability of model evaluation based on the time rescaling theorem

a nonparametric clustering algorithm with a quantile based likelihood estimator clustering is a representative of unsupervised learning and one of the important approaches in exploratory data analysis by its very nature clustering without strong assumption on data distribution is desirable information theoretic clustering is a class of clustering methods that optimize information theoretic quantities such as entropy and mutual information these quantities can be estimated in a nonparametric manner and information theoretic clustering algorithms are capable of capturing various intrinsic data structures it is also possible to estimate information theoretic quantities using a data set with sampling weight for each datum assuming the data set is sampled from a certain cluster and assigning different sampling weights depending on the clusters the cluster conditional information theoretic quantities are estimated in this letter a simple iterative clustering algorithm is proposed based on a nonparametric estimator of the log likelihood for weighted data sets the clustering algorithm is also derived from the principle of conditional entropy minimization with maximum entropy regularization the proposed algorithm does not contain a tuning parameter the algorithm is experimentally shown to be comparable to or outperform conventional nonparametric clustering methods

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

entropy satisfying schemes for computing selection dynamics in competitive interactions in this paper we present entropy satisfying schemes for solving an integro differential equation that describes the evolution of a population structured with respect to a continuous trait in p e jabin and g raoul j math biol 63 2011 pp 493 517 solutions are shown to converge toward the so called evolutionary stable distribution esd as time becomes large using the relative entropy at the discrete level the esd is shown to be the solution to a quadratic programming problem and can be computed by any well established nonlinear programing algorithm the schemes are then shown to satisfy the entropy dissipation inequality on the set where initial data are positive and the numerical solutions tend toward the discrete esd in time an alternative algorithm is presented to capture the global esd for nonnegative initial data which is made possible due to the mutation mechanism built into the modified scheme a series of numerical tests are given to confirm both accuracy and the entropy satisf

a semiparametric bayesian model for detecting synchrony among multiple neurons we propose a scalable semiparametric bayesian model to capture dependencies among multiple neurons by detecting their cofiring possibly with some lag time patterns over time after discretizing time so there is at most one spike at each interval the resulting sequence of 1s spike and 0s silence for each neuron is modeled using the logistic function of a continuous latent variable with a gaussian process prior for multiple neurons the corresponding marginal distributions are coupled to their joint probability distribution using a parametric copula model the advantages of our approach are as follows the nonparametric component i e the gaussian process model provides a flexible framework for modeling the underlying firing rates and the parametric component i e the copula model allows us to make inferences regarding both contemporaneous and lagged relationships among neurons using the copula model we construct multivariate probabilistic models by separating the modeling of univariate marginal distributions from the modeling of a dependence structure among variables our method is easy to implement using a computationally efficient sampling algorithm that can be easily extended to high dimensional problems using simulated data we show that our approach could correctly capture temporal dependencies in firing rates and identify synchronous neurons we also apply our model to spike train data obtained from prefrontal cortical areas

estimating the temporal interval entropy of neuronal discharge to better understand the role of timing in the function of the nervous system we have developed a methodology that allows the entropy of neuronal discharge activity to be estimated from a spike train record when it may be assumed that successive interspike intervals are temporally uncorrelated the so called interval entropy obtained by this methodology is based on an implicit enumeration of all possible spike trains that are statistically indistinguishable from a given spike train the interval entropy is calculated from an analytic distribution whose parameters are obtained by maximum likelihood estimation from the interval probability distribution associated with a given spike train we show that this approach reveals features of neuronal discharge not seen with two alternative methods of entropy estimation the methodology allows for validation of the obtained data models by calculation of confidence intervals for the parameters of the analytic distribution and the testing of the significance of the fit between the observed and analytic interval distributions by means of kolmogorov smirnov and anderson darling statistics the method is demonstrated by analysis of two different data sets simulated spike trains evoked by either poissonian or near synchronous pulsed activation of a model cerebellar purkinje neuron and spike trains obtained by extracellular recording from spontaneously discharging cultured rat hippocampal neurons

information capacity in the weak signal approximation we derive an approximate expression for mutual information in a broad class of discrete time stationary channels with continuous input under the constraint of vanishing input amplitude or power the approximation describes the input by its covariance matrix while the channel properties are described by the fisher information matrix this separation of input and channel properties allows us to analyze the optimality conditions in a convenient way we show that input correlations in memoryless channels do not affect channel capacity since their effect decreases fast with vanishing input amplitude or power on the other hand for channels with memory properly matching the input covariances to the dependence structure of the noise may lead to almost noiseless information transfer even for intermediate values of the noise correlations since many model systems described in mathematical neuroscience and biophysics operate in the high noise regime and weak signal conditions we believe that the described results are of potential interest also to researchers in these areas

dissipative conservative galerkin method using discrete partial derivatives for nonlinear evolution equations a new method is proposed for designing galerkin schemes that retain the energy dissipation or conservation properties of nonlinear evolution equations such as the cahn hilliard equation the korteweg de vries equation or the nonlinear schrodinger equation in particular as a special case dissipative or conservative finite element schemes can be derived the key device there is the new concept of discrete partial derivatives as examples of the application of the present method dissipative or conservative galerkin schemes are presented for the three equations with some numerical experiments

a direct derivation of the profile equations of buff lovett mou wertheim from the born green yvon equations for a non uniform equilibrium fluid abstract a functional approach to the born green yvon hierachy is developed out of the observation that the configuration space integral defining the free energy of a non uniform equilibrium fluid is invariant with respect to local coordinate transformations interpreting such a coordinate transformation as a deformation leads to the identification of this hierarchy as a force balance relation between the inter molecular forces and the external forces represinting the external constraints imposed on the system using the invariance for translations and rotations of the inter molecular forces it is shown that the profile equations of buff lovett mou wertheim can be obtained by taking appropriate moments of the born green yvon equations

a reproducing kernel hilbert space framework for spike train signal processing this letter presents a general framework based on reproducing kernel hilbert spaces rkhs to mathematically describe and manipulate spike trains the main idea is the definition of inner products to allow spike train signal processing from basic principles while incorporating their statistical description as point processes moreover because many inner products can be formulated a particular definition can be crafted to best fit an application these ideas are illustrated by the definition of a number of spike train inner products to further elicit the advantages of the rkhs framework a family of these inner products the cross intensity ci kernels is analyzed in detail this inner product family encapsulates the statistical description from the conditional intensity functions of spike trains the problem of their estimation is also addressed the simplest of the spike train kernels in this family provide an interesting perspective to others work as will be demonstrated in terms of spike train distance measures finally as an application example the rkhs framework is used to derive a clustering algorithm for spike trains from simple principles

comparison of two populations of curves with an application in neuronal data analysis often in neurophysiological studies scientists are interested in testing hypotheses regarding the equality of the overall intensity functions of a group of neurons when recorded under two different experimental conditions in this paper we consider such a hypothesis testing problem we propose two test statistics a parametric test similar to the modified hotelling s t2 statistic of behseta and kass statist med 2005 24 3523 3534 as well as a nonparametric one similar to the spatial signed rank test statistic of mottonen and oja j nonparametric statist 1995 5 201 213 we implement these tests on smooth curves obtained via fitting bayesian adaptive regression splines bars to the intensity functions of neuronal peri stimulus time histograms through simulation we show that the powers of our proposed tests are extremely high even when the number of sampled neurons and the number of trials per neuron are small finally we apply our methods on a group of motor cortex neurons recorded during a reaching task

pac bayesian analysis of co clustering and beyond we derive pac bayesian generalization bounds for supervised and unsupervised learning models based on clustering such as co clustering matrix tri factorization graphical models graph clustering and pairwise clustering we begin with the analysis of co clustering which is a widely used approach to the analysis of data matrices we distinguish among two tasks in matrix data analysis discriminative prediction of the missing entries in data matrices and estimation of the joint probability distribution of row and column variables in co occurrence matrices we derive pac bayesian generalization bounds for the expected out of sample performance of co clustering based solutions for these two tasks the analysis yields regularization terms that were absent in the previous formulations of co clustering the bounds suggest that the expected performance of co clustering is governed by a trade off between its empirical performance and the mutual information preserved by the cluster variables on row and column ids we derive an iterative projection algorithm for finding a local optimum of this trade off for discriminative prediction tasks this algorithm achieved state of the art performance in the movielens collaborative filtering task our co clustering model can also be seen as matrix tri factorization and the results provide generalization bounds regularization terms and new algorithms for this form of matrix factorization the analysis of co clustering is extended to tree shaped graphical models which can be used to analyze high dimensional tensors according to the bounds the generalization abilities of tree shaped graphical models depend on a trade off between their empirical data fit and the mutual information that is propagated up the tree levels we also formulate weighted graph clustering as a prediction problem given a subset of edge weights we analyze the ability of graph clustering to predict the remaining edge weights the analysis of co clustering easily extends to this problem and suggests that graph clustering should optimize the trade off between empirical data fit and the mutual information that clusters preserve on graph nodes

optimal pruning with applications to tree structured source coding and modeling an algorithm introduced by l breiman et al 1984 in the context of classification and regression trees is reinterpreted and extended to cover a variety of applications in source coding and modeling in which trees are involved these include variable rate and minimum entropy tree structured vector quantization minimum expected cost decision trees variable order markov modeling optimum bit allocation and computer graphics and image processing using quadtrees a concentration on the first of these and a detailed analysis of variable rate tree structured vector quantization are provided it is found that variable rate tree structured vector quantization outperforms not only the fixed rate variety but also full search vector quantization the successive approximation character of variable rate tree structured vector quantization permits it to degrade gracefully if the rate is reduced at the encoder this has applications to the problem of buffer overflow

learning and generalization with the information bottleneck the information bottleneck is an information theoretic framework that finds concise representations for an input random variable that are as relevant as possible for an output random variable this framework has been used successfully in various supervised and unsupervised applications however its learning theoretic properties and justification remained unclear as it differs from standard learning models in several crucial aspects primarily its explicit reliance on the joint input output distribution in practice an empirical plug in estimate of the underlying distribution has been used so far without any finite sample performance guarantees in this paper we present several formal results that address these difficulties we prove several finite sample bounds which show that the information bottleneck can provide concise representations with good generalization based on smaller sample sizes than needed to estimate the underlying distribution the bounds are non uniform and adaptive to the complexity of the specific model chosen based on these results we also present a preliminary analysis on the possibility of analyzing the information bottleneck method as a learning algorithm in the familiar performance complexity tradeoff framework in addition we formally describe the connection between the information bottleneck and minimal sufficient statistics

quantization based clustering algorithm in this paper a quantization based clustering algorithm qbca is proposed to cluster a large number of data points efficiently unlike previous clustering algorithms qbca places more emphasis on the computation time of the algorithm specifically qbca first assigns the data points to a set of histogram bins by a quantization function then it determines the initial centers of the clusters according to this point distribution finally qbca performs clustering at the histogram bin level rather than the data point level we also propose two approaches to improve the performance of qbca further i a shrinking process is performed on the histogram bins to reduce the number of distance computations and ii a hierarchical structure is constructed to perform efficient indexing on the histogram bins finally we analyze the performance of qbca theoretically and experimentally and show that the approach 1 can be easily implemented 2 identifies the clusters effectively and 3 outperforms most of the current state of the art clustering approaches in terms of efficiency

entropy energy stable schemes for evolutionary dispersal models in this paper we propose some entropy energy stable finite difference schemes for the reaction diffusion advection equation arising in the evolution of biased dispersal of population dynamics the peculiar feature of these active dispersal models is that the transient solution converges to the stable steady state when time goes to infinity for the numerical method to capture the long time pattern of persistence or extinction we use the relative entropy when the resource potential is logarithmic and explore the usual energy for other resource potentials the present schemes are shown to satisfy three important properties of the continuous model for the population density i positivity preserving ii equilibrium preserving and iii entropy or energy satisfying these ensure that our schemes provide a satisfying long time behavior thus revealing the desired dispersal pattern moreover we present several numerical results which confirm the second order accuracy for various resource potentials and underline the efficiency to preserve the large time asymptotic

concentration phenomena of a semilinear elliptic equation with large advection in an ecological model abstract we consider a reaction diffusion advection equation arising from a biological model of migrating species the qualitative properties of the globally attracting solution are studied and in some cases the limiting profile is determined in particular a conjecture of cantrell cosner and lou on concentration phenomena is resolved under mild conditions applications to a related parabolic competition system are also discussed

infinite boundary corrections for the lmbw equations and the tz formula of surface tension in the presence of spontaneous symmetry breaking abstract we investigate on a broad scale the so called lmbw hierarchy and various consequences thereof in the presence of spontaneous symmetry breaking and long range correlations we show by carefully performing the thermodynamic limit that in general additional boundary terms will survive which are usually brushed under the carpet and which in some applications may even be infinite we discuss the necessary modifications and make some sketchy remarks as to the relevance of our observations for the melting freezing problem the bulk of the practical applications consists however in a detailed analysis of the various derivations of the triezenberg zwanzig form of surface tension and its equivalence with the older kirkwood buff expression we show that in general tz is only one contribution occurring in kb the others being boundary terms which in some cases happen to be infinite it turns out that in that situation tz is ill defined and singular as well with the infinites of tz and the boundary contributions just cancelling each other yielding a well defined kb

mib using mutual information for biclustering gene expression data result of any biclustering or clustering algorithm depends on the choice of the similarity measure most of the biclustering algorithms are based on euclidean distance or correlation coefficient these measures capture only linear relationships between the genes but nonlinear dependencies may exist amongst them in this paper we propose an approach using mutual information for biclustering gene expression data mutual information is a more general measure to investigate relationships positive negative correlation and nonlinear relationships as well to the best of our knowledge none of the existing algorithms for biclustering have used mutual information as a similarity measure between two genes we obtained biclusters from the gene expression data of arabidopsis thaliana and compared our biclusters with those obtained by two other algorithms namely isa and bimax biological significance of the biclusters was checked using go database it was found that the genes belonging to our biclusters were significantly enriched with go terms with better p values as compared to the genes of the biclusters obtained by the other two algorithms to further investigate the biclusters we studied the promoter regions of the genes belonging to a bicluster for common patterns transcription factor binding sites tfbs or motifs promoter regions of the genes of most of our biclusters were found to have a common motif patterns which existed in the motif database of arabidopsis thaliana also the motifs extracted from our biclusters had better e values than those of others thus reconfirming that use of mutual information as a similarity measure will produce better biclusters

approximating the ideal free distribution via reaction diffusion advection equations abstract we consider reaction diffusion advection models for spatially distributed populations that have a tendency to disperse up the gradient of fitness where fitness is defined as a logistic local population growth rate we show that in temporally constant but spatially varying environments such populations have equilibrium distributions that can approximate those that would be predicted by a version of the ideal free distribution incorporating population dynamics the modeling approach shows that a dispersal mechanism based on local information about the environment and population density can approximate the ideal free distribution the analysis suggests that such a dispersal mechanism may sometimes be advantageous because it allows populations to approximately track resource availability the models are quasilinear parabolic equations with nonlinear boundary conditions

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

supervised learning of quantizer codebooks by information loss minimization this paper proposes a technique for jointly quantizing continuous features and the posterior distributions of their class labels based on minimizing empirical information loss such that the quantizer index of a given feature vector approximates a sufficient statistic for its class label informally the quantized representation retains as much information as possible for classifying the feature vector correctly we derive an alternating minimization procedure for simultaneously learning codebooks in the euclidean feature space and in the simplex of posterior class distributions the resulting quantizer can be used to encode unlabeled points outside the training set and to predict their posterior class distributions and has an elegant interpretation in terms of lossless source coding the proposed method is validated on synthetic and real data sets and is applied to two diverse problems learning discriminative visual vocabularies for bag of features image classification and image segmentation

bayesian curve fitting with free knot splines we describe a bayesian method for fitting curves to data drawn from an exponential family that uses splines for which the number and locations of knots are free parameters the method uses reversible jump markov chain monte carlo to change the knot configurations and a locality heuristic to speed up mixing for nonnormal models we approximate the integrated likelihood ratios needed to compute acceptance probabilities by using the bayesian information criterion bic under priors that make this approximation accurate our technique is based on a marginalised chain on the knot number and locations but we provide methods for inference about the regression coefficients and functions of them in both normal and nonnormal models simulation results suggest that the method performs well and we illustrate the method in two neuroscience applications

a computationally efficient method for nonparametric modeling of neural spiking activity with point processes point process models have been shown to be useful in characterizing neural spiking activity as a function of extrinsic and intrinsic factors most point process models of neural activity are parametric as they are often efficiently computable however if the actual point process does not lie in the assumed parametric class of functions misleading inferences can arise nonparametric methods are attractive due to fewer assumptions but computation in general grows with the size of the data we propose a computationally efficient method for nonparametric maximum likelihood estimation when the conditional intensity function which characterizes the point process in its entirety is assumed to be a lipschitz continuous function but otherwise arbitrary we show that by exploiting much structure the problem becomes efficiently solvable we next demonstrate a model selection procedure to estimate the lipshitz parameter from data akin to the minimum description length principle and demonstrate consistency of our estimator under appropriate assumptions finally we illustrate the effectiveness of our method with simulated neural spiking data goldfish retinal ganglion neural data and activity recorded in ca1 hippocampal neurons from an awake behaving rat for the simulated data set our method uncovers a more compact representation of the conditional intensity function when it exists for the goldfish and rat neural data sets we show that our nonparametric method gives a superior absolute goodness of fit measure used for point processes than the most common parametric and splines based approaches

a finite element semi lagrangian explicit runge kutta chebyshev method for convection dominated reaction diffusion problems explicit runge kutta chebyshev methods have proved to be efficient for reaction diffusion problems of moderate stiffness in this paper we extend such an efficiency to convection dominated reaction diffusion problems by giving a formulation of these methods in a semi lagrangian framework using c0 finite elements of degree m 2 as the space discretization method we also study the convergence in the l2 norm of the methods proposed in this paper

optimal bi level quantization and its application to multilevel quantization a special form of optimal quantization optimal bilevel quantization is studied a fixed point method is embedded in a search scheme to find all the locally optimal bilevel quantizers resulting in an algorithm for computing the globally optimal bilevel quantizer some interesting relations between the optimal bilevel quantizer and the mean and the median of the density function p x are explored efficient algorithms for computing optimal bilevel quantizers are proposed the application of these results to optimal quantization in general is discussed

nonconvergence in logistic and poisson models for neural spiking generalized linear models are an increasingly common approach for spike train data analysis for the logistic and poisson models one possible difficulty is that iterative algorithms for computing parameter estimates may not converge because of certain data configurations for the logistic model these configurations are called complete and quasi complete separation we show that these features are likely to occur because of refractory periods of neurons we use an example to study how standard software deals with this difficulty for the poisson model we show that the same difficulties arise this time possibly due to bursting or specifics of the binning we characterize the nonconvergent configurations for both models show that they can be detected by linear programming methods and discuss possible remedies

a spike train probability model poisson processes usually provide adequate descriptions of the irregularity in neuron spike times after pooling the data across large numbers of trials as is done in constructing the peristimulus time histogram when probabilities are needed to describe the behavior of neurons within individual trials however poisson process models are often inadequate in principle an explicit formula gives the probability density of a single spike train in great generality but without additional assumptions the firing rate intensity function appearing in that formula cannot be estimated we propose a simple solution to this problem which is to assume that the time at which a neuron fires is determined probabilistically by and only by two quantities the experimental clock time and the elapsed time since the previous spike we show that this model can be fitted with standard methods and software and that it may used successfully to fit neuronal data

does movement toward better environments always benefit a population abstract we study the effects of advection along environmental gradients on logistic reaction diffusion models for population growth the local population growth rate is assumed to be spatially inhomogeneous and the advection is taken to be a multiple of the gradient of the local population growth rate it is also assumed that the boundary acts as a reflecting barrier to the population we show that the effects of such advection depend crucially on the shape of the habitat of the population if the habitat is convex the movement in the direction of the gradient of the growth rate is always beneficial to the population while such advection could be harmful for certain non convex habitats

annealing and the normalized n cut we describe an annealing procedure that computes the normalized n cut of a weighted graph g the first phase transition computes the solution of the approximate normalized 2 cut problem while the low temperature solution computes the normalized n cut the intermediate solutions provide a sequence of refinements of the 2 cut that can be used to split the data to k clusters with 2

information transmission using non poisson regular firing in many cortical areas neural spike trains do not follow a poisson process in this study we investigate a possible benefit of non poisson spiking for information transmission by studying the minimal rate fluctuation that can be detected by a bayesian estimator the idea is that an inhomogeneous poisson process may make it difficult for downstream decoders to resolve subtle changes in rate fluctuation but by using a more regular non poisson process the nervous system can make rate fluctuations easier to detect we evaluate the degree to which regular firing reduces the rate fluctuation detection threshold we find that the threshold for detection is reduced in proportion to the coefficient of variation of interspike intervals

regular article finite difference schemes for u t x α δg δu that inherit energy conservation or dissipation property we propose a new procedure for designing by rote finite difference schemes that inherit energy conservation or dissipation property from nonlinear partial differential equations such as the korteweg de vries kdv equation and the cahn hilliard equation the most important feature of our procedure is a rigorous discretization of variational derivatives using summation by parts which implies that the inherited properties are satisfied exactly since the inherited properties are kept even if the time mesh size changes in the time evolution process we can use some appropriate time mesh adaptive methods to obtain numerical solutions through the derived schemes because of these properties the derived schemes are expected to be numerically stable and yield solutions converging to pde solutions and sufficiently flexible to treat the inheritance of the energy conservation and dissipation properties are verified numerically for the kdv equation and the cahn hilliard equation

analysis of operator splitting for advection diffusion reaction problems from air pollution modelling abstract operator or time splitting is often used in the numerical solution of initial boundary value problems for differential equations it is for example standard practice in computational air pollution modelling where we encounter systems of three dimensional time dependent partial differential equations of the advection diffusion reaction type for such systems little attention has been devoted to the analysis of splitting and to the question why splitting can work so well from the theoretical point of view the success of splitting is primarily determined by the splitting error this paper presents an analysis of operator splitting aimed at providing insight into the splitting error using the lie operator formalism a general expression is derived for a three term strang splitting in the pure initial value case for a class of advection diffusion reaction problems the splitting error is analyzed in greater detail a special case is discussed in which the splitting error can be reduced also some attention is paid to the use of operator splitting in initial boundary value problems

applying the multivariate time rescaling theorem to neural population models statistical models of neural activity are integral to modern neuroscience recently interest has grown in modeling the spiking activity of populations of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing however any statistical model must be validated by an appropriate goodness of fit test kolmogorov smirnov tests based on the time rescaling theorem have proven to be useful for evaluating point process based statistical models of single neuron spike trains here we discuss the extension of the time rescaling theorem to the multivariate neural population case we show that even in the presence of strong correlations between spike trains models that neglect couplings between neurons can be erroneously passed by the univariate time rescaling test we present the multivariate version of the time rescaling theorem and provide a practical step by step procedure for applying it to testing the sufficiency of neural population models using several simple analytically tractable models and more complex simulated and real data sets we demonstrate that important features of the population activity can be detected only using the multivariate extension of the test

a general framework for deriving integral preserving numerical methods for pdes a general procedure for constructing conservative numerical integrators for time dependent partial differential equations is presented in particular linearly implicit methods preserving a time discretized version of the invariant are developed for systems of partial differential equations with polynomial nonlinearities the framework is rather general and allows for an arbitrary number of dependent and independent variables with derivatives of any order it is proved formally that second order convergence is obtained the procedure is applied to a test case and numerical experiments are provided

rate distortion theory of finite point processes we study the compression of data in the case where the useful information is contained in a set rather than a vector i e the ordering of the data points is irrelevant and the number of data points is unknown our analysis is based on rate distortion theory and the theory of finite point processes we introduce fundamental information theoretic concepts and quantities for point processes and present general lower and upper bounds on the rate distortion function to enable a comparison with the vector setting we concretize our bounds for point processes of fixed cardinality in particular we analyze a fixed number of unordered gaussian data points and show that we can significantly reduce the required rates compared to the best possible compression strategy for gaussian vectors as an example of point processes with variable cardinality we study the best possible compression of poisson point processes for the specific case of a poisson point process with uniform intensity on the unit square our lower and upper bounds are separated by only a small gap and thus provide a good characterization of the rate distortion function

effective sampling and learning for mallows models with pairwise preference data learning preference distributions is a critical problem in many areas e g recommender systems ir social choice however many existing learning and inference methods impose restrictive assumptions on the form of user preferences that can be admitted as evidence we relax these restrictions by considering as data arbitrary pairwise comparisons of alternatives which represent the fundamental building blocks of ordinal rankings we develop the first algorithms for learning mallows models and mixtures thereof from pairwise comparison data at the heart of our technique is a new algorithm the generalized repeated insertion model grim which allows sampling from arbitrary ranking distributions and conditional mallows models in particular while we show that sampling from a mallows model with pairwise evidence is computationally difficult in general we develop approximate samplers that are exact for many important special cases and have provable bounds with pairwise evidence and derive algorithms for evaluating log likelihood learning mallows mixtures and non parametric estimation experiments on real world data sets demonstrate the effectiveness of our approach

the time rescaling theorem and its application to neural spike train data analysis measuring agreement between a statistical model and a spike train data series that is evaluating goodness of fit is crucial for establishing the model s validity prior to using it to make inferences about a particular neural system assessing goodness of fit is a challenging problem for point process neural spike train models especially for histogram based models such as perstimulus time histograms psth and rate functions estimated by spike train smoothing the time rescaling theorem is a well known result in probability theory which states that any point process with an integrable conditional intensity function may be transformed into a poisson process with unit rate we describe how the theorem may be used to develop goodness of fit tests for both parametric and histogram based point process models of neural spike trains we apply these tests in two examples a comparison of psth inhomogeneous poisson and inhomogeneous markov interval models of neural spike trains from the supplementary eye field of a macque monkey and a comparison of temporal and spatial smoothers inhomogeneous poisson inhomogeneous gamma and inhomogeneous inverse gaussian models of rat hippocampal place cell spiking activity to help make the logic behind the time rescaling theorem more accessible to researchers in neuroscience we present a proof using only elementary probability theory arguments we also show how the theorem may be used to simulate a general point process model of a spike train our paradigm makes it possible to compare parametric and histogram based neural spike train models directly these results suggest that the time rescaling theorem can be a valuable tool for neural spike train data analysis

spline based non parametric regression for periodic functions and its application to directional tuning of neurons the activity of neurons in the brain often varies systematically with some quantitative feature of a stimulus or action a well known example is the tendency of the firing rates of neurons in the primary motor cortex to vary with the direction of a subject s arm or wrist movement when this movement is constrained to vary in only two dimensions the direction of movement may be characterized by an angle and the neuronal firing rate can be written as a function of this angle the firing rate function has traditionally been fit with a cosine but recent evidence suggests that departures from cosine tuning occur frequently we report here a new non parametric regression method for fitting periodic functions and demonstrate its application to the fitting of neuronal data the method is an extension of bayesian adaptive regression splines bars and applies both to normal and non normal data including poisson data which commonly arise in neuronal applications we compare the new method to a periodic version of smoothing splines and some parametric alternatives and find the new method to be especially valuable when the smoothness of the periodic function varies unevenly across its domain

finite difference calculus invariant structure of a class of algorithms for the nonlinear klein gordon equation in a previous work the authors have presented a formalism for deriving systematically invariant symmetric finite difference algorithms for nonlinear evolution differential equations that admit conserved quantities this formalism is herein cast in the context of exact finite difference calculus the algorithms obtained from the proposed formalism are shown to derive exactly from discrete scalar potential functions using finite difference calculus in the same sense as that of the corresponding differential equation being derivable from its associated energy function a conserved quantity a clear ramification of this result is that the derived algorithms preserve certain discrete invariant quantities which are the consistent counterpart of the invariant quantities in the continuous case results on the nonlinear stability of a class of algorithms that are derived using the proposed formalism and that preserve energy or linear momentum are discussed in the context of finite difference calculus some

an adjustment to the time rescaling method for application to short trial spike train data it is important to validate models of neural data using appropriate goodness of fit measures models summarizing some response features for example spike count distributions or peristimulus time histograms can be assessed using standard statistical tools measuring the fit of a full point process model of spike trains is more difficult recently barbieri quirk frank wilson and brown 2001 and brown barbieri ventura kass and frank 2002 presented a method for rescaling time so that if an underlying description correctly describes the conditional intensity function of a point process the rescaling will convert the process into a homogeneous poisson process the corresponding interevent intervals are exponential with mean 1 and can be transformed to be uniform tests of the uniformity of the transformed intervals are thus tests of how well the model fits the data when the lengths of interevent intervals are comparable to the length of the observation window as can happen in common neurophysiology paradigms using short trials the fact that long intervals cannot be observed are censored can cause the tests based on time rescaling to reject a correct model inappropriately this article presents a simple adjustment to the time rescaling method to account for interval censoring avoiding inappropriate rejection of acceptable models for short trial data we illustrate the adjustment s effect using both simulated data and short trial data from monkey primary visual cortex

a theory of proximity based clustering structure detection by optimization abstract in this paper a systematic optimization approach for clustering proximity or similarity data is developed starting from fundamental invariance and robustness properties a set of axioms is proposed and discussed to distinguish different cluster compactness and separation criteria the approach covers the case of sparse proximity matrices and is extended to nested partitionings for hierarchical data clustering to solve the associated optimization problems a rigorous mathematical framework for deterministic annealing and mean field approximation is presented efficient optimization heuristics are derived in a canonical way which also clarifies the relation to stochastic optimization by gibbs sampling similarity based clustering techniques have a broad range of possible applications in computer vision pattern recognition and data analysis as a major practical application we present a novel approach to the problem of unsupervised texture segmentation which relies on statistical tests as a measure of homogeneity the quality of the algorithms is empirically evaluated on a large collection of brodatz like micro texture mondrians and on a set of real word images to demonstrate the broad usefulness of the theory of proximity based clustering the performances of different criteria and algorithms are compared on an information retrieval task for a document database the superiority of optimization algorithms for clustering is supported by extensive experiments

a statistical study of cochlear nerve discharge patterns in response to complex speech stimuli cochlear nerve discharge patterns in response to the synthesized consonant vowel stimulus da were collected from a population of 223 auditory nerve fibers from a single cat for each nerve fiber discharges were measured from multiple independent stimulus presentations with the means and variances of the post stimulus time histograms and fourier transforms of response generated from the ensemble of stimulus presentations the statistics were not consistent with those predicted via an inhomogeneous poisson counting process model specifically the synchronized components as measured by the fourier transforms of post stimulus time histogram responses have variances that are as much as a factor of 3 times lower than that predicted by the poisson model to account for the non poisson nature of the statistics the markov process model of siebert gaumond was adopted using the maximum likelihood and minimum description length algorithms introduced by miller j acoust soc am 77 1452 1464 1985 and ma

an energy conserving galerkin scheme for a class of nonlinear dispersive equations a galerkin scheme is presented for a class of conservative nonlinear dispersive equations such as the camassa holm equation and the regularized long wave equation the scheme has two advantageous features first it is conservative in that it keeps the discrete analogue of the continuous energy conservation property in the original equations second it can be formulated only with cheap h 1 elements even if the original equations include third derivative u x x x numerical experiments confirm the stability and effectiveness of the proposed scheme

population stratification using a statistical model on hypergraphs population stratification is a problem encountered in several areas of natural science engineering and public health we tackle this problem by mapping a population and its element attributes onto a hypergraph a natural extension of the concept of graph or network to encode associations among any number of elements on this hypergraph we construct a statistical model reflecting our intuition about how the element attributes can emerge from a postulated population structure finally we introduce the concept of stratification representativeness as a mean to identify the simplest stratification already containing most of the information about the population structure we demonstrate the power of this framework stratifying an animal and a human population based on phenotypic and genotypic properties respectively

studying amphiphilic self assembly with soft coarse grained models highly coarse grained models for investigating the self assembly of lipids and copolymer materials are discussed soft interactions between segments that represent many atoms naturally arise in the course of systematic coarse graining and they are necessary for modeling fluctuation effects whose strengths is dictated by a large invariant degree of polymerization the soft non bonded interactions of the coarse grained models are related to the excess free energy functional of an equivalent field theoretic description the connection between the particle based model and the field theoretic description helps to identify the physical significance of the model interactions non bonded interactions which describe the complex phase behavior of compressible mixtures or include local fluid like packing effects of the coarse grained segments can be systematically constructed based on liquid state theory or classical density functional theory details of the computational implementation and limitations of soft coarse grained models are discussed two computational techniques field theoretic force matching and umbrella sampling are devised for computing a free energy functional from a particle based description they can be employed to i derive the non bonded free energy functional of a soft coarse grained model from a more detailed computational model or to ii derive a field theoretic description from a particle based model moreover different strategies for accurately calculating free energies of self assembled systems are described and selected applications presented

information based clustering in an age of increasingly large data sets investigators in many different disciplines have turned to clustering as a tool for data analysis and exploration existing clustering methods however typically depend on several nontrivial assumptions about the structure of data here we reformulate the clustering problem from an information theoretic perspective that avoids many of these assumptions in particular our formulation obviates the need for defining a cluster prototype does not require an a priori similarity metric is invariant to changes in the representation of the data and naturally captures nonlinear relations we apply this approach to different domains and find that it consistently produces clusters that are more coherent than those extracted by existing algorithms finally our approach provides a way of clustering based on collective notions of similarity rather than the traditional pairwise measures

lautum information a popular way to measure the degree of dependence between two random objects is by their mutual information defined as the divergence between the joint and product of marginal distributions we investigate an alternative measure of dependence the lautum information defined as the divergence between the product of marginal and joint distributions i e swapping the arguments in the definition of mutual information some operational characterizations and properties are provided for this alternative measure of information