distribution estimation consistent in total variation and in two types of information divergence the problem of the nonparametric estimation of a probability distribution is considered from three viewpoints the consistency in total variation the consistency in information divergence and consistency in reversed order information divergence these types of consistencies are relatively strong criteria of convergence and a probability distribution cannot be consistently estimated in either type of convergence without any restrictions on the class of probability distributions allowed histogram based estimators of distribution are presented which under certain conditions converge in total variation in information divergence and in reversed order information divergence to the unknown probability distribution some a priori information about the true probability distribution is assumed in each case as the concept of consistency in information divergence is stronger than that of convergence in total variation additional assumptions are imposed in the cases of informational divergences

statistical consistency of kernel canonical correlation analysis while kernel canonical correlation analysis cca has been applied in many contexts the convergence of finite sample estimates of the associated functions to their population counterparts has not yet been established this paper gives a mathematical proof of the statistical convergence of kernel cca providing a theoretical justification for the method the proof uses covariance operators defined on reproducing kernel hilbert spaces and analyzes the convergence of their empirical estimates of finite rank to their population counterparts which can have infinite rank the result also gives a sufficient condition for convergence on the regularization coefficient involved in kernel cca this should decrease as n 1 3 where n is the number of data

some multivariate tests of independence based on ranks of nearest neighbors abstractseveral parametric and nonparametric tests of independence between two random vectors are available in the literature but many of them perform poorly for high dimensional data and are not applicable when the dimension exceeds the sample size in this article we propose some tests based on ranks of nearest neighbors which can be conveniently used in high dimension low sample size situations several simulated and real data sets are analyzed to show the utility of the proposed tests codes for implementation of the proposed tests are available as supplementary materials

bootstrap and permutation tests of independence for point processes motivated by a neuroscience question about synchrony detection in spike train analysis we deal with the independence testing problem for point processes we introduce non parametric test statistics which are rescaled general u statistics whose corresponding critical values are constructed from bootstrap and randomization permutation approaches making as few assumptions as possible on the underlying distribution of the point processes we derive general consistency results for the bootstrap and for the permutation w r t to wasserstein s metric which induce weak convergence as well as convergence of second order moments the obtained bootstrap or permutation independence tests are thus proved to be asymptotically of the prescribed size and to be consistent against any reasonable alternative a simulation study is performed to illustrate the derived theoretical results and to compare the performance of our new tests with existing ones in the neuroscientific literature

consistent distribution free k sample and independence tests for univariate random variables a popular approach for testing if two univariate random variables are statistically independent consists of partitioning the sample space into bins and evaluating a test statistic on the binned data the partition size matters and the optimal partition size is data dependent while for detecting simple relationships coarse partitions may be best for detecting complex relationships a great gain in power can be achieved by considering finer partitions we suggest novel consistent distribution free tests that are based on summation or maximization aggregation of scores over all partitions of a fixed size we show that our test statistics based on summation can serve as good estimators of the mutual information moreover we suggest regularized tests that aggregate over all partition sizes and prove those are consistent too we provide polynomial time algorithms which are critical for computing the suggested test statistics efficiently we show that the power of the regularized tests is excellent compared to existing tests and almost as powerful as the tests based on the optimal yet unknown in practice partition size in simulations as well as on a real data example

multi order power spectrum and its application abstract a new technique was explored to investigate the time correlations in time series this is an extension of the ordinary power spectrum which is called the order q power spectrum i have extended the order q power spectrum further and made the multi order version of it this technique is able to be applied to various processes which have more than two states this approach is applied to a chaotic system which exhibits intermittent switching and shows us a surprising result appropriate choices for parameters show us various correlations that are not able to be observed in the ordinary power spectra

a fast algorithm for optimally increasing the edge connectivity let g v e be an undirected unweighted graph with n nodes m edges and edge connectivity lambda given an input parameter delta the edge augmentation problem is to find the smallest set of edges to add to g so that its edge connectivity is increased by delta in this paper we present a solution to this problem which runs in o delta 2 nm delta 3 n 2 n f g where f g is the time to perform one maximum flow on g in fact our solution gives the optimal augmentation for every delta 1 le delta le delta in the same time bound by introducing minor modifications to the solution we can solve the problem without knowing delta in advance and we can also solve the node weighted version and the degree constrained version of the problem if delta 1 then our solution is particularly simple it runs in o nm time and it is a natural generalization of the algorithm in k eswaran and r e tarjan siam j comput 5 1976 pp 653 665 for the case where lambda delta 2 we also solve the converse problem in the same time bound given an input number k increase the connectivity of g as much as possible by adding at most k edges our solution makes extensive use of the structure of particular sets of cuts

kernel methods for measuring independence we introduce two new functionals the constrained covariance and the kernel mutual information to measure the degree of independence of random variables these quantities are both based on the covariance between functions of the random variables in reproducing kernel hilbert spaces rkhss we prove that when the rkhss are universal both functionals are zero if and only if the random variables are pairwise independent we also show that the kernel mutual information is an upper bound near independence on the parzen window estimate of the mutual information analogous results apply for two correlation based dependence functionals introduced earlier we show the kernel canonical correlation and the kernel generalised variance to be independence measures for universal kernels and prove the latter to be an upper bound on the mutual information near independence the performance of the kernel dependence functionals in measuring independence is verified in the context of independent component analysis

kernel distribution embeddings universal kernels characteristic kernels and kernel metrics on distributions kernel mean embeddings have become a popular tool in machine learning they map probability measures to functions in a reproducing kernel hilbert space the distance between two mapped measures defines a semi distance over the probability measures known as the maximum mean discrepancy mmd its properties depend on the underlying kernel and have been linked to three fundamental concepts of the kernel literature universal characteristic and strictly positive definite kernels the contributions of this paper are three fold first by slightly extending the usual definitions of universal characteristic and strictly positive definite kernels we show that these three concepts are essentially equivalent second we give the first complete characterization of those kernels whose associated mmd distance metrizes the weak convergence of probability measures third we show that kernel mean embeddings can be extended from probability measures to generalized measures called schwartz distributions and analyze a few properties of these distribution embeddings

scaling structures for dynamics of intermittent switching just after the three band crisis abstract the statistical properties of temporal fluctuations just after the three band crisis of the logistic map are studied by means of the fluctuation spectrum and the order q time correlation function we obtain the relevant characteristic functions in several ways remarkable intermittent characteristics are shown to be described as scaling laws of those functions the scaling laws lead to the self similarity of temporal fluctuations near the band crisis point

amortized efficiency of a path retrieval data structure a data structure is presented for the problem of maintaining a digraph under an arbitrary sequence of two kinds of operations an add operation that inserts an arc in the digraph and a searchpath operation that checks the presence of a path between a pair of nodes our data structure supports both operations in o n amortized time and requires o n2 space where n is the number of nodes in the digraph

on the thermodynamic approach towards time series analysis a scheme is presented which allows for the computation of the topological pressure and power spectra associated with equilibrium measures from a given time series its convergence properties are demonstrated on numerical data sets obtained from the lozi and the henon map numerical evidence for sharp peaks in the power spectrum of the non hyperbolic phase is given and supported by estimates using periodic orbit expansions

directlingam a direct method for learning a linear non gaussian structural equation model structural equation models and bayesian networks have been widely used to analyze causal relations between continuous variables in such frameworks linear acyclic models are typically used to model the data generating process of variables recently it was shown that use of non gaussianity identifies the full structure of a linear acyclic model that is a causal ordering of variables and their connection strengths without using any prior knowledge on the network structure which is not the case with conventional methods however existing estimation methods are based on iterative search algorithms and may not converge to a correct solution in a finite number of steps in this paper we propose a new direct method to estimate a causal ordering and connection strengths based on non gaussianity in contrast to the previous methods our algorithm requires no algorithmic parameters and is guaranteed to converge to the right solution within a small fixed number of steps if the data strictly follows the model that is if all the model assumptions are met and the sample size is infinite

kernel independent component analysis we present a class of algorithms for independent component analysis ica which use contrast functions based on canonical correlations in a reproducing kernel hilbert space on the one hand we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence on the other hand building on recent developments in kernel methods we show that these criteria and their derivatives can be computed efficiently minimizing these criteria leads to flexible and robust algorithms for ica we illustrate with simulations involving a wide variety of source distributions showing that our algorithms outperform many of the presently known algorithms

on the asymptotic properties of a nonparametric l sub 1 test statistic of homogeneity we present two simple and explicit procedures for testing homogeneity of two independent multivariate samples of size n the nonparametric tests are based on the statistic t sub n which is the l sub 1 distance between the two empirical distributions restricted to a finite partition both tests reject the null hypothesis of homogeneity if t sub n becomes large i e if t sub n exceeds a threshold we first discuss chernoff type large deviation properties of t sub n this results in a distribution free strong consistent test of homogeneity then the asymptotic null distribution of the test statistic is obtained leading to an asymptotically spl alpha level test procedure

measures of entropy from data using infinitely divisible kernels information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing clustering dimensionality reduction classification and so forth however the use of information theoretic quantities as test statistics that is as quantities obtained from empirical data poses a challenging estimation problem that often leads to strong simplifications such as gaussian models or the use of plug in density estimators that are restricted to certain representation of the data in this paper a framework to nonparametrically obtain measures of entropy directly from data using operators in reproducing kernel hilbert spaces defined by infinitely divisible kernels is presented the entropy functionals which bear resemblance with quantum entropies are defined on positive definite matrices and satisfy similar axioms to those of renyi s definition of entropy convergence of the proposed estimators follows from concentration results on the difference between the ordered spectrum of the gram matrices and the integral operators associated to the population quantities in this way capitalizing on both the axiomatic definition of entropy and on the representation power of positive definite kernels the proposed measure of entropy avoids the estimation of the probability distribution underlying the data moreover estimators of kernel based conditional entropy and mutual information are also defined numerical experiments on independence tests compare favorably with state of the art

the general structure of edge connectivity of a vertex subset in a graph and its incremental maintenance odd case let g v e be an undirected graph s be a subset of its vertices frak c s be the set of minimum edge cuts partitioning s and lambda s be the cardinality of such a cut we suggest a graph structure called the connectivity carcass of s that represents both cuts in frak c s and the partition of v by all these cuts its size is o min e lambda s v in this paper we present general constructions and study in detail the case lambda s odd the specifics of the case lambda s even are considered elsewhere for an adequate description of the connectivity carcass we introduce a new type of graph locally orientable graphs which generalize digraphs the connectivity carcass consists of a locally orientable quotient graph of g a cactus tree in case lambda s odd just a tree representing all distinct partitions of s by cuts in frak c s and a mapping connecting them one can build it in o s max flow computations in g for an arbitrary sequence of u edge insertions not changing lambda s the connectivity carcass can be maintained in time o v min e lambda s v u for two vertices of g queries asking whether they are separated by a cut in frak c s are answered in o 1 worst case time per query another possibility is to maintain the carcass in o s min e lambda s v u time but to answer the queries in o 1 time only if at least one of the vertices belongs to s

on moderate and large deviations in multinomial distributions on presente des theoremes pour les statistiques du rapport de vraisemblance et du khi deux de pearson

directed vertex connectivity augmentation the problem of finding a smallest set of new edges to be added to a given directed graph to make it k vertex connected was shown to be polynomially solvable recently in 6 for any target connectivity k 1 however the algorithm given there relied on the ellipsoid method here we refine some results of 6 which makes it possible to construct a combinatorial algorithm which is polynomial for any fixed k short proofs for extensions of some earlier results related to this problem will also be given

canonical correlation analysis when the data are curves summary it is not immediately straightforward to extend canonical correlation analysis to the context of functional data analysis where the data are themselves curves or functions the obvious approach breaks down and it is necessary to use a method involving smoothing in some way such a method is introduced and discussed with reference to a data set on human gait the breakdown of the unsmoothed method is illustrated in a practical context and is demonstrated theoretically a consistency theorem for the smoothed method is proved in an increasing number of problems in a wide range of fields the data observed are not the univariate or multivariate observations of classical statistics but are func tions observed continuously ramsay and dalzell 1991 give the name functional data analysis to the analysis of data of this kind in most cases the observations will be functions of time or a closely related variable but there are clearly applications where the functions are surfaces observed over two or three dimensional space the motivating example for the present paper is a set of data on children s gait collected by the motion analysis laboratory at children s hospital san diego california see olshen et al 1989 for full details for each of a number of children several angles made by the child s joints knee hip etc are observed during the child s gait cycle one aim of the study is to gain understanding of the gait cycle of a normal child to make comparisons with children suffering from walking diffi culties these data motivated rice and silverman 1991 to discuss the extension of principal component analysis to the functional setting and to explain how smoothing can be incorporated into the analysis in a natural way in a substantial complemen tary paper ramsay and dalzell 1991 discuss different approaches to regression and principal component analysis in functional data analysis illustrating their work by a meteorological example the insights of functional data analysis may be helpful in chemometrics for instance a standard chemometric problem is the analysis of spectra observed in chromatography and these are to all intents and purposes functional observations another obvious area of potential relevance is the analysis of growth curve data the focus in the present paper will be on canonical correlation analysis cca in the gait example one might ask how variability in the knee angle cycle is related to

uniformly powerful goodness of fit tests the simple hypothesis is tested that the distribution of independent random variables x1 x2 xn is a given probability measure po let 7r be any sequence of partitions the alternative hypothesis is the set of probability measures p with ea ip a po a i 8 where 8 0 note the dependence of this set of alternatives on the sample size it is shown that if the effective cardinality of the partitions is of the same order as the sample size then sequences of tests exist with uniformly exponentially small probabilities of error conversely if the effective cardinality is of larger order than the sample size then no such sequence of tests exists the effective cardinality is the number of sets in the partition which exhaust all but a negligible portion of the probability under the null hypothesis

nonlinear canonical analysis and independence tests measures of association between two random variables r v that are symmetric nondecreasing functions of the canonical coefficients provided by the nonlinear canonical analysis of the r v s are studied these measures can be used to characterize independence their estimators are obtained by estimating a suitable approximation to nonlinear canonical analysis when some conditions ae satisfied asymptotic distributions of the estimators both under the independence hypothesis and under dependence are given a class of tests of independence with asymptotic level of significance can be investigated 1 introduction several measures of association between two random variables r v based on the canonical analysis of these r v s have been introduced in the literature and a review of such measures can be found in cramer and nicewander 1979 and lazraq and cleroux 1988 a general enough study of these measures is proposed in lin 1987 and more recently in dauxois and nkiet 1997 where a global study of the induced tests is developed in these works the measures considered are constructed using linear canonical analysis lca of the r v s and thus characterize only lack of a linear relationship it can be interesting to look for another class of measures of association which characterize independence the properties of nonlinear canonical analysis nlca suggest that such a class can be derived from this analysis but there is little work in this direction the canonical coefficients from nlca have already been used for testing independence but uniquely when the related random variables are categorical see for instance tsai and sen 1990 that is restrictive because one knows that in this particular case nlca is a lca of suitable random vectors and thus it suffices to use classical measures of association consequently it appears that the use of nlca is more interesting when the related random variables are not categorical in this paper we propose a class of measures of association between random variables with values in any measurable spaces constructed using symmetric nondecreasing functions and the canonical coefficients derived from nlca the properties of these measures show that they are appropriate to characterize independence without any assumption on the distribution of

efficiencies of chi square and likelihood ratio goodness of fit tests the classical problem of choice of number of classes in testing goodness of fit is considered for a class of alternatives for the chi square and likelihood ratio statistics pitman and bahadur efficiencies are used to compare the two statistics and also to analyse the effect for each statistic of changing the number of classes for the case where the number of classes increases asymptotically with the number of observations overall the results suggest that if the class of alternatives is suitably restricted the number of classes should not be very large

equivalence of distance based and rkhs based statistics in hypothesis testing we provide a unifying framework linking two classes of statistics used in two sample and independence testing on the one hand the energy distances and distance covariances from the statistics literature on the other maximum mean discrepancies mmd that is distances between embeddings of distributions to reproducing kernel hilbert spaces rkhs as established in machine learning in the case where the energy distance is computed with a semimetric of negative type a positive definite kernel termed distance kernel may be defined such that the mmd corresponds exactly to the energy distance conversely for any positive definite kernel we can interpret the mmd as energy distance with respect to some negative type semimetric this equivalence readily extends to distance covariance using kernels on the product space we determine the class of probability distributions for which the test statistics are consistent against all alternatives finally we investigate the performance of the family of distance kernels in two sample and independence tests we show in particular that the energy distance most commonly employed in statistics is just one member of a parametric family of kernels and that other choices from this family can yield more powerful tests

green kubo formula derived from large deviation statistics abstract numerical calculations for many physical systems including chaos and turbulence have been greatly developed in recent years we often investigate the characteristics of the time series for chaotic systems and study the essential laws underlying its time series we show the important characteristics which are sometimes not pointed out one of them is the green kubo formula which holds for the time series that satisfies the large deviation statistics extended version for the relation is shown in this paper and a simple example is given the relationship between the green kubo formula and the large deviation statistics is emphasized

consistent nonparametric tests of independence three simple and explicit procedures for testing the independence of two multi dimensional random variables are described two of the associated test statistics l1 log likelihood are defined when the empirical distribution of the variables is restricted to finite partitions a third test statistic is defined as a kernel based independence measure two kinds of tests are provided distribution free strong consistent tests are derived on the basis of large deviation bounds on the test statistics these tests make almost surely no type i or type ii error after a random sample size asymptotically level tests are obtained from the limiting distribution of the test statistics for the latter tests the type i error converges to a fixed non zero value and the type ii error drops to zero for increasing sample size all tests reject the null hypothesis of independence if the test statistics become large the performance of the tests is evaluated experimentally on benchmark data

on the influence of the kernel on the consistency of support vector machines in this article we study the generalization abilities of several classifiers of support vector machine svm type using a certain class of kernels that we call universal it is shown that the soft margin algorithms with universal kernels are consistent for a large class of classification problems including some kind of noisy tasks provided that the regularization parameter is chosen well in particular we derive a simple sufficient condition for this parameter in the case of gaussian rbf kernels on the one hand our considerations are based on an investigation of an approximation property the so called universality of the used kernels that ensures that all continuous functions can be approximated by certain kernel expressions this approximation property also gives a new insight into the role of kernels in these and other algorithms on the other hand the results are achieved by a precise study of the underlying optimization problems of the classifiers furthermore we show consistency for the maximal margin classifier as well as for the soft margin svm s in the presence of large margins in this case it turns out that also constant regularization parameters ensure consistency for the soft margin svm s finally we prove that even for simple noise free classification problems svm s with polynomial kernels can behave arbitrarily badly