a note on bayes factors for log linear contingency table models with vague prior information spiegelhalter and smith 1982 hereafter ssproposed an approximate method for calculating the bayes factor for a log linear model mo for a contingency table against the saturated model m1 with vague prior information we adopt their notation and denote by ss n equation n of ss suppose x1 xk have a multinomial distribution with parameters 014 4k where xi 0 0 x4i 1 we write yt log x1 log xk 01 log 01 log dk and y diag x1 xk then if m1 is the saturated model and mo is the nested log linear model defined by setting the contrasts co1 0 where c is an s x k matrix with rank s and rows summing to zero the approximate bayes factor bo1 for mo against m1 is given by ss 32 this however is indeterminate if any of the cell frequencies in the table is zero this is because of the use by ss of a prior density proportional to h i if however we use instead the standard jeffreys prior density proportional to i1i 1 2 the problem no longer arises then by the arguments of ss and lindley 1964 the resulting bayes factor is still given by ss 32 with xi replaced by xi 2 in the definitions of y and y i 1 k and ss 33 replaced by c 1 2 s2 1 cct i 1 2 if this solution is adopted the prior is proper and so in principle the problem of assigning an arbitrary multiplicative constant for which the ss approach was primarily devised need not arise one could in theory simply apply bayes theorem directly and so obtain the bayes factor exactly however in practice this is difficult to do and i know of no general solution to the problem even for simpler more specific contingency table and related models such as those of independence or equiprobability finding the bayes factor exactly is not too easy see for example crook and good 1982 altham 1971 gunel and dickey 1974 gunel 1982 broniatowski 1981 and references therein the second purpose of this note is to point out that conditionally on mo

estimating the errors on measured entropy and mutual information abstract information entropy and the related quantity mutual information are used extensively as measures of complexity and to identify nonlinearity in dynamical systems expressions for the probability distribution of entropies and mutual informations calculated from finite amounts of data exist in the literature but the expressions have seldom been used in the field of nonlinear dynamics in this paper formulae for estimating the errors on observed information entropies and mutual informations are derived using the standard error analysis familiar to physicists their validity is demonstrated by numerical experiment for illustration the formulae are then used to evaluate the errors on the time lagged mutual information of the logistic map

assessing the order of dependence for partially exchangeable binary data abstract the problem we consider is how to assess the order of serial dependence within partially exchangeable binary sequences we obtain exact conditional tests comparing any two orders by finding the conditional distribution of data given certain transition counts these tests are facilitated with a new monte carlo scheme asymptotic tests are also discussed in particular we show that the likelihood ratio tests have an asymptotic chi squared distribution thus generalizing the results of billingsley for the particular case of markov chains we apply these methods to several datasets and perform a simulation to study their properties

information based detection of nonlinear granger causality in multivariate processes via a nonuniform embedding technique we present an approach framed in information theory to assess nonlinear causality between the subsystems of a whole stochastic or deterministic dynamical system the approach follows a sequential procedure for nonuniform embedding of multivariate time series whereby embedding vectors are built progressively on the basis of a minimization criterion applied to the entropy of the present state of the system conditioned to its past states a corrected conditional entropy estimator compensating for the biasing effect of single points in the quantized hyperspace is used to guarantee the existence of a minimum entropy rate at which to terminate the procedure the causal coupling is detected according to the granger notion of predictability improvement and is quantified in terms of information transfer we apply the approach to simulations of deterministic and stochastic systems showing its superiority over standard uniform embedding effects of quantization data length and noise contamination are investigated as practical applications we consider the assessment of cardiovascular regulatory mechanisms from the analysis of heart period arterial pressure and respiration time series and the investigation of the information flow across brain areas from multichannel scalp electroencephalographic recordings

self organized criticality resulting from minimization of perpetual disequilibration abstract a bootstrapping system is proposed in order to describe nonlogical properties of living systems in which the velocity of observation propagation is finite a cellular automaton ca model of the system combines two logics which are the a priori non boolean logic and the a posteriori boolean logic because of inconsistency between the two logics the system evolves perpetually generating disequilibrium and perpetually equilibrating the generated disequilibrium the system tends to self organize to the border between order and disorder and shows critical behaviours as class iv ca although the critical behaviors of the two systems are similar the bootstrapping ssstem has twice stronger undecidability than class iv ca or edge of chaos dynamics it is stressed that real living systems are not computable and should be described by strongly undecidable systems

can we neglect the multi layer structure of functional networks functional networks i e networks representing dynamic relationships between the components of a complex system have been instrumental for our understanding of among others the human brain due to limited data availability the multi layer nature of numerous functional networks has hitherto been neglected and nodes are endowed with a single type of links even when multiple relationships coexist at different physical levels a relevant problem is the assessment of the benefits yielded by studying a multi layer functional network against the simplicity guaranteed by the reconstruction and use of the corresponding single layer projection here i tackle this issue by using as a test case the functional network representing the dynamics of delay propagation through european airports neglecting the multi layer structure of a functional network has dramatic consequences on our understanding of the underlying system a fact to be taken into account when a projection is the only available information

exact significance test for markov order abstract we describe an exact significance test of the null hypothesis that a markov chain is n th order the procedure utilizes surrogate data to yield an exact test statistic distribution valid for any sample size surrogate data are generated using a novel algorithm that guarantees per shot a uniform sampling from the set of sequences that exactly match the n th order properties of the observed data using the test the markov order of tel aviv rainfall data is examined

persistent random motion with maximally correlated fluctuations how often should a random walker change its direction of motion in order to maximize correlation in velocity fluctuations over a finite time interval we address this optimal diffusion problem in the context of the one dimensional persistent random walk where we evaluate the correlation and mutual information in velocity trajectories as a function of the persistence level and the observation time we find the optimal persistence level corresponds to the average number of direction reversals asymptotically scaling as the square root of the observation time this square root scaling law makes the relative growth between the average number of direction reversals and the persistence length invariant with respect to changes in the overall time duration of the random walk

detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble a method to characterize dynamical interdependence among nonlinear systems is derived based on mutual nonlinear prediction systems with nonlinear correlation will show mutual nonlinear prediction when standard analysis with linear cross correlation might fail mutual nonlinear prediction also provides information on the directionality of the coupling between systems furthermore the existence of bidirectional mutual nonlinear prediction in unidirectionally coupled systems implies generalized synchrony numerical examples studied include three classes of unidirectionally coupled systems systems with identical parameters nonidentical parameters and stochastic driving of a nonlinear system this technique is then applied to the activity of motoneurons within a spinal cord motoneuron pool the interrelationships examined include single neuron unit firing the total number of neurons discharging at one time as measured by the integrated monosynaptic reflex and intracellular measurements of integrated excitatory postsynaptic potentials epsp s dynamical interdependence perhaps generalized synchrony was identified in this neuronal network between simultaneous single unit firings between units and the population and between units and intracellular epsp s

tsallis conditional mutual information in investigating long range correlation in symbol sequences abstract the presence of long range correlation lrc as opposed to markov chain of a finite order is of primary interest in the study of symbol sequences such as dna among others methods using information theory have been developed for markov chain order estimation in this work we consider the tsallis entropy and define the tsallis conditional mutual information tcmi for markov chain order estimation which is more suitable to identify large orders suggesting effectively lrc the tcmi of order m for a tsallis parameter q is computed for increasing m and a significance test is performed for each m until tcmi is found non significant randomization and parametric significance tests for tcmi are developed for the latter the null distribution of tcmi is approximated with a gamma distribution deriving analytic expressions for its parameters we assess the accuracy of order estimation with the two tests for tcmi and compare them to the respective tests using the shannon entropy extended simulations on markov chains of different orders and structures of the transition probability matrix show that shannon and tsallis tests with parameter q 2 have similar performance for small orders when the problem becomes more demanding for higher markov chain orders and lrcs tsallis tests for q 2 have better approximation to the correct order with the sequence length n finally the tsallis tests are favorably compared to shannon tests to real dna sequences

context tree based image contour coding using a geometric prior efficient encoding of object contours in images can facilitate advanced image video compression techniques such as shape adaptive transform coding or motion prediction of arbitrarily shaped pixel blocks we study the problem of lossless and lossy compression of detected contours in images specifically we first convert a detected object contour into a sequence of directional symbols drawn from a small alphabet to encode the symbol sequence using arithmetic coding we compute an optimal variable length context tree vct mathcal t via a maximum a posterior map formulation to estimate symbols conditional probabilities map can avoid overfitting given a small training set mathcal x of past symbol sequences by identifying a vct mathcal t with high likelihood p mathcal x mathcal t of observing mathcal x given mathcal t using a geometric prior p mathcal t stating that image contours are more often straight than curvy for the lossy case we design fast dynamic programming dp algorithms that optimally trade off coding rate of an approximate contour hat text x given a vct mathcal t with two notions of distortion of hat text x with respect to the original contour x to reduce the size of the dp tables a total suffix tree is derived from a given vct mathcal t for compact table entry indexing reducing complexity experimental results show that for lossless contour coding our proposed algorithm outperforms state of the art context based schemes consistently for both small and large training datasets for lossy contour coding our algorithms outperform comparable schemes in the literature in rate distortion performance

financial networks based on granger causality a case study connectivity analysis is performed on a long financial record of 21 international stock indices employing a linear and a nonlinear causality measure the conditional granger causality index cgci and the partial mutual information on mixed embedding pmime respectively both measures aim to specify the direction of the interrelationships among the international stock indexes and portray the links of the resulting networks by the presence of direct couplings between variables exploiting all available information however their differences are assessed due to the presence of nonlinearity the weighted networks formed with respect to the causality measures are transformed to binary ones using a significance test the financial networks are formed on sliding windows in order to examine the network characteristics and trace changes in the connectivity structure subsequently two statistical network quantities are calculated the average degree and the average shortest path length the empirical findings reveal interesting time varying properties of the constructed network which are clearly dependent on the nature of the financial cycle

uniform approach to linear and nonlinear interrelation patterns in multivariate time series currently a variety of linear and nonlinear measures is in use to investigate spatiotemporal interrelation patterns of multivariate time series whereas the former are by definition insensitive to nonlinear effects the latter detect both nonlinear and linear interrelation in the present contribution we employ a uniform surrogate based approach which is capable of disentangling interrelations that significantly exceed random effects and interrelations that significantly exceed linear correlation the bivariate version of the proposed framework is explored using a simple model allowing for separate tuning of coupling and nonlinearity of interrelation to demonstrate applicability of the approach to multivariate real world time series we investigate resting state functional magnetic resonance imaging rsfmri data of two healthy subjects as well as intracranial electroencephalograms ieeg of two epilepsy patients with focal onset seizures the main findings are that for our rsfmri data interrelations can be described by linear cross correlation rejection of the null hypothesis of linear ieeg interrelation occurs predominantly for epileptogenic tissue as well as during epileptic seizures

relative entropy and identification of gibbs measures in dynamical systems in this work we explore the idea of using the relative entropy of ergodic measures for the identification of gibbs measures in dynamical systems the question we face is how to estimate the thermodynamic potential together with a grammar from a sample produced by the corresponding gibbs state

information and entropy in strange attractors a technique for analyzing time series data from experiments is presented that provides estimates of four basic characteristics of a system 1 the measure theoretic entropy 2 the accuracy of the measurements 3 the number of measurements necessary to specify a system state and 4 the best delay time t to use in order to construct phase space portraits by the method of delays these characteristics are obtained by separating the entropy of measurements into a part due to noise and parts due to deterministic effects for the technique to work the noise associated with each measurement must be independent of the noise associated with all other measurements an algorithm for implementing the analysis is presented with three examples

on some criteria for estimating the order of a markov chain tong 1975 has proposed a procedure for estimating the order of a markov chain based on akaike s information criterion aic in this paper the asymptotic distribution of the aic estimator is derived and it is shown that the estimator is inconsistent as an alternative to the aic procedure the bayesian information criterion bic proposed by schwarz 1978 is shown to be consistent these two procedures yield different estimated orders when applied to specific samples of meteorological observations for parameters based on these meteorological examples the aic and bic procedures are compared by means of simulation for finite samples the results obtained have practical implications concerning whether in the routine fitting of precipitation data it is necessary to consider higher than first order markov chains

optimal embedding parameters a modelling paradigm the reconstruction of a dynamical system from a time series requires the selection of two parameters the embedding dimension de and the embedding lag many competing criteria to select these parameters exist and all are heuristic within the context of modelling the evolution operator of the underlying dynamical system we show that one only need be concerned with the product de we introduce an information theoretic criterion for the optimal selection of the embedding window dw de for infinitely long time series this method is equivalent to selecting the embedding lag that minimises the nonlinear model prediction error for short and noisy time series we find that the results of this new algorithm are data dependent and are superior to estimation of embedding parameters with the standard techniques

testing the order of discrete markov chains using surrogate data tests of the null hypothesis that a symbolic sequence is of nth markov order are described these tests are based on resampling methods similar to the method of surrogate data used to test continuous time series for nonlinear structure the size and power of these symbolic surrogate tests are estimated using binary and heptary test sequences of known order conditioning on a sufficient statistic under the null hypothesis yields an estimated true size which closely approximates the nominal size and increases the power of the test with respect to the unconditional test we apply one method to a symbolic sequence derived from the rossler system and to a binary and heptary representation of an interspike interval sequence of an electroreceptor of the weakly electric eel apteronotus leptorhynchus

markov chain order estimation with conditional mutual information abstract we introduce the conditional mutual information cmi for the estimation of the markov chain order for a markov chain of k symbols we define cmi of order m i c m as the mutual information of two variables in the chain being m time steps apart conditioning on the intermediate variables of the chain we find approximate analytic significance limits based on the estimation bias of cmi and develop a randomization significance test of i c m where the randomized symbol sequences are formed by random permutation of the components of the original symbol sequence the significance test is applied for increasing m and the markov chain order is estimated by the last order for which the null hypothesis is rejected we present the appropriateness of cmi testing on monte carlo simulations and compare it to the akaike and bayesian information criteria the maximal fluctuation method peres shields estimator and a likelihood ratio test for increasing orders using divergence the order criterion of cmi testing turns out to be superior for orders larger than one but its effectiveness for large orders depends on data availability in view of the results from the simulations we interpret the estimated orders by the cmi testing and the other criteria on genes and intergenic regions of dna chains

kernel recursive least squares tracker for time varying regression in this paper we introduce a kernel recursive least squares krls algorithm that is able to track nonlinear time varying relationships in data to this purpose we first derive the standard krls equations from a bayesian perspective including a sensible approach to pruning and then take advantage of this framework to incorporate forgetting in a consistent way thus enabling the algorithm to perform tracking in nonstationary scenarios the resulting method is the first kernel adaptive filtering algorithm that includes a forgetting factor in a principled and numerically stable manner in addition to its tracking ability it has a number of appealing properties it is online requires a fixed amount of memory and computation per time step incorporates regularization in a natural manner and provides confidence intervals along with each prediction we include experimental results that support the theory as well as illustrate the efficiency of the proposed algorithm

nonuniform state space reconstruction and coupling detection we investigate the state space reconstruction from multiple time series derived from continuous and discrete systems and propose a method for building embedding vectors progressively using information measure criteria regarding past current and future states the embedding scheme can be adapted for different purposes such as mixed modeling cross prediction and granger causality in particular we apply this method in order to detect and evaluate information transfer in coupled systems as a practical application we investigate in records of scalp epileptic eeg the information flow across brain areas

sampling methods for the nystrom method the nystrom method is an efficient technique to generate low rank matrix approximations and is used in several large scale learning applications a key aspect of this method is the procedure according to which columns are sampled from the original matrix in this work we explore the efficacy of a variety of fixed and adaptive sampling schemes we also propose a family of ensemble based sampling algorithms for the nystrom method we report results of extensive experiments that provide a detailed comparison of various fixed and adaptive sampling techniques and demonstrate the performance improvement associated with the ensemble nystrom method when used in conjunction with either fixed or adaptive sampling schemes corroborating these empirical findings we present a theoretical analysis of the nystrom method providing novel error bounds guaranteeing a better convergence rate of the ensemble nystrom method in comparison to the standard nystrom method

sketched ridge regression optimization perspective statistical perspective and model averaging we address the statistical and optimization impacts of the classical sketch and hessian sketch used to approximately solve the matrix ridge regression mrr problem prior research has quantified the effects of classical sketch on the strictly simpler least squares regression lsr problem we establish that classical sketch has a similar effect upon the optimization properties of mrr as it does on those of lsr namely it recovers nearly optimal solutions by contrast hessian sketch does not have this guarantee instead the approximation error is governed by a subtle interplay between the mass in the responses and the optimal objective value for both types of approximation the regularization in the sketched mrr problem results in significantly different statistical properties from those of the sketched lsr problem in particular there is a bias variance trade off in sketched mrr that is not present in sketched lsr we provide upper and lower bounds on the bias and variance of sketched mrr these bounds show that classical sketch significantly increases the variance while hessian sketch significantly increases the bias empirically sketched mrr solutions can have risks that are higher by an order of magnitude than those of the optimal mrr solutions we establish theoretically and empirically that model averaging greatly decreases the gap between the risks of the true and sketched solutions to the mrr problem thus in parallel or distributed settings sketching combined with model averaging is a powerful technique that quickly obtains near optimal solutions to the mrr problem while greatly mitigating the increased statistical risk incurred by sketching

sketch kernel ridge regression using circulant matrix algorithm and theory kernel ridge regression krr is a powerful method for nonparametric regression the time and space complexity of computing the krr estimate directly are mathcal o n 3 and mathcal o n 2 respectively which are prohibitive for large scale data sets where n is the number of data in this article we propose a novel random sketch technique based on the circulant matrix that achieves savings in storage space and accelerates the solution of the krr approximation the circulant matrix has the following advantages it can save time complexity by using the fast fourier transform fft to compute the product of matrix and vector its space complexity is linear and the circulant matrix whose entries in the first column are independent of each other and obey the gaussian distribution is almost as effective as the i i d gaussian random matrix for approximating krr combining the characteristics of the circulant matrix and our careful design theoretical analysis and experimental results demonstrate that our proposed sketch method making the estimate kernel methods scalable and practical for large scale data problems outperforms the state of the art krr estimates in time complexity while retaining similar accuracies meanwhile our sketch method provides the theoretical bound that keeps the optimal convergence rate for approximating krr

state space reconstruction parameters in the analysis of chaotic time series the role of the time window length abstract the most common state space reconstruction method in the analysis of chaotic time series is the method of delays mod many techniques have been suggested to estimate the parameters of mod i e the time delay and the embedding dimension m we discuss the applicability of these techniques with a critical view as to their validity and point out the necessity of determining the overall time window length w for successful embedding emphasis is put on the relation between w and the dynamics of the underlying chaotic system and we suggest to set w p the mean orbital period p is approximated from the oscillations of the time series the procedure is assessed using the correlation dimension for both synthetic and real data for clean synthetic data values of w larger than p always give good results given enough data and thus p can be considered as a lower limit w p for noisy synthetic data and real data an upper limit is reached for w which approaches p for increasing noise amplitude

markov regression models for time series a quasi likelihood approach this paper discusses a quasi likelihood ql approach to regression analysis with time series data we consider a class of markov models referred to by cox 1981 scandinavian journal of statistics 8 93 115 as observation driven models in which the conditional means and variances given the past are explicit functions of past outcomes the class includes autoregressive and markov chain models for continuous and categorical observations as well as models for counts e g poisson and continuous outcomes with constant coefficient of variation e g gamma we focus on poisson and gamma data for illustration analogous to ql for independent observations large sample properties of the regression coefficients depend only on correct specification of the first conditional moment

embedding as a modeling problem abstract standard approaches to time delay embedding will often fail to provide an embedding that is useful for many common applications this happens in particular when there are multiple timescales in the dynamics we present a modified procedure non uniform embedding which overcomes such problems in many cases for more complex nonlinear dynamics we introduce variable embedding where in a suitable sense the embedding changes with the state of the system we show how to implement these procedures by combining embedding and modeling into a single procedure with a single optimization goal

estimating mutual information we present two classes of improved estimators for mutual information m x y from samples of random points distributed according to some joint probability density mu x y in contrast to conventional estimators based on binnings they are based on entropy estimates from k nearest neighbor distances this means that they are data efficient with k 1 we resolve structures down to the smallest possible scales adaptive the resolution is higher where data are more numerous and have minimal bias indeed the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale giving typically systematic errors which scale as functions of k n for n points numerically we find that both families become exact for independent distributions i e the estimator m x y vanishes up to statistical fluctuations if mu x y mu x mu y this holds for all tested marginal distributions and for all dimensions of x and y in addition we give estimators for redundancies between more than two random variables we compare our algorithms in detail with existing algorithms finally we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from independent component analysis ica for improving ica and for estimating the reliability of blind source separation

mutual information functions versus correlation functions this paper studies one application of mutual information to symbolic sequences the mutual information functionm d this function is compared with the more frequently used correlation function d an exact relation betweenm d and d is derived for binary sequences for sequences with more than two symbols no such general relation exists in particular d 0 may or may not lead tom d 0 this linear but not general independence between symbols separated by a distance is studied for ternary sequences also included is the estimation of the finite size effect on calculating mutual information finally the concept of symbolic noise is discussed

statistical dynamics of clustering in the genome structure clustering and long range correlations in the nucleotide sequences of different categories of organisms are discussed clustering mostly observed in higher eucaryotes can be found at different length scales in dna and central limit theorems are used as links between these length scales several dynamical statistical mean field models are proposed based on biologically motivated dynamical mechanisms and they successfully reproduce both the short range behavior observed in coding dna and the long range out of equilibrium features of non coding dna such dynamical mechanisms include aggregation of oligonucleotides influx and dna length reduction schemes transpositions and fusions of large dna macromolecules fractality can be inferred from the short and long range correlations observed in the sequence structure of higher eucaryotes where the non coding part is relatively extended in these organisms the dna coding non coding alternation has the characteristics of finite size fractal random sets

multiobjective optimization for model selection in kernel methods in regression regression plays a major role in many scientific and engineering problems the goal of regression is to learn the unknown underlying function from a set of sample vectors with known outcomes in recent years kernel methods in regression have facilitated the estimation of nonlinear functions however two major interconnected problems remain open the first problem is given by the bias versus variance tradeoff if the model used to estimate the underlying function is too flexible i e high model complexity the variance will be very large if the model is fixed i e low complexity the bias will be large the second problem is to define an approach for selecting the appropriate parameters of the kernel function to address these two problems this paper derives a new smoothing kernel criterion which measures the roughness of the estimated function as a measure of model complexity then we use multiobjective optimization to derive a criterion for selecting the parameters of that kernel the goal of this criterion is to find a tradeoff between the bias and the variance of the learned function that is the goal is to increase the model fit while keeping the model complexity in check we provide extensive experimental evaluations using a variety of problems in machine learning pattern recognition and computer vision the results demonstrate that the proposed approach yields smaller estimation errors as compared with methods in the state of the art

sparse model from optimal nonuniform embedding of time series an approach to obtaining a parsimonious polynomial model from time series is proposed an optimal minimal nonuniform time series embedding schema is used to obtain a time delay kernel this scheme recursively optimizes an objective functional that eliminates a maximum number of false nearest neighbors between successive state space reconstruction cycles a polynomial basis is then constructed from this time delay kernel a sparse model from this polynomial basis is obtained by solving a regularized least squares problem the constraint satisfaction problem is made computationally tractable by keeping the ratio between the number of constraints to the number of variables small by using fewer samples spanning all regions of the reconstructed state space this helps the structure selection process from an exponentially large combinatorial search space a forward stagewise algorithm is then used for fast discovery of the optimization path results are presented for the mackey glass system

dynamics from multivariate time series abstract multivariate time series data are common in experimental and industrial systems if the generating system has nonlinear dynamics we may be able to construct a model that reproduces the dynamics and can be used for control and other purposes in principle multivariate time series are not necessary for recovering dynamics according to the embedding theorem only one time series should be needed however for real data there may be large gains in using all of the measurements in this paper we examine the issues of how to use multiple data streams most effectively for modeling and prediction for example perhaps the data are redundant in that only a subset of the data streams is useful and how should we embed the data if indeed embedding is required at all we show how these questions can be answered and describe some numerical experiments which show that using multivariate time series can significantly improve predictability we also demonstrate a somewhat surprising synchronization between different reconstructions

species independence of mutual information in coding and noncoding dna we explore if there exist universal statistical patterns that are different in coding and noncoding dna and can be found in all living organisms regardless of their phylogenetic origin we find that i the mutual information function symbol see text has a significantly different functional form in coding and noncoding dna we further find that ii the probability distributions of the average mutual information symbol see text are significantly different in coding and noncoding dna while iii they are almost the same for organisms of all taxonomic classes surprisingly we find that symbol see text is capable of predicting coding regions as accurately as organism specific coding measures

granger causality in multivariate time series using a time ordered restricted vector autoregressive model granger causality has been used for the investigation of the inter dependence structure of the underlying systems of multivariate time series in particular the direct causal effects are commonly estimated by the conditional granger causality index cgci in the presence of many observed variables and relatively short time series cgci may fail because it is based on vector autoregressive models var involving a large number of coefficients to be estimated in this paper the var is restricted by a scheme that modifies the recently developed method of backward in time selection bts of the lagged variables and the cgci is combined with bts further the proposed approach is compared favorably to other restricted var representations such as the top down strategy the bottom up strategy and the least absolute shrinkage and selection operator lasso in terms of sensitivity and specificity of cgci this is shown by using simulations of linear and nonlinear low and high dimensional systems and different time series lengths for nonlinear systems cgci from the restricted var representations are compared with analogous nonlinear causality indices further cgci in conjunction with bts and other restricted var representations is applied to multichannel scalp electroencephalogram eeg recordings of epileptic patients containing epileptiform discharges cgci on the restricted var and bts in particular could track the changes in brain connectivity before during and after epileptiform discharges which was not possible using the full var representation

determining and interpreting the order of a two state markov chain application to models of daily precipitation we illustrate the result that lumping together some of the states of a many state first order markov chain does not in general give a first order markov chain with a smaller number of states if a series generated in this way is nevertheless assumed to have been produced by a two state markov chain standard statistical procedures using the akaike and bayesian information criteria may indicate that it should be fitted by a higher order than first stochastic models based on a markov chain are often used to model precipitation series it is normal to classify days as dry and wet and fit a two state process in some cases second or higher order chains are preferred by reference to information criteria in the light of the foregoing we suggest that this might be because a many state process possibly of only first order would actually be a better choice than a two state process

dna viewed as an out of equilibrium structure the complexity of the primary structure of human dna is explored using methods from nonequilibrium statistical mechanics dynamical systems theory and information theory a collection of statistical analyses is performed on the dna data and the results are compared with sequences derived from different stochastic processes the use of 2 tests shows that dna can not be described as a low order markov chain of order up to r 6 although detailed balance seems to hold at the level of a binary alphabet it fails when all four base pairs are considered suggesting spatial asymmetry and irreversibility furthermore the block entropy does not increase linearly with the block size reflecting the long range nature of the correlations in the human genomic sequences to probe locally the spatial structure of the chain we study the exit distances from a specific symbol the distribution of recurrence distances and the hurst exponent all of which show power law tails and long range characteristics these results suggest that human dna can be viewed as a nonequilibrium structure maintained in its state through interactions with a constantly changing environment based solely on the exit distance distribution accounting for the nonequilibrium statistics and using the monte carlo rejection sampling method we construct a model dna sequence this method allows us to keep both long and short range statistical characteristics of the native dna data the model sequence presents the same characteristic exponents as the natural dna but fails to capture spatial correlations and point to point details 2014 american physical society

an approach for characterizing coupling in dynamical systems abstract the study of coupling in dynamical systems dates back to christian hyugens who in 1665 discovered that pendulum clocks with the same length pendulum synchronize when they are near to each other in that case the observed synchronous motion was out of phase in this paper we propose a new approach for measuring the degree of coupling and synchronization of a dynamical system consisting of interacting subsystems the measure is based on quantifying the active degrees of freedom e g correlation dimension of the coupled system and the constituent subsystems the time delay embedding scheme is extended to coupled systems and used for attractor reconstruction of the coupled dynamical system we use the coupled lorenz rossler and henon model systems with a coupling strength variable for evaluation of the proposed approach results show that we can measure the active degrees of freedom of the coupled dynamical systems and can quantify and distinguish the degree of synchronization or coupling in each of the dynamical systems studied furthermore using this approach the direction of coupling can be determined

clustered nystrom method for large scale manifold learning and dimension reduction kernel or similarity matrix plays a key role in many machine learning algorithms such as kernel methods manifold learning and dimension reduction however the cost of storing and manipulating the complete kernel matrix makes it infeasible for large problems the nystrom method is a popular sampling based low rank approximation scheme for reducing the computational burdens in handling large kernel matrices in this paper we analyze how the approximating quality of the nystrom method depends on the choice of landmark points and in particular the encoding powers of the landmark points in summarizing the data our non probabilistic error analysis justifies a clustered nystrom method that uses the k means clustering centers as landmark points our algorithm can be applied to scale up a wide variety of algorithms that depend on the eigenvalue decomposition of kernel matrix or its variant such as kernel principal component analysis laplacian eigenmap spectral clustering as well as those involving kernel matrix inverse such as least squares support vector machine and gaussian process regression extensive experiments demonstrate the competitive performance of our algorithm in both accuracy and efficiency

statistical analysis of gene and intergenic dna sequences abstract much of the on going statistical analysis of dna sequences is focused on the estimation of characteristics of coding and non coding regions that would possibly allow discrimination of these regions in the current approach we concentrate specifically on genes and intergenic regions to estimate the level and type of correlation in these regions we apply various statistical methods inspired from nonlinear time series analysis namely the probability distribution of tuplets the mutual information and the identical neighbour fit the methods are suitably modified to work on symbolic sequences and they are first tested for validity on sequences obtained from well known simple deterministic and stochastic models then they are applied to the dna sequence of chromosome 1 of arabidopsis thaliana the results suggest that correlations do exist in the dna sequence but they are weak and that intergenic sequences tend to be more correlated than gene sequences the use of statistical tests with surrogate data establish these findings in a rigorous statistical manner

a model for high order markov chains on introduit un modele pour des chaines de markov du leme ordre qui comporte seulement un parametre additionnel pour chaque retard

a unifying approach to some old and new theorems on distribution and clustering abstract a unifying approach is proposed to studying the distributions of eigenvalues and singular values of toeplitz matrices associated with a fourier series and multilevel toeplitz matrices associated with a multidimensional fourier series obtained are the extensions of the szego and avram parter theorems where the generating function is now required to belong to l 2 and not necessarily to l analogous extensions are given for multilevel toeplitz matrices in particular it is proved that if f x 1 x p l 2 then the p level complex toeplitz matrices allied with f have their singular values distributed as f x 1 x p the distribution results for the cesaro optimal circulants hold even if f l 1 also suggested are new theorems on clustering that have to do with the preconditioning of multilevel toeplitz matrices by multilevel circulants

long range correlation properties of coding and noncoding dna sequences genbank analysis an open question in computational molecular biology is whether long range correlations are present in both coding and noncoding dna or only in the latter to answer this question we consider all 33301 coding and all 29453 noncoding eukaryotic sequences each of length larger than 512 base pairs bp in the present release of the genbank to dtermine whether there is any statistically significant distinction in their long range correlation properties standard fast fourier transform fft analysis indicates that coding sequences have practically no correlations in the range from 10 bp to 100 bp spectral exponent beta 0 00 0 04 where the uncertainty is two standard deviations in contrast for noncoding sequences the average value of the spectral exponent beta is positive 0 16 0 05 which unambiguously shows the presence of long range correlations we also separately analyze the 874 coding and the 1157 noncoding sequences that have more than 4096 bp and find a larger region of power law behavior we calculate the probability that these two data sets coding and noncoding were drawn from the same distribution and we find that it is less than 10 10 we obtain independent confirmation of these findings using the method of detrended fluctuation analysis dfa which is designed to treat sequences with statistical heterogeneity such as dna s known mosaic structure patchiness arising from the nonstationarity of nucleotide concentration the near perfect agreement between the two independent analysis methods fft and dfa increases the confidence in the reliability of our conclusion

effects of coarse graining on the scaling behavior of long range correlated and anti correlated signals abstract we investigate how various coarse graining signal quantization methods affect the scaling properties of long range power law correlated and anti correlated signals quantified by the detrended fluctuation analysis specifically for coarse graining in the magnitude of a signal we consider i the floor ii the symmetry and iii the centro symmetry coarse graining methods we find that for anti correlated signals coarse graining in the magnitude leads to a crossover to random behavior at large scales and that with increasing the width of the coarse graining partition interval this crossover moves to intermediate and small scales in contrast the scaling of positively correlated signals is less affected by the coarse graining with no observable changes when 1 while for 1 a crossover appears at small scales and moves to intermediate and large scales with increasing for very rough coarse graining 3 based on the floor and symmetry methods the position of the crossover stabilizes in contrast to the centro symmetry method where the crossover continuously moves across scales and leads to a random behavior at all scales thus indicating a much stronger effect of the centro symmetry compared to the floor and the symmetry method for coarse graining in time where data points are averaged in non overlapping time windows we find that the scaling for both anti correlated and positively correlated signals is practically preserved the results of our simulations are useful for the correct interpretation of the correlation and scaling properties of symbolic sequences

multivariate phase space reconstruction by nearest neighbor embedding with different time delays a recently proposed nearest neighbor based selection of time delays for phase space reconstruction is extended to multivariate time series with an iterative selection of variables and time delays a case study of numerically generated solutions of the x and z coordinates of the lorenz system and an application to heart rate and respiration data are used for illustration

novel perturbation expansion for the langevin equation we discuss the randomly driven systemdx dt w x f t wheref t is a gaussian random function or stirring force with f t f t 2 t t andw x is of the formgx1 2 the parameter is a measure of the nonlinearity of the equation we show how to obtain the correlation functions x t f t x t n f as a power series in we obtain three terms in the expansion and show how to use pade approximants to analytically continue the answer in the variable by using scaling relations we show how to get a uniform approximation to the equal time correlation functions valid for allg and

smooth s cobordisms of elliptic 3 manifolds the main result of this paper states that a symplectic s cobordism of elliptic 3 manifolds is diffeomorphic to a product assuming a canonical contact structure on the boundary based on this theorem we conjecture that a smooth s cobordism of elliptic 3 manifolds is smoothly a product if its universal cover is smoothly a product we explain how the conjecture fits naturally into the program of taubes of constructing symplectic structures on an oriented smooth 4 manifold with b 1 from generic self dual harmonic forms the paper also contains an auxiliary result of independent interest which generalizes taubes theorem sw gr to the case of symplectic 4 orbifolds

reconstructing embedding spaces of coupled dynamical systems from multivariate data a method for reconstructing dimensions of subspaces for weakly coupled dynamical systems is offered the tool is able to extrapolate the subspace dimensions from the zero coupling limit where the division of dimensions as per the algorithm is exact implementation of the proposed technique to multivariate data demonstrates its effectiveness in disentangling subspace dimensionalities also in the case of emergent synchronized motions for both numerical and experimental systems

the consistency of the bic markov order estimator the bayesian information criterion bic estimates the order of a markov chain with finite alphabet a from observation of a sample path x 1 x 2 x n as that value k k that minimizes the sum of the negative logarithm of the kth order maximum likelihood and the penalty term a k a 1 2 log n we show that k equals the correct order of the chain eventually almost surely as n oo thereby strengthening earlier consistency results that assumed an apriori bound on the order a key tool is a strong ratio typicality result for markov sample paths we also show that the bayesian estimator or minimum description length estimator of which the bic estimator is regarded as an approximation fails to be consistent for the uniformly distributed i i d process

retargeted least squares regression algorithm this brief presents a framework of retargeted least squares regression relsr for multicategory classification the core idea is to directly learn the regression targets from data other than using the traditional zero one matrix as regression targets the learned target matrix can guarantee a large margin constraint for the requirement of correct classification for each data point compared with the traditional least squares regression lsr and a recently proposed discriminative lsr models relsr is much more accurate in measuring the classification error of the regression model furthermore relsr is a single and compact model hence there is no need to train two class binary machines that are independent of each other the convex optimization problem of relsr is solved elegantly and efficiently with an alternating procedure including regression and retargeting as substeps the experimental evaluation over a range of databases identifies the validity of our method

regularized local linear prediction of chaotic time series abstract local linear prediction based on the ordinary least squares ols approach is one of several methods that have been applied to prediction of chaotic time series apart from potential linearization errors a drawback of this approach is the high variance of the predictions under certain conditions here a different set of so called linear regularization techniques originally derived to solve ill posed regression problems are compared to ols for chaotic time series corrupted by additive measurement noise these methods reduce the variance compared to ols but introduce more bias a main tool of analysis is the singular value decomposition svd and a key to successful regularization is to damp the higher order svd components several of the methods achieve improved prediction compared to ols for synthetic noise corrupted data from well known chaotic systems similar results were found for real world data from the r r intervals of ecg signals good results are also obtained for real sunspot data compared to published predictions using nonlinear techniques

local rademacher complexities and oracle inequalities in risk minimization let be a class of measurable functions f s 0 1 defined on a probability space s mathcal a p given a sample x1 xn of i i d random variables taking values in s with common distribution p let pn denote the empirical measure based on x1 xn we study an empirical risk minimization problem pnf min f given a solution fn of this problem the goal is to obtain very general upper bounds on its excess risk mathcal e p hat f n p hat f n inf f in mathcal f pf expressed in terms of relevant geometric parameters of the class using concentration inequalities and other empirical processes tools we obtain both distribution dependent and data dependent upper bounds on the excess risk that are of asymptotically correct order in many examples the bounds involve localized sup norms of empirical and rademacher processes indexed by functions from the class we use these bounds to develop model selection techniques in abstract risk minimization problems that can be applied to more specialized frameworks of regression and classification

approximate confidence intervals for the parameters of a stationary binary markov chain the power divergence family of test statistics introduced by cressie and read 1984 is used to obtain approximate confidence intervals for the parameters of a stationary first order binary markov chain intervals based on inverting the likelihood ratio pearson and freeman tukey statistics are included in this family small sample comparisons are used to select the best intervals for the different parameters the results show that the best power divergence intervals for the probability of success and in particular the likelihood ratio interval have error rates closer to nominal levels than the sample proportion based intervals suggested by crow 1979 the properties of power divergence joint confidence regions for the transition probabilities are also explored

nearest neighbor embedding with different time delays a nearest neighbor based selection of time delays for phase space reconstruction is proposed and compared to the standard use of time delayed mutual information the possibility of using different time delays for consecutive dimensions is considered a case study of numerically generated solutions of the lorenz system is used for illustration the effect of contamination with various levels of additive gaussian white noise is discussed

randomized sketches of convex programs with sharp guarantees random projection rp is a classical technique for reducing storage and computational costs we analyze rp based approximations of convex programs in which the original optimization problem is approximated by solving a lower dimensional problem such dimensionality reduction is essential in computation limited settings since the complexity of general convex programming can be quite high e g cubic for quadratic programs and substantially higher for semidefinite programs in addition to computational savings rp is also useful for reducing memory usage and has useful properties for privacy preserving optimization we prove that the approximation ratio of this procedure can be bounded in terms of the geometry of the constraint set for a broad class of rps including those based on various sub gaussian distributions as well as randomized hadamard and fourier transforms the data matrix defining the cost function can be projected to a dimension proportional to the squared gaussian width of the tangent cone of the constraint set at the original solution this effective dimension of the convex program is often substantially smaller than the original dimension we illustrate consequences of our theory for various cases including unconstrained and ell 1 constrained least squares support vector machines low rank matrix estimation and discuss implications for privacy preserving optimization as well as connections with denoising and compressed sensing

application of statistical information criteria for optimal fuzzy model construction theoretical studies have shown that fuzzy models are capable of approximating any continuous function on a compact domain to any degree of accuracy however constructing a good fuzzy model requires finding a good tradeoff between fitting the training data and keeping the model simple a simpler model is not only easily understood but also less likely to overfit the training data even though heuristic approaches to explore such a tradeoff for fuzzy modeling have been developed few principled approaches exist in the literature due to the lack of a well defined optimality criterion in this paper we propose several information theoretic optimality criteria for fuzzy models construction by extending three statistical information criteria 1 the akaike information criterion aic 1974 2 the bhansali downham information criterion bdic 1977 and 3 the information criterion of schwarz 1978 and rissanen 1978 sric we then describe a principled approach to explore the fitness complexity tradeoff using these optimality criteria together with a fuzzy model reduction technique based on the singular value decomposition svd the role of these optimality criteria in fuzzy modeling is discussed and their practical applicability is illustrated using a nonlinear system modeling example

estimation of the direction of the coupling by conditional probabilities of recurrence we introduce a method to detect and quantify the asymmetry of the coupling between two interacting systems based on their recurrence properties this method can detect the direction of the coupling in weakly as well as strongly coupled systems it even allows detecting the asymmetry of the coupling in the more challenging case of structurally different systems and it is very robust against noise we also address the problem of detecting the asymmetry of the coupling in passive experiments i e when the strength of the coupling cannot be systematically changed which is of great relevance for the analysis of experimental time series

surrogate time series before we apply nonlinear techniques e g those inspired by chaos theory to dynamical phenomena occurring in nature it is necessary to first ask if the use of such advanced techniques is justified by the data while many processes in nature seem very unlikely a priori to be linear the possible nonlinear nature might not be evident in specific aspects of their dynamics the method of surrogate data has become a very popular tool to address such a question however while it was meant to provide a statistically rigorous foolproof framework some limitations and caveats have shown up in its practical use in this paper recent efforts to understand the caveats avoid the pitfalls and to overcome some of the limitations are reviewed and augmented by new material in particular we will discuss specific as well as more general approaches to constrained randomisation providing a full range of examples new algorithms will be introduced for unevenly sampled and multivariate data and for surrogate spike trains the main limitation which lies in the interpretability of the test results will be illustrated through instructive case studies we will also discuss some implementational aspects of the realisation of these methods in the tisean software package

local rademacher complexities we propose new bounds on the error of learning algorithms in terms of a data dependent notion of complexity the estimates we establish give optimal rates and are based on a local and empirical version of rademacher averages in the sense that the rademacher averages are computed from the data on a subset of functions with small empirical error we present some applications to classification and prediction with convex function classes and with kernel classes in particular

spatio temporal dynamics of human eeg electroencephalogram eeg recording of spontaneous brain electrical activity resulting from collective dynamical behaviour of the neural mass was traditionally treated as a random signal and processed by stochastic methods like spectral analysis qualitatively new views were opened by approaches derived from synergetics non linear dynamics and theory of deterministic chaos introduced into eeg research in the last six years see refs 1 2 and references therein in this approach the eeg signal is supposed to be a deterministic signal generated by chaotic dynamics of finite and even low dimensional dynamical system evolving on its strange attractor processing eeg data consists then of the reconstruction of a hypothetical strange attractor from experimental data and computations of relevant dynamical topological invariants fractal dimensions lyapunov exponents kolmogorov entropy the goal of this treatment is using measures of complexity and chaos for characterization of brain processes reflected in the eeg signal i e developing new tools for computerized eeg analysis with applications in psychiatry neurology and pharmacology of psychoactive drugs eeg signals can be simultaneously registered from several locations on the scalp 16 32 64 two ways of reconstructing the brain attractor are presently used one channel reconstruction a one dimensional one channel eeg signal registered from a particular location on the scalp is considered as a smooth projection of n dimensional chaotic dynamics on the strange attractor an n dimensional trajectory is reconstructed using the time delay method based affiliated to 3rd school of medicine charles university prague e mail pcpmp csearn bitnet 0378 4371 92 05 00 1992 elsevier science publishers b v all rights reserved

a new maximum relevance minimum multicollinearity mrmmc method for feature selection and ranking a new maximum relevanceminimum multicollinearity mrmmc method is proposed the proposed mrmrc algorithm was applied to a number of real life datasets experimental results are reported and compared with several state of the art methods numerical analysis results confirmed the promising performance of the proposed method a substantial amount of datasets stored for various applications are often high dimensional with redundant and irrelevant features processing and analysing data under such circumstances is time consuming and makes it difficult to obtain efficient predictive models there is a strong need to carry out analyses for high dimensional data in some lower dimensions and one approach to achieve this is through feature selection this paper presents a new relevancy redundancy approach called the maximum relevanceminimum multicollinearity mrmmc method for feature selection and ranking which can overcome some shortcomings of existing criteria in the proposed method relevant features are measured by correlation characteristics based on conditional variance while redundancy elimination is achieved according to multiple correlation assessment using an orthogonal projection scheme a series of experiments were conducted on eight datasets from the uci machine learning repository and results show that the proposed method performed reasonably well for feature subset selection

methods and criteria for model selection model selection is an important part of any statistical analysis and indeed is central to the pursuit of science in general many authors have examined the question of model selection from both frequentist and bayesian perspectives and many tools for selecting the best model have been suggested in the literature this paper considers the various proposals from a bayesian decision theoretic perspective

revisiting the nystrom method for improved large scale machine learning we reconsider randomized algorithms for the low rank approximation of symmetric positive semi definite spsd matrices such as laplacian and kernel matrices that arise in data analysis and machine learning applications our main results consist of an empirical evaluation of the performance quality and running time of sampling and projection methods on a diverse suite of spsd matrices our results highlight complementary aspects of sampling versus projection methods they characterize the effects of common data preprocessing steps on the performance of these algorithms and they point to important differences between uniform sampling and nonuniform sampling methods based on leverage scores in addition our empirical results illustrate that existing theory is so weak that it does not provide even a qualitative guide to practice thus we complement our empirical results with a suite of worst case theoretical bounds for both random sampling and random projection methods these bounds are qualitatively superior to existing bounds e g improved additive error bounds for spectral and frobenius norm error and relative error bounds for trace norm error and they point to future directions to make these algorithms useful in even larger scale machine learning applications

reconstructing attractors from scalar time series a comparison of singular system and redundancy criteria abstract a delay vector phase space reconstruction in which the delay time satisfiesa minimum redundancy criterion is compared with a reconstruction obtained using a singular system approach minimum redundancy produces the better reconstruction the reconstructions are compared using a distortion functional d which measures how well the location of a point in the original phase space can be determined on the basis of its image under the reconstruction process the superiority of the redundancy analysis over the singular system analysis is found to arise from the former s foundation on the notion of general independence as opposed to the latter s foundation on the notion of linear independence

multikernel least mean square algorithm the multikernel least mean square algorithm is introduced for adaptive estimation of vector valued nonlinear and nonstationary signals this is achieved by mapping the multivariate input data to a hilbert space of time varying vector valued functions whose inner products kernels are combined in an online fashion the proposed algorithm is equipped with novel adaptive sparsification criteria ensuring a finite dictionary and is computationally efficient and suitable for nonstationary environments we also show the ability of the proposed vector valued reproducing kernel hilbert space to serve as a feature space for the class of multikernel least squares algorithms the benefits of adaptive multikernel mk estimation algorithms are illuminated in the nonlinear multivariate adaptive prediction setting simulations on nonlinear inertial body sensor signals and nonstationary real world wind signals of low medium and high dynamic regimes support the approach