measuring the strangeness of strange attractors we study the correlation exponent v introduced recently as a characteristic measure of strange attractors which allows one to distinguish between deterministic chaos and random noise the exponent v is closely related to the fractal dimension and the information dimension but its computation is considerably easier its usefulness in characterizing experimental data which stem from very high dimensional systems is stressed algorithms for extracting v from the time series of a single variable are proposed the relations between the various measures of strange attractors and between them and the lyapunov exponents are discussed it is shown that the conjecture of kaplan and yorke for the dimension gives an upper bound for v various examples of finite and infinite dimensional systems are treated both numerically and analytically

partially ordered permutation complexity of coupled time series abstract a new information theoretic complexity measure of coupling among multiple time series is proposed based on ordinal patterns we regard ordinal patterns extracted from time series as total orders and exploit their algebraic properties to introduce a multiplication among them the result of the multiplication is in general a partial order a kullback leibler divergence for the partial orders gives rise to a complexity measure of coupling in the sense that it vanishes in two extreme cases when time series are uncoupled and when they are identical the performance of the proposed complexity measure is tested on two model multivariate dynamical systems and its advantage over the existing complexity measures is discussed

detection of couplings in ensembles of stochastic oscillators the problem of detection and quantitative characterization of directional couplings in an ensemble of noisy oscillators from a time series is addressed we suggest estimators for the strengths of couplings which are based on modeling the observed oscillations with a set of stochastic phase oscillators and easily interpreted from a physical viewpoint moreover we present an analytic formula for a statistical significance level allowing to reveal an architecture of couplings reliably from a relatively short time series the technique applies to weakly coupled nonsynchronized oscillators it is introduced for oscillators with close basic frequencies but can be readily generalized to the case of arbitrary frequencies efficiency of the technique is demonstrated in numerical experiments

dimensional reduction of conditional algebraic multi information via transcripts abstract symbolic representation is a standard and powerful technique in time series analysis in an ordinal symbolic representation the symbols are the so called ordinal patterns which can be identified with permutations transcripts exploit the fact that permutations build a group the transcript of a pair of permutations being the product of the second permutation times the inverse of the first one this particular setting can be easily generalized to any representation which elements belong to an algebraic group the dimensional reduction of conditional multi information via transcripts proved in this paper perfectly shows the potential of such algebraic symbolic representations specifically given n m group valued random variables the multi information of n variables conditioned on the other m variables can also be calculated as a multi information of n transcripts conditioned on m 1 transcripts under some restrictions such a dimensional reduction can be crucial when estimating a conditional multi information from short time series applications include two popular ordinal indicators of the information flow in coupled time series namely symbolic transfer entropy and momentary sorting information transfer as a by product of the above results two new information directionality indicators based on ordinal transcripts are proposed the simplest one being an unconditioned mutual information

momentary information transfer as a coupling measure of time series we propose a method to analyze couplings between two simultaneously measured time series our approach is based on conditional mutual sorting information it is related to other concepts for detecting coupling directions the old idea of marko for directed information and the more recent concept of schreiber s transfer entropy by setting suitable conditions we first of all consider momentary information in both time series this enables the detection not only of coupling directions but also delays sorting information refers to ordinal properties of time series which makes the analysis robust with respect to strictly monotonous distortions and thus very useful in the analysis of proxy data in climatology fortunately ordinal analysis is easy and fast to compute we consider also the problem of reliable estimation from finite time series

additive conjoint isotonic probabilistic models adisop the isop model or model of twodimensional or bi isotonicity scheiblechner 1995 postulates that the probabilities of ordered response categories increase isotonically in the order of subject ability and item easiness adding a conventional cancellation axiom for the factors of subjects and items gives the adisop model where the c d f s of response categories are functions of an additive item and subject parameter and an ordinal category parameter extending cancellation to the interactions of subjects and categories as well as of items and categories independence axiom of the category factor from the subject and item factor gives the cadisop model completely additive model in which the parallel c d f s are functions of the sum of subject item and category parameters the cadisop model is very close to the unidimensional version of the polytomous rasch model with the logistic item category characteristic s replaced by nonparametric axioms and statistics the axioms representation theorems and algorithms for model fitting of the additive models are presented

detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble a method to characterize dynamical interdependence among nonlinear systems is derived based on mutual nonlinear prediction systems with nonlinear correlation will show mutual nonlinear prediction when standard analysis with linear cross correlation might fail mutual nonlinear prediction also provides information on the directionality of the coupling between systems furthermore the existence of bidirectional mutual nonlinear prediction in unidirectionally coupled systems implies generalized synchrony numerical examples studied include three classes of unidirectionally coupled systems systems with identical parameters nonidentical parameters and stochastic driving of a nonlinear system this technique is then applied to the activity of motoneurons within a spinal cord motoneuron pool the interrelationships examined include single neuron unit firing the total number of neurons discharging at one time as measured by the integrated monosynaptic reflex and intracellular measurements of integrated excitatory postsynaptic potentials epsp s dynamical interdependence perhaps generalized synchrony was identified in this neuronal network between simultaneous single unit firings between units and the population and between units and intracellular epsp s

non linear vibrations of a beam with cantilever hertzian contact boundary conditions abstract the non linear vibrations of a linear beam with cantilever hertizian contact boundary conditions are investigated the method of multiple scales is used to analyze this problem in which it is assumed that the beam remains in contact with the moving surface at all times one primary result from this analysis is the amplitude frequency relation for the various flexural modes the amplitude frequency curves exhibit softening behavior as expected the amount of softening is shown to depend on the linear contact stiffness as well as the specific mode in addition the associated non linear normal modes of this system are derived the modes include a non linear modification to the linear harmonic component as well as a static offset term and second and third harmonic components

estimation of the information by an adaptive partitioning of the observation space we demonstrate that it is possible to approximate the mutual information arbitrarily closely in probability by calculating the relative frequencies on appropriate partitions and achieving conditional independence on the rectangles of which the partitions are made empirical results including a comparison with maximum likelihood estimators are presented

hopf and two multiple semi stable limit cycle bifurcations of a restrained plate subjected to subsonic flow abstract this paper is mainly about the stabilities and bifurcations of a cantilevered plate with nonlinear motion constraints in an axial subsonic flow the galerkin method is employed to discretize the governing partial differential equation the fixed points and their stabilities are presented in a parameter space based on qualitative analysis and numerical studies the system loses stability by flutter and undergoes limit cycle oscillations after instability due to the nonlinearity the stabilities of the limit cycle oscillations are addressed on the basis of the equivalent linearized method the type of hopf bifurcation subcritical or supercritical is dependent on the location of the nonlinear motion constraints interestingly for some certain states the hopf bifurcations are both subcritical and supercritical the two multiple semi stable limit cycle bifurcation which is due to the extreme point of the flutter curve is also determined results of the numerical integration method sufficiently support the analytical method presented in this paper

direction of coupling from phases of interacting oscillators an information theoretic approach a directionality index based on conditional mutual information is proposed for application to the instantaneous phases of weakly coupled oscillators its abilities to distinguish unidirectional from bidirectional coupling as well as to reveal and quantify asymmetry in bidirectional coupling are demonstrated using numerical examples of quasiperiodic chaotic and noisy oscillators as well as real human cardiorespiratory data

robustly estimating the flow direction of information in complex physical systems we propose a new measure phase slope index to estimate the direction of information flux in multivariate time series this measure a is insensitive to mixtures of independent sources b gives meaningful results even if the phase spectrum is not linear and c properly weights contributions from different frequencies these properties are shown in extended simulations and contrasted to granger causality which yields highly significant false detections for mixtures of independent sources an application to electroencephalography data eyes closed condition reveals a clear front to back information flow

normal modes of vibration for non linear continuous systems a definition and a constructive methodology for normal nodes of motion are developed for a class of vibratory systems the dynamics of which are governed by non linear partial differential equations the definition for normal modes is given in terms of the dynamics on two dimensional invariant manifolds in the system phase space these manifolds are a continuation of the planes which represent the well known normal modes of the linearized system a local asymptotic approximation of the geometric structure of these manifolds can be obtained by using an approach which follows that used for generating center manifolds the procedure also provides the non linear ordinary differential equations which govern individual modal dynamics and a physical description of the system configuration when it is undergoing a modal motion in this paper the general theory is described for the application of vibrations of continuous media but it can easily be extended to other situations in order to demonstrate the power of the approach and its unique procedural aspects three examples involving beam vibrations are worked out in detail the examples are conservative simply supported beams the first demonstrates the methodology for a linear beam model the second is a beam on a non linear elastic foundation and the third example is a beam with non linear torsional springs attached at each end in these examples the simplicity of the mode shapes of the linearized model yields relatively simple calculations that do not obscure the important features of the procedure

time series and dependent variables we present a new method for analyzing time series which is designed to extract inherent deterministic dependencies in the series the method is particularly suited to series with broad band spectra such as chaotic series with or without noise we derive quantities j e based on conditional probabilities whose magnitude roughly speaking is an indicator of the extent to which the kth element in the series is a deterministic function of the k j th element to within a measurement uncertainty e we apply our method to a number of deterministic time series generated by chaotic processes such as the tent logistic and h non maps as well as to sequences of quasi random numbers in all cases the 6j correctly indicate the expected dependencies we also show that the j are robust to the addition of substantial noise in a deterministic process in addition we derive a predictability index which is a measure of the extent to which a time series is predictable given some tolerance e finally we discuss the behavior of the 6i as e approaches zero

directionality of coupling from bivariate time series how to avoid false causalities and missed connections we discuss some problems encountered in inference of directionality of coupling or in the case of two interacting systems in inference of causality from bivariate time series we identify factors and influences that can lead to either decreased test sensitivity or false detections and propose ways to cope with them in order to perform tests with high sensitivity and a low rate of false positive results

estimating mutual information we present two classes of improved estimators for mutual information m x y from samples of random points distributed according to some joint probability density mu x y in contrast to conventional estimators based on binnings they are based on entropy estimates from k nearest neighbor distances this means that they are data efficient with k 1 we resolve structures down to the smallest possible scales adaptive the resolution is higher where data are more numerous and have minimal bias indeed the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale giving typically systematic errors which scale as functions of k n for n points numerically we find that both families become exact for independent distributions i e the estimator m x y vanishes up to statistical fluctuations if mu x y mu x mu y this holds for all tested marginal distributions and for all dimensions of x and y in addition we give estimators for redundancies between more than two random variables we compare our algorithms in detail with existing algorithms finally we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from independent component analysis ica for improving ica and for estimating the reliability of blind source separation

non linear forced oscillation of a beam coupled to an actuator via hertzian contact abstract forced harmonic oscillation of a system involving a piezoelectric actuator coupled via hertzian contact to an elastic beam is modeled hertzian non linearities are replaced by an asymptotic series and the resulting system of equations are solved by using complex fourier series and harmonic balance techniques the complex coefficients of the complex fourier series are evaluated recursively and the resulting complex solution is used to obtain approximate real solutions results show that natural frequencies vary monotonically with the mean level of hertzian normal approach also resonance of harmonically components may occur at forcing frequencies equal to the natural frequency of the system divided by an integer this harmonic resonance can result in severe waveform distortion

bootstrap rank ordered conditional mutual information brocmi a nonlinear input variable selection method for water resources modeling the input variable selection problem has recently garnered much interest in the time series modeling community especially within water resources applications demonstrating that information theoretic nonlinear based input variable selection algorithms such as partial mutual information pmi selection pmis provide an improved representation of the modeled process when compared to linear alternatives such as partial correlation input selection pcis pmis is a popular algorithm for water resources modeling problems considering nonlinear input variable selection however this method requires the specification of two nonlinear regression models each with parametric settings that greatly influence the selected input variables other attempts to develop input variable selection methods using conditional mutual information cmi an analog to pmi have been formulated under different parametric pretenses such as k nearest neighbor knn statistics or kernel density estimates kde in this paper we introduce a new input variable selection method based on cmi that uses a nonparametric multivariate continuous probability estimator based on edgeworth approximations ea we improve the ea method by considering the uncertainty in the input variable selection procedure by introducing a bootstrap resampling procedure that uses rank statistics to order the selected input sets we name our proposed method bootstrap rank ordered cmi brocmi we demonstrate the superior performance of brocmi when compared to cmi based alternatives ea kde and knn pmis and pcis input variable selection algorithms on a set of seven synthetic test problems and a real world urban water demand uwd forecasting experiment in ottawa canada

generalized redundancies for time series analysis abstract extensions to various information theoretic quantities such as entropy redundancy and mutual information are discussed in the context of their role in nonlinear time series analysis we also discuss linearized versions of these quantities and their use as benchmarks in tests for nonlinearity many of these quantities can be expressed in terms of the generalized correlation integral and this expression permits us to more clearly exhibit the relationships of these quantities to each other and to other commonly used nonlinear statistics such as the bds and green savit statistics further numerical estimation of these quantities is found to be more accurate and more efficient when the the correlation integral is employed in the computation finally we consider several local versions of these quantities including a local kolmogorov sinai entropy which gives an estimate of variability of the short term predictability

inferring the directionality of coupling with conditional mutual information uncovering the directionality of coupling is a significant step in understanding drive response relationships in complex systems in this paper we discuss a nonparametric method for detecting the directionality of coupling based on the estimation of information theoretic functionals we consider several different methods for estimating conditional mutual information the behavior of each estimator with respect to its free parameter is shown using a linear model where an analytical estimate of conditional mutual information is available numerical experiments in detecting coupling directionality are performed using chaotic oscillators where the influence of the phase extraction method and relative frequency ratio is investigated

mean phase coherence as a measure for phase synchronization and its application to the eeg of epilepsy patients abstract we apply the concept of phase synchronization of chaotic and or noisy systems and the statistical distribution of the relative instantaneous phases to electroencephalograms eegs recorded from patients with temporal lobe epilepsy using the mean phase coherence as a statistical measure for phase synchronization we observe characteristic spatial and temporal shifts in synchronization that appear to be strongly related to pathological activity in particular we observe distinct differences in the degree of synchronization between recordings from seizure free intervals and those before an impending seizure indicating an altered state of brain dynamics prior to seizure activity

isotonic ordinal probabilistic models isop the concept of an ordinal instrumental probabilistic comparison is introduced it relies on an ordinal scale given a priori and on the concept of stochastic dominance it is used to define a weakly independently ordered system or isotonic ordinal probabilistic isop model which allows the construction of separate sample free ordinal scales on a set of subjects and a set of items the isop model is a common nonparametric theoretical structure for unidimensional models for quantitative ordinal and dichotomous variables

from granger causality to long term causality application to climatic data quantitative characterization of interaction between processes from time series is often required in different fields of natural science including geophysics and biophysics typically one estimates short term influences e g the widely used granger causality is defined via one step ahead predictions such an approach does not reveal how strongly the long term behavior of one process under study is affected by the others to overcome this problem we introduce the concept of long term causality which extends the concept of granger causality the long term causality is estimated from data via empirical modeling and analysis of model dynamics under different conditions apart from mathematical examples we apply both approaches to find out how strongly the global surface temperature gst is affected by variations in carbon dioxide atmospheric content solar activity and volcanic activity during the last 150 years influences of all the three factors on gst are detected with the granger causality however the long term causality shows that the rise in gst during the last decades can be explained only if the anthropogenic factor co 2 is taken into account in a model

coarse grained entropy rates for characterization of complex time series abstract a method for classification of complex time series using coarse grained entropy rates cer s is presented the cer s which are computed from information theoretic funcionals redundancies are relative measures of regularity and predictability and for data generated by dynamical systems they are related to kolmogorov sinai entropy a deterministic dynamical origin of the data under study however is not a necessary condition for the use of the cer s since the entropy rates can be defined for stochastic processes as well sensitivity of the cer s to changes in data dynamics and their robustness with respect to noise are tested by using numerically generated time series resulted from both deterministic chaotic and stochastic processes potential application of the cer s in analysis of physiological signals or other complex time series is demonstrated by using examples from pharmaco eeg and tremor classification

characterizing synchronization in time series using information measures extracted from symbolic representations we present a methodology to characterize synchronization in time series based on symbolic representations each time series is mapped onto a sequence of p dimensional delay vectors that are subsequently transformed into symbols by means of a rank ordering of their values based on these representations we propose a transcription scheme between symbols of the respective time series to study synchronization properties group theoretical considerations and the use of information measures allow us to classify regimes of synchronization and to assess its strength we apply our method to a prototype nonlinear system which reveals a rich variety of coupled dynamics we investigate in detail the robustness of the derived synchronization measure against noise and compare its value with that of the established measures

detecting couplings between interacting oscillators with time varying basic frequencies instantaneous wavelet bispectrum and information theoretic approach in the natural world the properties of interacting oscillatory systems are not constant but evolve or fluctuating continuously in time thus the basic frequencies of the interacting oscillators are time varying which makes the system analysis complex for studying their interactions we propose a complementary approach combining wavelet bispectral analysis and information theory we show how these methods uncover the interacting properties and reveal the nature strength and direction of coupling wavelet bispectral analysis is generalized as a technique for detecting instantaneous phase time dependence for the case of two or more coupled nonlinear oscillators whereas the information theory approach can uncover the directionality of coupling and extract driver response relationships in complex systems we generate bivariate time series numerically to mimic typical situations that occur in real measured data apply both methods to the same time series and discuss the results the approach is applicable quite generally to any system of coupled nonlinear oscillators

causation entropy identifies indirect influences dominance of neighbors and anticipatory couplings abstract inference of causality is central in nonlinear time series analysis and science in general a popular approach to infer causality between two processes is to measure the information flow between them in terms of transfer entropy using dynamics of coupled oscillator networks we show that although transfer entropy can successfully detect information flow in two processes it often results in erroneous identification of network connections under the presence of indirect interactions dominance of neighbors or anticipatory couplings such effects are found to be profound for time dependent networks to overcome these limitations we develop a measure called causation entropy and show that its application can lead to reliable identification of true couplings

statistical validation of mutual information calculations comparison of alternative numerical algorithms given two time series x and y their mutual information i x y i y x is the average number of bits of x that can be predicted by measuring y and vice versa in the analysis of observational data calculation of mutual information occurs in three contexts identification of nonlinear correlation determination of an optimal sampling interval particularly when embedding data and in the investigation of causal relationships with directed mutual information in this contribution a minimum description length argument is used to determine the optimal number of elements to use when characterizing the distributions of x and y however even when using partitions of the x and y axis indicated by minimum description length mutual information calculations performed with a uniform partition of the xy plane can give misleading results this motivated the construction of an algorithm for calculating mutual information that uses an adaptive partition this algorithm also incorporates an explicit test of the statistical independence of x and y in a calculation that returns an assessment of the corresponding null hypothesis the previously published fraser swinney algorithm for calculating mutual information includes a sophisticated procedure for local adaptive control of the partitioning process when the fraser and swinney algorithm and the algorithm constructed here are compared they give very similar numerical results less than 4 difference in a typical application detailed comparisons are possible when x and y are correlated jointly gaussian distributed because an analytic expression for i x y can be derived for that case based on these tests three conclusions can be drawn first the algorithm constructed here has an advantage over the fraser swinney algorithm in providing an explicit calculation of the probability of the null hypothesis that x and y are independent second the fraser swinney algorithm is marginally the more accurate of the two algorithms when large data sets are used with smaller data sets however the fraser swinney algorithm reports structures that disappear when more data are available third the algorithm constructed here requires about 0 5 of the computation time required by the fraser swinney algorithm

reducing the bias of causality measures measures of the direction and strength of the interdependence between two time series are evaluated and modified to reduce the bias in the estimation of the measures so that they give zero values when there is no causal effect for this point shuffling is employed as used in the frame of surrogate data this correction is not specific to a particular measure and it is implemented here on measures based on state space reconstruction and information measures the performance of the causality measures and their modifications is evaluated on simulated uncoupled and coupled dynamical systems and for different settings of embedding dimension time series length and noise level the corrected measures and particularly the suggested corrected transfer entropy turn out to stabilize at the zero level in the absence of a causal effect and detect correctly the direction of information flow when it is present the measures are also evaluated on electroencephalograms eeg for the detection of the information flow in the brain of an epileptic patient the performance of the measures on eeg is interpreted in view of the results from the simulation study

information transfer in continuous processes we discuss a recently proposed quantity called transfer entropy which uses time series data to measure the amount of information transferred from one process to another in order to understand its foundation merits and limitations we review some aspects of information theoretic functionals while for symbol sequences these measures have an intuitive interpretation their application to continuous state processes and in particular their estimation from finite data sets is problematic for mutual information finite length scale estimates converge from below and can thus be used to reject the assumption that the observed processes are independent however mutual information does not provide any directional information conversely transfer entropy does resolve the directionality of information exchange but no similar monotonic convergence seems to hold thus only in the case of zero transfer entropy in one direction we can reliably infer an asymmetry of the information exchange

testing for nonlinearity in time series the method of surrogate data we describe a statistical approach for identifying nonlinearity in time series the method first specifies some linear process as a null hypothesis then generates surrogate data sets which are consistent with this null hypothesis and finally computes a discriminating statistic for the original and for each of the surrogate data sets if the value computed for the original data is significantly different than the ensemble of values computed for the surrogate data then the null hypothesis is rejected and nonlinearity is detected we discuss various null hypotheses and discriminating statistics the method is demonstrated for numerical data generated by known chaotic systems and applied to a number of experimental time series which arise in the measurement of superfluids brain waves and sunspots we evaluate the statistical significance of the evidence for nonlinear structure in each case and illustrate aspects of the data which this approach identifies

improved surrogate data for nonlinearity tests current tests for nonlinearity compare a time series to the null hypothesis of a gaussian linear stochastic process for this restricted null assumption random surrogates can be constructed which are constrained by the linear properties of the data we propose a more general null hypothesis allowing for nonlinear rescalings of a gaussian linear process we show that such rescalings cannot be accounted for by a simple amplitude adjustment of the surrogates which leads to spurious detection of nonlinearity an iterative algorithm is proposed to make appropriate surrogates which have the same autocorrelations as the data and the same probability distribution

information directionality in coupled time series using transcripts in ordinal symbolic dynamics transcripts describe the algebraic relationship between ordinal patterns using the concept of transcript we exploit the mathematical structure of the group of permutations to derive properties and relations among information measures of the symbolic representations of time series these theoretical results are then applied for the assessment of coupling directionality in dynamical systems where suitable coupling directionality measures are introduced depending only on transcripts these measures improve the reliability of the information flow estimates and reduce to well established coupling directionality quantifiers when some general conditions are satisfied furthermore by generalizing the definition of transcript to ordinal patterns of different lengths several of the commonly used information directionality measures can be encompassed within the same framework

some remarks on scheiblechner s treatment of isop models scheiblechner 1995 proposes a probabilistic axiomatization of measurement called isop isotonic ordinal probabilistic models that replaces rasch s 1980 specific objectivity assumptions with two interesting ordinal assumptions special cases of scheiblechner s model include standard unidimensional factor analysis models in which the loadings are held constant and the rasch model for binary item responses closely related are the doubly monotone item response models of mokken 1971 see also mokken lewis 1982 sijtsma 1988 molenaar 1991 sijtsma junker 1996 and sijtsma hemker in press more generally strictly unidimensional latent variable models have been considered in some detail by holland and rosenbaum 1986 ellis and van den wollenberg 1993 and junker 1990 1993 the purpose of this note is to provide connections with current research in foundations and nonparametric latent variable and item response modeling that are missing from scheiblechner s 1995 paper and to point out important related work by hemker sijtsma molenaar junker 1996 1997 ellis and junker in press and junker and ellis 1997 we also discuss counterexamples to three major theorems in the paper by carrying out these three tasks we hope to provide researchers interested in the foundations of measurement and item response modeling the opportunity to give the isop approach the careful attention it deserves