# Statistical Complexity of Low- and High-Dimensional Systems

Authors: Vladimir Ryabov; Dmitry Nerukh
Journal: Journal of Atomic, Molecular, and Optical Physics
 (2012)

---

## Abstract

We suggest a new method for the analysis of experimental time series that can distinguish high-dimensional dynamics from stochastic motion. It is based on the idea of statistical complexity, that is, the Shannon entropy of the so-calledϵ-machine (a Markov-type model of the observed time series). This approach has been recently demonstrated to be efficient for making a distinction between a molecular trajectory in water and noise. In this paper, we analyse the difference between chaos and noise using the Chirikov-Taylor standard map as an example in order to elucidate the basic mechanism that makes the value of complexity in deterministic systems high. In particular, we show that the value of statistical complexity is high for the case of chaos and attains zero value for the case of stochastic noise. We further study the Markov property of the data generated by the standard map to clarify the role of long-time memory in differentiating the cases of deterministic systems and stochastic motion.


---

## Body

## 1. Introduction

Statistical complexity is a measure that had been introduced by Crutchfield and Young in 1989 [1]. It has been proven useful for describing various complex systems, including those with hundreds of degrees of freedom [2]. According to our earlier paper [3], the statistical complexity of high-dimensional trajectories generated by the dynamics of an ensemble of water molecules grows up to the time scale of 1 microsecond, that is, an extremely long-time interval for a typical molecular dynamics simulation. Moreover, this property is much less pronounced for so-called surrogate time series that have exactly the same power spectrum and, hence, autocorrelation function as the original time series.For example, in Figure1 we plot the dependence of statistical complexity on the length of the time series for the symbolic data obtained from a Poincaré the section of 3D velocities describing the motion of a hydrogen atom in an ensemble of 392 water molecules [3]. The details of computing the atomic trajectories as well as the method used for partitioning the phase space and obtaining a symbolic string from the initially floating point data can be found in [4]. In the same figure, we draw the curves calculated for so-called phase-shuffled surrogate time series [5], the data having identical autocorrelation function, and hence power spectrum as the original velocity trajectories. One can notice significant differences between the statistical complexity of the physical and the artificially generated data.Figure 1
Statistical complexity versus the (log of) length of the analysis interval for the hydrogen velocity time series (top curve) and four surrogate time series (bottom curves): three independent realisations of the phase-shuffling algorithm (red, green, and blue), and single time series of a white noise passed through a low-pass linear filter (black) (from [3]). Note that the value of statistical complexity for the data obtained from simple random number generator is close to zero and does not depend on the length of time series for large enough value of the latter (not shown).We then put forward a hypothesis that this property, that is a high value of statistical complexity, can be used for distinguishing between deterministic and stochastic systems (see also [6]). The phenomenon of the complexity growth with the length of time series that ensures the difference between the cases of deterministic and stochastic behaviour remains still unexplored. In order to elucidate the mechanism that makes the value of complexity high, we performed numerical experiments with the standard map (known also as the Chirikov-Taylor map) [7], one of the most studied paradigmatic models in nonlinear dynamics. We observed that statistical complexity was high indeed in the case of the standard map, and it had much lower value for the surrogate time series, being close to zero for the case of noncorrelated noise from a random number generator.For the purpose of estimating statistical complexity for a symbolic time series, we utilize the CSSR algorithm [8] that had been reported as an efficient, reliable, and easy to use software. The algorithm constructs an ϵ-machine, a Markov-chain with l-step memory, which constitutes a probabilistic model for the analysed data series. Statistical complexity measures an information content of the ϵ-machine via its Shannon entropy. Our analysis shows, however, that although the CSSR algorithm always converges well and produces a finite value of complexity, in some cases the approximation of data with a Markov-chain-type model is inadequate, making the complexity value dependent on the length of the analysed data. Finally, we came to the conclusion that, at least in the case of standard map, the main reason for the growth of complexity is the property of stickiness of periodic islands in the chaotic sea, a generic phenomenon in Hamiltonian systems [9]. It has been noted in [10] that due to the sticking property of the regular component in a subcritical domain (K<0.9716), the dynamics of the standard map is subdiffusive that can be well approximated with a continuous time random walk model. Anomalous properties of the temporal behaviour of nonextensive entropy, a generalization of the usual Boltzmann-Gibbs entropy, have been also analysed in [11].In the present work, we mainly study the domain ofK≫1 where the area occupied by periodic islands is small, and the chaotic motion can be expected to be strongly mixing and ergodic. Nevertheless, as our results show, the presence of stickiness is still an important factor defining the long-term statistical measures. In terms of the CSSR algorithm, the property of stickiness breaks the independence of the data points separated by a history long-time interval, thus making the Markov-chain approximation invalid.Finally, we discuss a conjecture that the property of the non-Markovianity of theϵ-machine and growth of statistical complexity can be used in a constructive way for distinguishing deterministic and stochastic behaviours. The problem of detecting determinism in a noise looking chaotic time series is a long standing one. An extensive review of the issues related to the difference between chaos and noise and to inherent difficulties encountered in the high-dimensional cases can be found in [12]. We suggest that there is a significant difference between the statistical complexities of Hamiltonian chaos and coloured noise with identical power spectrum, the main reason for which consists in the presence of the long-time memory in time series obtained from Hamiltonian systems. This property originates from the stickiness of periodic islands that are abundant in the chaotic sea due to multiple resonances that occur in the phase space.We would also like to note that since the phase space of Hamiltonian systems has a complicated structure of chaotic areas intermingled with periodic islands, this leads in some cases to the necessity of distinguishing between chaos and complex quasiperiodic motion. A measure calledorbital complexity had been introduced for this purpose in the context of analysing the orbital motions of planets [13–15]. This measure, although being based on the calculation of the Shannon entropy (but in the spectral domain), has quite different meaning, purpose, and scope of applicability compared to statistical complexity.


## 2. Systems and Method of Analysis

The standard map is defined as(1)pn+1=pn+Ksin⁡θnmod⁡2π,θn+1=θn+pn+1mod⁡2π,
where K is a single parameter defining the dynamics of this system. In all the calculations below, the value of the parameter K has been chosen at K=6.908745.First, at the step called “symbolization,” the original real-valued time series is transformed to a symbolic sequence by introducing a suitable partitioning of the phase space (Figure2(a)).The standard map. Symbolization with a three-symbol alphabet (a). Two periodic islands are embedded into the chaotic sea. One of them zoomed is shown in (b).
(a)(b)At the next stage, the sequence of symbols is transformed to the sequence of histories, thel-symbol strings representing a refinement of the partitioning in the phase space [3]. ϵ-machine reconstruction requires a grouping of histories to “causal states,” based on the analysis of the predictive properties of each history by one step forward in time. Finally, the statistical complexity is calculated from the ϵ-machine as Shannon entropy of the probability distribution of the causal states:(2)Cμ=-∑i=1Ncpilog⁡pi,
here pi are probabilities of the causal states in the ϵ-machine, and Nc is the total number of the causal states.


## 3. Numerical Experiments

We have calculated the statistical complexity using the algorithm called CSSR [8] for the standard map and plotted the graphs of complexity versus the amount of data (the length of symbolic sequence). We have also studied how the calculated values depend on the method of partitioning the map, initial conditions, and the parameter K of the system.A typical plot of statistical complexityCμ and the number of causal states for the history length l=2⋯9 are shown in Figure 3.The values ofCμ and the number of causal states for various history lengths (from bottom to top l=2⋯9) for the standard map trajectory as a function of the data length.
(a)(b)The results for the surrogate data generated using the same trajectory of the standard map are shown in Figure4. Changing the initial conditions, the type of partitioning the phase space at the stage of symbolization, and/or value of the parameter K brings qualitatively the same results, that is, there is a significant difference between the complexity values calculated for the data obtained from the map and those for the surrogate time series. It should be noted though that the complexity value moderately increases with parameter K as shown in Figure 5. This behaviour is similar to that of other characteristics used in nonlinear dynamics, like Lyapunov exponents or measure of the chaotic area reported in [16].Same as in Figure3 but for the random surrogate.
(a)(b)Figure 5
Dependence of statistical complexity on the parameterK (the number of data points is 108).


## 4. A Hypothesis on Markov Property

In this section, we would like to demonstrate that the large complexity values observed in the case of the standard map are caused by the presence of certain segments in the chaotic trajectory (which become histories after symbolization) that do not possess a property necessary for building a Markov-chain from the data. Consider the stage when the symbolic string has been converted to the sequence of histories, that is, symbolic words of lengthl. The Markov-chain (ϵ-machine) can be built from such a sequence, if the conditional probability distribution of the next symbol in the symbolic sequence depends only on the l-symbol string preceding the symbol, and it is independent on the previous symbol, that is, the symbol that occurred l+1 time steps before. In other words, if we consider the conditional probability distribution for a given history, it should not change (in statistical sense), if we increase the length of a history by one symbol to the past.In Figure6, we present a scatter diagram that demonstrates the distribution of the conditional probabilities for each history at the history length l=8. Every point in the diagram corresponds to a single history. The large spread around the point with coordinates (1/3,1/3) evidences significant difference compared to the case of uniform distribution. The change from l=8 to l=9 does not change the overall pattern of point distribution shown in Figure 6. However, the analysis of the conditional probabilities for individual histories reveals huge changes in the position of points depending on the extra symbol added at the beginning of the history. In Figure 7, we depict the conditional distribution for the next symbol for two histories: one that shows strong deviation from Markov property and a “normal” one, that is, a randomly chosen history. The large deviation in the distribution of conditional probabilities can be concluded from a big distance between the vertexes of the upper triangle (distribution of the conditional probabilities at l=9) and the cross corresponding to the conditional probability at l=8. The probabilities at l=9 are computed by adding one of the three symbols (012) at the beginning of the history of l=8.Figure 6
Conditional distributions of the next symbol for all histories at history lengthl=8. Every point represents a history. The total number of points (histories) is 38.Figure 7
Conditional probability distribution for the history with the largest deviation from the Markov property (black triangle). Adding a symbol to the history changes drastically the position of the point in the diagram. Three circles correspond to adding “0,” “1,” or “2” to the historyl=8 (crosses). The red triangle represents a randomly chosen history with clear Markov property, that is, the distribution of probabilities does not depend on the added symbol.Finally, we would like to show that the segments of chaotic trajectories that correspond to the history with large deviation from the Markov property are located in the areas of the phase space close to periodic islands. For this purpose, we plotted in Figure8 only the points that correspond to the history with large deviation from the Markov-chain property. A comparison to Figure 2 suggests that the history with large deviation in the distributions is located close to the periodic islands. Therefore, we suppose that the breaking of Markovianity can be interpreted as a manifestation of the well-known phenomenon of “stickiness” [9] of trajectories in the areas close to periodic islands. Prolonged wandering of a trajectory around the island is equivalent to existing of long-time memory in the corresponding segments of the chaotic time series. Figure 8 should be also compared to Figure 9, which presents the segments of the chaotic trajectory corresponding to a history possessing the Markov property. Apparently, it has no relation to periodic islands. Such histories represent a vast majority in the ensemble of 3l histories, the non-Markovian histories constituting only a fraction of percent.Figure 8
Parts of chaotic trajectory corresponding to the history with large deviation from the Markov-chain property. Only points corresponding to the central three symbols in the history are shown. Apparently, the history includes one of the two periodic islands shown in Figure2.Figure 9
Parts of chaotic trajectory corresponding to the history with no deviation from the Markov-chain property. Only points corresponding to the central three symbols in the history are shown. Apparently, the history does not include any of the two periodic islands shown in Figure2.


## 5. Discussion

It has been demonstrated in this paper that statistical complexity appears to be a useful measure for distinguishing Hamiltonian chaos in low- and high-dimensional systems from correlated noise with identical autocorrelation function. Its value for the symbolic time series calculated from the dynamics of Hamiltonian systems is substantially larger than that for a white noise time series or the time series obtained from the phase shuffling surrogate algorithm. Our explanation of the origin of this phenomenon in terms of Markov-chain theory consists in breaking down the Markov property by the symbolic sequences obtained from Hamiltonian systems.We believe that the large value of complexity observed in our numerical experiments is defined by the presence of periodic islands with sticky borders in the phase space of Hamiltonian systems. The stickiness of certain areas in the phase space leads to long-time memory effects that are responsible for breaking down the statistical independence of the future states from the past ones. This, in turn, makes the procedure of grouping the histories into causal states constituting the core of CSSR algorithm unstable. As a result, the algorithm finds more and more causal states necessary for building theϵ-machine as a Markov chain, and the value of complexity grows with the number of causal states.



---

*Source: 589651-2012-06-20.xml*