# Complex Systems, Emergence, and Multiscale Analysis: A Tutorial and Brief Survey
## 2 Basics of Complex Systems and Emergence
### Estimation of fractal dimension and Kolmogorov entropy
The KS entropy is then  
\[K=-\lim_{\tau\to 0}\ \lim_{\epsilon\to 0}\ \lim_{d\to\infty}\ \frac{1}{d\tau}\ \sum_{i_{1},\cdots,i_{d}}\ p(i_{1},\cdots,i_{d})\ln p(i_{1},\cdots,i_{d}). \tag{50}\]  
where \(K\) characterizes the rate of creation of entropy. To see this, we can start from the block entropy:  
\[H_{d}(\epsilon,\tau)=-\sum_{i_{1},\cdots,i_{d}}p(i_{1},\cdots,i_{d})\ln p(i_{ 1},\cdots,i_{d}). \tag{51}\]  
It is on the order of \(d\tau K\). The difference between \(H_{d+1}(\epsilon,\tau)\) and \(H_{d}(\epsilon,\tau)\) gives the rate:  
\[h_{d}(\epsilon,\tau)=\frac{1}{\tau}[H_{d+1}(\epsilon,\tau)-H_{d}(\epsilon, \tau)]. \tag{52}\]  
Let  
\[h(\epsilon,\tau)=\lim_{d\to\infty}h_{d}(\epsilon,\tau). \tag{53}\]  
Taking proper limits in Equation (53), we obtain the KS entropy:  
\[K=\lim_{\tau\to 0}\ \lim_{\epsilon\to 0}\ h(\epsilon,\tau)=\lim_{\tau\to 0}\ \lim_{\epsilon\to 0}\ \lim_{d\to\infty}\ \frac{1}{\tau}[H_{d+1}(\epsilon,\tau)-H_{d}(\epsilon,\tau)]. \tag{54}\]  
The KS entropy can be generalized to the order-\(q\) Renyi entropies:  
\[D_{I}=\lim_{\epsilon\to 0}\frac{\sum_{i=1}^{n}p_{i}\log p_{i}}{\log \epsilon}. \tag{46}\]0  
When \(q\to 1\), \(K_{q}\to K\). Like the correlation dimension, the correlation entropy \(K_{2}\) can be computed by the Grassberger-Procaccia algorithm by the following equation:  
\[D_{I}=\lim_{\epsilon\to 0}\frac{\sum_{i=1}^{n}p_{i}\log p_{i}}{\log \epsilon}. \tag{46}\]1  
where \(\tau=L\delta t\) is the actual delay time. The above equation can also be expressed as  
\[D_{I}=\lim_{\epsilon\to 0}\frac{\sum_{i=1}^{n}p_{i}\log p_{i}}{\log \epsilon}. \tag{46}\]2  
Although the above equations involve taking limits, in practice, data are of finite length, and one really looks for power-law scaling behaviors between \(C_{m}(\epsilon)\) and \(\epsilon\) when \(m\) is changed. When power law relations hold, in log-log scale, one should observe a series of curves, which are straight over a significant range of \(\epsilon\), and the curves for smaller embedding dimension \(m\) lie above those for larger \(m\). In certain applications, one may just fix \(\epsilon\) to some small value \(\epsilon^{*}\), say 10% or 15% of the standard deviation of the original time series, then compute \(K_{2}(\epsilon^{*})\). This \(K_{2}(\epsilon^{*})\) is called sample entropy, which has been widely used in various kinds of physiological data analyses. Sample entropy can also be computed for filtered data. When the filter is simply the moving average, which is the simplest ever known, the resulting series of entropies corresponding to different parameters for the moving average is called multiscale entropy. For more details, we refer to [86]. Before ending this subsection, we note a simple but very interesting and useful technique for testing nonlinearity. It is called the surrogate data approach [87; 88]. The basic idea is to examine whether the original time series is distinctly different from a random time series sharing some basic properties of the original time series, such as the distribution or the power-spectral density. In the former case, the random time series can be readily obtained by simply shuffling the original time series. In the latter case, one can randomize the phase of the Fourier transform of the original time series and take the inverse transform.