# Parallelism Strategies for Big Data Delayed Transfer Entropy Evaluation
## 3 Material and Methods
### Transfer Entropy  
Transfer Entropy (TE) measurement, shown in Equation (2), was introduced by [PERSON] [15] and is used to measure information transfer between two time series. TE has an asymmetric nature, it being possible to determine information direction [81], is  
\[TE_{X\to Y}=\sum_{y_{n+1},y_{n}x_{n}}p(y_{n+1},y_{n}^{(k)},x_{n}^{(l)})\log_{2 }\frac{p(y_{n+1}|y_{n}^{(k)},x_{n}^{(l)})}{p(y_{n+1}|y_{n}^{(k)})}, \tag{2}\]  
where \(y_{n}\) and \(x_{n}\) denote values of \(X\) and \(Y\) at time \(n\); \(y_{n+1}\) the value of \(Y\) at time \(n+1\); \(p\) is the probability of parenthesis content; \(l\) and \(k\) are the number of time slices used to calculate probability density function (PDF) using past values of \(X\) and \(Y\), respectively; chosen \(log_{2}\) means that TE results are given in bits. Figure 1: Data analysis and significance levels. From input data, a series of surrogate signals \(X_{n}^{S}(t)\) is estimated, and for each of the surrogate signals, time delayed transfer entropy is determined. The \(\mathbf{n}\) repetitions indicate a significance level from Equation (1) against random probabilities with the same power spectra and amplitude distribution as real data. Assuming the number of surrogate \(\mathbf{n}\) equals three as shown in this figure, DTE Threshold line displayed on the biggest ensemble plot has a significance level of 75%. Assuming \(k=1\) and \(l=1\) to simplify analysis (also called as D1 TE by [PERSON] et al. [82]), the TE algorithm is demanding regarding computational power [36], with its computational complexity being \(O(B^{3})\)[36], where \(B\) is the chosen number of bins in PDF. An extension to D1 TE proposed by [PERSON] et al. [82] is delayed transfer entropy (DTE--Equation (3), which is a D1 TE with variable causal delay range. This way, a parameter \(d\) represents a variable delay between \(y\) and \(x\). DTE is a useful metric to determine where, within \(d\) range, the biggest transfer of information from \(X\) to \(Y\) occurs. DTE is defined as  
\[DTE_{X\to Y}(d)=\sum_{y_{n+1},y_{n},x_{n+1-d}}p(y_{n+1},y_{n},x_{n+1-d})\log_ {2}\frac{p(y_{n+1}|y_{n},x_{n+1-d})}{p(y_{n+1}|y_{n})}, \tag{3}\]  
where \(y_{n}\) denotes value of \(Y\) at time \(n\); \(y_{n+1}\) the value of \(Y\) at time \(n+1\); \(x_{n+1-d}\) denotes value of \(X\) at time \(n+1\) delayed by parameter \(d\); \(p\) is the probability of parenthesis content; chosen \(log_{2}\) means that DTE results are given in bits. The fundamental reason for which DTE was used instead of Mutual Information (MI) is that, for the biosignals data and the power analysis mentioned above, it was important to know the direction of the information flow, and this property is achieved by TE (and DTE) but not by MI, since \(MI(X;Y)=MI(I;X)\)[83]. Studies with DTE usage were negatively affected by high processing power demands [36], therefore often limiting data size scope or even number of surrogate datasets for DTE analysis. With an objective to clarify program flow, the serial version of DTE analysis is shown in Algorithm 2. The program calculates embedding parameters for each channel and calculates DTE for each two channel permutation. Finally, surrogate signals representing each permutation are generated, and DTEs are calculated. The more surrogates, the better, since it contributes to increasing causality statistical evidence.