# Causality Analysis with Different Probabilistic Distributions Using Transfer Entropy
## 2 Mathematical Modeling
### Transfer Entropy  
Transfer Entropy provides an information-theoretic interpretation of [PERSON]'s definition of causality. Essentially, it quantifies the amount of information transferred from one variable to another by calculating the reduction in uncertainty when predictability is assumed [27]. It can be expressed by Equation (1):  
\[T_{x\to y}=\sum_{y_{i+h}Y_{i},X_{j}}p(y_{i+h},Y_{i},X_{j})\cdot\frac{p(y_{i+h} |Y_{i},X_{j})}{p(y_{i+h}|Y_{i})}, \tag{1}\]  
where the variable \(Y_{i}=[y_{i},y_{i-\tau},\ldots,y_{i-(l-1)\tau}]\), the variable \(X_{j}=[x_{j},x_{j-\tau},\ldots,x_{j-(k-1)\tau}]\), \(l\) and \(k\) are dimensional delay embedding vectors, \(p\) is defined as a complete or conditional Probability Density Function, \(\tau\) represents sampling, and \(h\) denotes the prediction horizon. Transfer Entropy (TE) provides valuable information about causality without requiring delayed information. It measures the difference between information obtained from the simultaneous observation of past values of both \(x\) and \(y\), and information obtained from the past values of \(x\) alone regarding a future observation of \(x\)[28].  
To implement the Transfer Entropy approach between two variables, it is necessary to simplify the formula in Equation (1) to the form presented in Equation (2). This simplified form is used in practical applications.  
\[T_{x\to y}=\sum_{y_{i},y_{i-t},x_{i-\tau}}p(y_{i},y_{i-t},x_{i-\tau})\cdot \frac{p(y_{i},y_{i-t},x_{i-\tau})p(y_{i-t})}{p(y_{i-t},x_{i-\tau})p(y_{i},y_{i- t})}. \tag{2}\]  
The formula for Transfer Entropy involves a conditional Probability Density Function (PDF), denoted by \(p\). The time lags in \(x\) and \(y\) are represented by \(\tau\) and \(t\), respectively. For this type of analysis, the probability density is typically estimated using a Gaussian density estimation method. If the time series is short, \(t\) is set to 1 based on the assumption that the datapoint immediately preceding the target value in \(y\) provides the maximum auto-transference of information.  
Transfer Entropy is inherently asymmetric and is based on transition probabilities, making it a natural method for incorporating directional and/or dynamic information [29]. One of the main benefits of using an information-theoretic approach such as Transfer Entropy to detect causality is that it does not assume any particular model for the interaction between variables. This makes it particularly useful for exploratory analyses, as its sensitivity to all order correlations is advantageous compared to model-based approaches such as cross-correlation, Granger Causality, and Partial Directed Coherence [30]. This is especially relevant when trying to detect unknown nonlinear interactions, as Transfer Entropy tends to produce more accurate results.  
On the other side, TE is restricted by a stationary statement, in that the dynamic properties of the analyzed process do not change over the set of data used [18]. It is common for the assumption of stationarity to be invalid in time series analysis, and the level of noise present (which may be nonstationary) is often higher than anticipated. In addition, calculations may involve parameters such as the prediction horizon \(h\) or embedding dimensions. As the TE approach is based on the Probability Distribution Function, it is highly dependent on accurate estimation of the PDF. The PDF can have any non-Gaussian form. In this paper, the calculation results are based on the selection of the best-fitted distributions from among the Cauchy, \(\alpha\)-stable, Laplace, Huber, and t-location scale distributions.