# Parallelism Strategies for Big Data Delayed Transfer Entropy Evaluation
## 1 Introduction  
Recently, the amount of data being generated and collected have been rising with the popularization of technologies such as Internet of Things, social media, and smartphone [1]. The increasing amount of data led the creation of the term big data, with one definition given by [PERSON] et al. [2], as a set of technologies and techniques to discover hidden information from diverse, complex and massive scale datasets. One class of hidden information is causality, which [PERSON] et al. [3] discuss and propose a framework to deal with commonly found big data biases such as confounding and sampling selection. The interaction among random variables and subsystems in complex dynamical multivariate models as seen in big data are a developing research area that had been widely applied in a variety of fields, such as climatic processes [4; 5], brain analysis [6; 7], pathology propagation [8; 9], circadian networks [10; 11], among others. The knowledge about causality in such complex systems can be useful for the evaluation of their dynamics and also to their modelling since it can lead to topological simplifications. In many cases, the assessment of interaction or coupling is subject to bias [12]. Among the tools to infer causal relationships, there is Mutual Information used by [PERSON] et al. [13] to infer neuron connectivity by [14] to optimization of meteorological network design. Additionally, Transfer Entropy (TE) exists, which allows identification of a cause-effect relationship by not accounting for straightforward and uniquely shared information [15]. TE has been applied to many complex problems from diverse research fields e.g., oscillation analysis [16], finance [17; 18], sensors [19; 20; 21; 22], biosignals [23; 24], thermonuclear fusion [25], complex networks [26], geophysical phenomena [27; 28], industrial energy consumption network [29] and algorithmic theory of information [30]. In addition, TE has been implemented in non-Gaussian distributions, such as: multivariate exponential, logistic, Pareto (type I-IV) and Burr distributions [31]. A derivation of TE metric called Delayed TE (DTE) is explored in [PERSON] et al. [32] to extract topology in complex networks, where they mentioned an ad hoc complex network example. [PERSON] et al. [33]make inferences in nonlinear systems with this tool; in addition, [PERSON] et al. [34] use DTE to estimate externally applied forces to a robot using low-cost sensors. However, DTE calculation requires a high demanding processing power [35; 36], which is aggravated with large datasets as those found in big data. Many approaches were proposed to overcome such performance issues, e.g., an implementation using a GPU made by [PERSON] et al. [37] and an implementation using an FPGA made by [PERSON] et al. [36]. Parallel programs should be optimised to extract maximum performance from hardware on architecture case by case [38], which is far from trivial according to [PERSON] et al. [39]. There exist different and combined manners to explore parallelism such as Data Parallelism and Task Parallelism [40]. [PERSON] et al. [41] stated that choosing the configuration of parallel programs is a \"mysterious art\" in a study in which they created a model aiming at maximum speedup by balancing different parallelism strategies for both cluster and cloud computing environments. Moreover, [PERSON] and [PERSON] [42] discuss the importance of parallel computation using big data in industrial sectors. In this paper, we address DTE performance issue by using a previously not described approach to decrease DTE execution time using a Beowulf cluster; then, we present and execute two parallelism strategies on big data time series and compare run time difference. An additional importance support to this theme is given by [PERSON] et al. [35], where they concluded that TE performance is about 3000 times slower than Granger Causality, making TE unsuited for many Big Data analysis.