# Learning Optimal Time Series Combination and Pre-Processing by Smart Joins
## 3 Proposed Solution: Smart Join
### Description of The Methodology  
The general concept of the methodology of the smart join is explained next:  
1. First, the joining model is fitted using training data; in other words, the optimal joining solution of the process is obtained. This needs to be done for each feature separately. 2. Then, resampled data is predicted by applying the selected join method to the test data. 3. Finally, the model is validated using resampling error. Suppose we have a time series slice \(y\) of the selected feature that needs to be resampled to be joined with a desired time index. First of all, the fit method is used in order to obtain the \"optimal\" join method. The inputs needed for the join are the original time series slice (\(y\) with the original time index) and the desired time index. Other optional parameter can be a fill NA function as it can affect selecting the \"optimal\" method. Then, another slice of the same feature (\(z\)) is used for the testing by the use of the method score. Finally, the optimal joined method is used for resampling other time slices of the features with the predict method. The structure of the different methods can be depicted as in Figure 1.  
The fitting process to find an optimal joining model could be mathematically represented as follows:  
Figure 1: Smart join methodology implementation structure. Firstly fit method is used for the selection of the optimal join method and then, predict and score methods are used to resample other slices of the time series and in order to control the error produced by the join. Suppose we have the time series \(y(t_{O})\) where \(t_{O}=[t_{O1},t_{O2},\ldots,t_{Om}]\) is the initial temporal reference system. Let \(j\) be a join method from the available methods set \(J\) (_left_, _backward_, _forward_, _nearest_). We need to obtain a new time series \(\hat{y}(t_{D};j,y)\) with the desired temporal reference system \(t_{D}=[t_{D1},t_{D2},\ldots,t_{Dn}]\). The smart join algorithm aims to find the optimal join method \(j\in J=\{\textit{left},\textit{backward},\textit{forward},\textit{nearest}\}\) that minimizes an error function \(E(y,\hat{y})\). The parameters for applying the smart join method are the function meant to fill unavailable measurement values \(f\in F=\{\textit{None},\textit{bfill},\textit{bfill},\textit{nearest}\}\). In case of not being specified, default values will be used (in which case \(f=\textit{None}\)). The possible values of the imputation function \(f\) are _None_ (not filling NA values), _bfill_ (using subsequent value that is nearest) and _fill_ (using prior value that is nearest). The optimization problem is defined as:  
\[\operatorname*{arg\,min}_{j\in J}E(y,\hat{y}) \tag{1}\]  
With respect to the second contribution put forward by the present paper, the error function \(E(y,\hat{y})\) proposed is defined by Equation (2). \[ \begin{split} E(y,\hat{y})=& w_{1}\cdot\textit{ NaEl}(\hat{y})+w_{2}\cdot\textit{MissEl}(y,\hat{y})+w_{3}\cdot\textit{DelEl}(y, \hat{y})\\ &+w_{4}\cdot\textit{DelT}(y,\hat{y})+w_{5}\cdot\textit{AntEl}(y, \hat{y})+w_{6}\cdot\textit{AntT}(y,\hat{y})\\ &+w_{7}\cdot\textit{Diff}(y,\hat{y}),\end{split}  \tag{2}\]  
where \(w_{i}>0\) with \(i\in\{1,2,\ldots,7\}\) and \(\sum_{i=1}^{7}w_{i}=1\) are the weights for the total error calculation and, in case of not being specified, their default value is \(w_{i}=1/7\ \forall i\). In the following paragraphs, each function that takes part in the error \(E(y,\hat{y})\) is presented. Suppose \(k\in\{1,2,\ldots,n\}\) and \(l\in\{1,2,\ldots,m\}\) indicate the index of elements in \(\hat{y}\) and \(y\) respectively. \(\textit{NaEl}(\hat{y})\) represents the percentage of NA elements of \(\hat{y}\) after the application of \(f\). NA values can be problematic in machine learning applications implying for example the need to remove data points with NA value on the model training process or the impossibility to predict an output value using the trained model. \[\textit{NaEl}(\hat{y})=\frac{\sum_{k=1}^{n}s_{k}}{n}\text{, where }s_{k}=