# Learning Optimal Time Series Combination and Pre-Processing by Smart Joins
## 3 Proposed Solution: Smart Join
### Description of The Methodology
\begin{cases}1& \text{if }\hat{y}_{k}\text{ is NA}\\ 0&\text{otherwise}\end{cases}  \tag{3}\]  
\(\textit{MissEl}(y,\hat{y})\) is the percentage of elements from \(y\) that are not used in \(\hat{y}\). This value is related to the lost of information from the original time series due to the resampling needed. \[\textit{MissEl}(y,\hat{y})=\frac{\sum_{l=1}^{m}x_{l}}{m}\text{, where }x_{l}= \begin{cases}1&\text{if }y_{l}\
otin\hat{y}\\ 0&\text{otherwise}\end{cases}  \tag{4}\]  
\(\textit{DelEl}(y,\hat{y})\) indicates the percentage of delayed elements. If most of the data points from \(y\) are delayed, the reality for the machine learning model is displaced. Depending on the application environment, taking decisions supported by the machine learning system that could not adequately represent the current situation can be problematic. \[\textit{DelEl}(y,\hat{y})=\frac{\sum_{k=1}^{n}d_{k}}{n}\text{, where }d_{k}= \begin{cases}1&\text{if }(\hat{y}_{k}=y_{l})\text{ and }(t_{Dk}>t_{Ol})\ \forall l\\ 0&\text{otherwise}\end{cases}  \tag{5}\]  
\(\textit{DelT}(y,\hat{y})\) is the maximum difference in time between a delayed element used in \(\hat{y}\) and its original time position normalized by the time window of \(y\). Whereas the previous case considers the frequency of delayed elements, \(\textit{DelT}(y,\hat{y})\) takes into account the magnitude of the displacement. \[\textit{DelT}(y,\hat{y})=\frac{\max(e_{k})}{t_{Om}-t_{O1}}\text{,where }e_{k}= \begin{cases}t_{Dk}-t_{Ol}&\text{if }(\hat{y}_{k}=y_{l})\text{ and }(t_{Dk}>t_{Ol})\ \forall l\\ 0&\text{otherwise}\end{cases}  \tag{6}\]\(AntEl(y,\hat{y})\) and \(AntT(y,\hat{y})\) are equivalent functions but in this case for anticipated elements. \[AntEl(y,\hat{y})=\frac{\sum_{k=1}^{n}a_{k}}{n},\,\text{where}\,\,a_{k}= \begin{cases} 1&\text{if}\,\,(\hat{y}_{k}=y_{l})\,\,\text{and}\,\,(t_{Dk}<t_{OI})\,\,\forall l \\ 0&\text{otherwise}\end{cases}  \tag{7}\]  
and  
\[AntT(y,\hat{y})=\frac{\max(b_{k})}{t_{Om}-t_{OI}},\,\text{where}\,\,b_{k}= \begin{cases}t_{OI}-t_{Dk}&\text{if}\,\,(\hat{y}_{k}=y_{l})\,\,\text{and}\,\,(t_{Dk }<t_{OI})\,\,\forall l\\ 0&\text{otherwise}\end{cases}  \tag{8}\]  
On the one side, the use of anticipated data is equivalent to the use of future information for prediction and results can be misleading and the used approximation should be sound enough to deal with value forecasting. On the other side, using future data could imply a need to wait for the arrival of a new observation to be able to make a prediction, or a correction would be needed once the predicted value and the real one are compared. Finally \(Diff(y,\hat{y})\) calculates the difference between the two time series (original and resampled). This value could represent the magnitude of the distortion committed due to the need of a joined data with synchronized temporal reference system. \[Diff(y,\hat{y})=\frac{\text{mean}(\text{abs}(y_{inter}-\hat{y}_{inter}))}{\max( y)-\min(y)}, \tag{9}\]  
where \(y_{inter}\) and \(\hat{y}_{inter}\) are obtained by means of linear interpolation of time series \(y\) and \(\hat{y}\) respectively for time values in \(t_{O}\bigcup t_{D}\). Each part of the sum of the error calculation Equations (3)-(9) is normalized to guarantee that the result is in range \([0,1]\) so different errors are comparable between them. The fitting method can be seen graphically in Figure 2.  
Validating the joined method in different time slices of the time series is crucial. If the slice of data used to train the joining model is adequately selected, the errors should be similar in different time  
Figure 2: Fitting method diagram. First, \(\hat{y}_{j}\) resampled time series are generated from each joining method (\(j\in J\)). Using the generated resampled time series, error is calculated in each case and the optimal solution is selected \(j_{opt}\). windows. Depending on the stability of the feature, retraining may be required as the optimal join method could not be the most adequate during all time period.