# Forecasting Multivariate Time Series
## 3 Experimental Results and Discussion
### Experiment 2: Multivariate Series
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Rank** & **Preprocessing** & **Model** & **Settings** & **MAE** & **RMSE** \\ \hline \multicolumn{6}{l}{_Part 1: Stock Market Multivariate Time Series with No Data Quality Issues_} \\ \hline
1 & NA & GRU & Epoch = 100, units = 50 & 3.92 & 5.13 \\
2 & NA & RNN & Epoch = 100, units = 50 & 3.93 & 5.13 \\
3 & NA & RNN & Epoch = 25, units = 50 & 3.95 & 5.19 \\
4 & NA & MLP & Epoch = 100, units = 20 & 4.04 & 5.19 \\
17 & NA & Random forest & 100 trees, split MSE & 5.47 & 7.08 \\
23 & NA & VARMA & (2, 1, 3) & 407.98 & 657.08 \\
24 & NA & VMA & (0, 1, 1) & 365.41 & 752.85 \\
25 & NA & VAR & Order = 5 & 963.64 & 1678.42 \\ \hline \multicolumn{6}{l}{_Part 2: Individual Household Electric Power Consumption Multivariate Time Series with Missing Data_} \\ \hline
1 & Exponential & GRU & Epoch = 100, units = 20 & 0.12 & 0.64 \\
2 & CMA & GRU & Epoch = 100, units = 20 & 0.13 & 0.64 \\
3 & Bayesian Ridge & MLP & Epoch = 100, units = 50 & 0.13 & 0.65 \\
4 & Random forest & Stacked LSTM & Epoch = 100, units = 50 & 0.13 & 0.65 \\
330 & KNNImputer 5 & VMA & (0,1,1) & 0.43 & 1.88 \\
331 & KNNImputer 5 & VARMA & (2,1,3) & 0.43 & 1.88 \\
449 & polynomial\_3 & RNN & Epoch = 25, units = 20 & 9.35 & 29.31 \\
450 & polynomial\_3 & DT regressor & Split MSE & 14.53 & 58.72 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Model performance summary of experiment based on multivariate series. Figure 10: (**a**) GRU performance comparison for multivariate time series data during testing. (**b**) VAR model performance comparison for multivariate time series data during testing. The comprehensive report generated by Preptimize began by profiling the data, analyzing each attribute individually. A correlation heatmap, depicted previously in Figure 8b, illustrates the pairwise relationships between columns. Notably, global intensity and global active power exhibited a strong positive correlation, while sub-metering 3, associated with climate control systems, correlated strongly with global active power and global intensity. In contrast, global reactive power showed no correlation with other attributes, particularly voltage and sub-metering 3. Furthermore, the report assessed the missing data percentage, approximately 37%, identifying columns such as sub-metering 1 and sub-metering 2 as non-normally distributed with high kurtosis values. Preptimize classified all four columns with missing data as MNAR type. Attempts to normalize sub-metering 1 and sub-metering 2 distributions were hindered by the possibility of zero or negative values, precluding Box-Cox transformation. After trend and season decomposition, various imputation techniques, including iterative methods, were applied. Each imputed dataset underwent evaluation using 18 different forecasting models, resulting in a comprehensive performance assessment. Preptimize generated an evaluation sheet encompassing 450 models, detailing the preprocessing techniques employed during imputation. As shown previously, Table 3 presents the performance metrics of select models, highlighting the best-performing model as GRU imputed with an exponential moving average, yielding an RMSE of 0.6393 and MAE of 0.1225. Conversely, the decision tree regressor, imputed with polynomial interpolation of degree 3, exhibited the poorest performance, with an RMSE of 14.5308 and MAE of 58.7152. Remarkably, statistical models demonstrated consistent performance when data were imputed using KNNImputer with two neighbors, with VMA, VARMA, and VAR ranking closely. However, some machine learning models fared worse. Performance graphs for GRU and decision tree regressor models are provided in Figure 11a,b, respectively. Figure 11: (**a**) GRU imputed with exponential moving average: performance comparison for power consumption series data during testing. (**b**) Decision tree imputed with polynomial interpolation of degree 3: performance comparison for power consumption series during testing.