# Forecasting Multivariate Time Series
## 3 Experimental Results and Discussion
### Experiment 1: Univariate Series
The results indicated that the Vanilla LSTM deep learning model performed best on the dataset, achieving an RMSE of 0.0294 with 50 epochs and 40 units. The predicted values generated by this model were compared with the expected values on the test set, as shown in Figure 4a. Among the statistical techniques, exponential moving average (EMA) performed the best, with an MAE of 2.0925 and an RMSE of 2.5814. EMA was ranked 127 th out of 139 models. Conversely, the cumulative moving average (CMA)-based model exhibited the worst performance, with an MAE of 27.2085 and an RMSE of 35.1176. Figure 4b depicts a comparison between the actual values and the forecasted values produced by the CMA model. Figure 3: (a) The graph shows normal data distribution of MSFT closing prices for 2 years. (b) Decomposition graph of closing prices to split the trend and seasonality component. The graph represents a clear upward trend and no seasonality. ARIMA stands out as one of the most effective and efficient techniques for time series forecasting. In Pretimize, ARIMA was implemented with an optimized order of (2, 1, 2), validated through the Ljung-Box test. However, despite these optimizations, the resulting RMSE of 3.6001 was not particularly impressive. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Rank** & **Preprocessing** & **Model** & **Settings** & **MAE** & **RMSE** \\ \hline _Part 1: Stock Market Time Series with No Data Quality Issues_ & & & & \\
1 & NA & Vanilla LSTM & Epoch = 50, units = 40 & 0.02 & 0.03 \\
2 & NA & GRU & Epoch = 75, units = 50 & 0.02 & 0.03 \\
3 & NA & Vanilla LSTM & Epoch = 25, units = 50 & 0.02 & 0.03 \\
4 & NA & GRU & Epoch = 50, units = 40 & 0.03 & 0.04 \\
127 & NA & Exponential & - & 2.09 & 2.58 \\
135 & NA & Decision tree regressor & Split = MSE & 7.57 & 8.03 \\
136 & NA & Random & 100 trees, split = MSE & 7.69 & 8.15 \\
138 & NA & AR & - & 21.11 & 26.88 \\
139 & NA & CMA & - & 27.21 & 35.12 \\ \hline \hline \end{tabular}
\begin{tabular}{c c c c c} \hline \hline
1 & CMA & ARIMA & (7, 1, 2) & 0.57 & 0.61 \\
2 & Spline 3 & ARIMA & (7, 1, 7) & 0.65 & 0.84 \\
6 & Exponential & Exponential & - & 1.92 & 2.43 \\
59 & Time & SVR & Kernel = poly & 3.64 & 4.73 \\
64 & Derivatives & SVR & Kernel = RBF & 3.68 & 4.75 \\
69 & Piecewise polynomial & Stacked LSTM & Epoch = 100, units = 50 & 3.66 & 4.76 \\
70 & Derivatives & RNN & Epoch = 25, units = 50 & 3.67 & 4.76 \\
1269 & Polynomial 7 & CMA & - & 27.82 & 35.76 \\
1270 & Polynomial 9 & CMA & - & 28.46 & 36.52 \\ \hline \hline \end{tabular} _Part 3: Cryptocurrency Time Series with missing data_  
\begin{tabular}{c c c c c} \hline \hline
1 & GAIN & Bidirectional LSTM & Epoch = 100, units = 50 & 9.09 & 10.03 \\
2 & Index & Stacked LSTM & Epoch = 100, units = 50 & 8.86 & 10.64 \\
3 & pchip & Vanilla LSTM & Epoch = 100, units = 50 & 10.82 & 12.11 \\
10 & Exponential & GRU-bidirectional & Epoch = 25, units = 50 & 12.95 & 14.67 \\
693 & Cubic & AR & - & 243.97 & 291.00 \\
703 & GAIN & DT regressor & Split = MSE & 260.87 & 324.39 \\
729 & pchip & Random forest & 100 trees, split = MSE & 292.03 & 362.15 \\
824 & CMA & SVR & Kernel = polynomial & 2575.77 & 2576.76 \\
892 & Spline 5 & CMA & - & 7431.54 & 9344.14 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Model performance summary of experiment based on univariate time series. Figure 4: **(a)** Comparison of actual and forecasted data generated by Vanilla LSTM using 50 epochs and 40 units. **(b)** Comparison of actual and forecasted data generated by cumulative moving average (CMA). The stock market dataset was used in the second part of the experiment with missing data problems. As mentioned previously, the data were intentionally removed to test the framework performance by comparing them with the original data. Preptimize was applied to this dataset containing the missing tuples; the framework generated a report of more than 3500 pages with a detailed description of the steps that were applied to the given dataset, including preprocessing and final analysis.