# An Efficient Multivariate Autoscaling Framework Using Bi-LSTM for Cloud Computing
## 4 Experiment and Evaluation
### Experiment Results
#### 4.4.2 Prediction Performance Results  
1. GWA-T-12 dataset: The MAE, MSE, and RMSE metric values for the multivariate Bi-LSTM model were lower than those for the univariate Bi-LSTM model, as shown in Table 5. The proposed multivariate Bi-LSTM had an RMSE prediction error 1.84-times smaller than that of the univariate one. These results indicate that Bi-LSTM had a better prediction accuracy. Figure 4 shows the absolute error values for the first 100 observations of the univariate and multivariate Bi-LSTM models. We can see in the figure that the multivariate Bi-LSTM had smaller absolute error values in every observation than those of the univariate Bi-LSTM.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Pair of** & **Pearson Correlation** \\
**Metrics** & **Value** \\ \hline corr(network, CPU) & 0.388 \\ corr(network, memory) & **0.422** \\ corr(network, disk read) & \(-\)0.019 \\ corr(network, disk write) & 0.393 \\ \hline \hline \end{tabular} The bold value indicates the best value.  
\end{table}
Table 4: Feature selection of the GWA-T-13 dataset.  
Figure 3: Model loss of the MSE metric for the proposed multivariate Bi-LSTM model.  
Table 6 also indicates that the multivariate Bi-LSTM model had smaller prediction error values for the MAE, MSE, and RMSE metrics than those of the multivariate LSTM and CNN-LSTM models. Moreover, the multivariate Bi-LSTM had better accuracy than those of the other models. We plotted the absolute error values of the first 100 observations using different multivariate prediction models, as shown in Figure 5. In the figure, we can clearly see that the multivariate Bi-LSTM had smaller absolute error values in all 100 observations. This proves that the time series data in the forward direction do not provide effective information other than to integrate the backward direction of the time series, causing the LSTM and CNN-LSTM models to become less effective;  
2. GWA-T-13 dataset: Table 7 lists the results obtained by the univariate and multivariate Bi-LSTM models on the GWA-T-13 dataset. The multivariate Bi-LSTM model outperformed the univariate one, reducing the RMSE prediction error by 3.4% compared with that of the univariate model. Figures 6 and 7 show the results obtained on the testing dataset. The multivariate Bi-LSTM model had better prediction results. We can clearly see this by observing some high peak points where the multivariate model had closer predicted values to the actual values than those of the univariate model. Figure 8 visualizes the absolute error for the first 100 observations of both models and shows that the multivariate Bi-LSTM had smaller prediction absolute error values than those of the univariate one. When compared with other well-known multivariate models listed in Table 8, the Bi-LSTM model also had better prediction accuracy than those of the LSTM and CNN-LSTM models. It reduced the RMSE prediction error by 6.7% and 5.4% when compared with the LSTM and CNN-LSTM models, respectively. Figures 9 and 10 show the predicted values versus the actual values on the testing dataset. Figure 9 tends to over-predict over time when compared with Figures 7 and 10. Figure 11 shows the absolute error for the first 100 observations of the three multivariate models, where Bi-LSTM had smaller prediction absolute error values than those of the LSTM and CNN-LSTM models.  
Figure 8: Absolute error for the first 100 observations of the univariate and multivariate Bi-LSTM models using the GWA-T-13 dataset.  
Figure 6: Result of univariate Bi-LSTM using the GWA-T-13 dataset.  
Figure 7: Result of multivariate Bi-LSTM using the GWA-T-13 dataset.  
Figure 11: Absolute error for the first 100 observations of the multivariate models using the GWA-T-13 dataset.  
Figure 10: Result of multivariate CNN-LSTM using the GWA-T-13 dataset.  
Figure 9: Result of multivariate LSTM using the GWA-T-13 dataset.