# An Efficient Multivariate Autoscaling Framework Using Bi-LSTM for Cloud Computing
## 4 Experiment and Evaluation
### Metric for Validation  
Three standard statistical metrics, mean absolute error (MAE), MSE, and RMSE, were used to evaluate the prediction accuracy of the models. The equations of these metrics are expressed in Equations (22)-(24), respectively.  
\[MAE =\frac{1}{n}\sum_{i=1}^{n}\lvert(v_{i}-l_{i})\rvert \tag{22}\] \[MSE =\frac{1}{n}\sum_{i=1}^{n}(v_{i}-l_{i})^{2}\] (23) \[RMSE =\sqrt{MSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(v_{i}-l_{i})^{2}} \tag{24}\]  
We denote \(v_{i}\) and \(l_{i}\) as predicted and true values, respectively. The MAE is the difference between the predicted and true values. The MSE calculates the average squared difference between the predicted and true values. The RMSE is the square root of the average of the squared differences between the prediction and true observation. These  
\begin{table}
\begin{tabular}{c c} \hline Number of layers & 2 layers (forward and backward) \\ Number of features & 2 \\ Input size & 3 neural cells \\ Number of hidden units & 50 \\ per neural cell & \\ Loss function & MSE \\ Batch size & 64 \\ Epochs & 10 \\ Activation function & ReLU \\ \hline \end{tabular}
\end{table}
Table 2: Multivariate Bi-LSTM model configuration.  
metrics are the three most commonly used metrics for measuring the accuracy of continuous data, where smaller values indicate higher prediction accuracy and vice versa.  
To evaluate the resource provisioning efficiency, we applied the system-oriented elasticity metrics presented by the Research Group of the Standard Performance Evaluation Corporation (SPEC) [29]. Various studies [3; 9; 30] have also used these metrics to evaluate the resource provisioning efficiency of their autoscalers. The equations of the metrics are shown in Equations (25)-(29), where \(d_{n}\) and \(p_{n}\) are the resource demand and provisioned resources, respectively.  
The under-provisioning resource metric, \(\Theta_{U}\), calculates the number of required VMs that must be provisioned to meet the service-level objectives normalized by the measured time. The over-provisioning resource metric, \(\Theta_{O}\), calculates the number of excess VMs that are provisioned by the autoscaler. The values of \(\Theta_{U}\) and \(\Theta_{O}\) range between 0 and infinity, where 0 is the best value. \(T_{U}\) and \(T_{O}\) are the durations the system is under-provisioned and over-provisioned, respectively. \(T_{U}\) and \(T_{O}\) have values ranging from 0 to 100, with 0 being the optimal value when no under-provisioning or over-provisioning occurs throughout the measurement period. Finally, the elastic speed-up metric, \(\epsilon_{na}\), is the efficiency of the autoscaling compared with that of the no-autoscaling case, where \(na\) indicates no autoscaler is used and \(a\) indicates otherwise. The default \(\epsilon_{na}\) value of \(na\) is 1.0. If the proposed autoscaler (\(a\) case) has an \(\epsilon_{na}\) value greater than 1.0, it means that the autoscaler has an autoscaling gain over the no-autoscaler case; otherwise, it has none.  
\[\Theta_{U}[\%] =\frac{100}{T}\sum_{t=1}^{T}\frac{max(d_{n}-p_{n},0)}{d_{n}}\Delta t \tag{25}\] \[\Theta_{O}[\%] =\frac{100}{T}\sum_{t=1}^{T}\frac{max(p_{n}-d_{n},0)}{d_{n}}\Delta t\] (26) \[T_{U}[\%] =\frac{100}{T}\sum_{t=1}^{T}max(sgn(d_{n}-p_{n}),0)\Delta t\] (27) \[T_{O}[\%] =\frac{100}{T}\sum_{t=1}^{T}max(sgn(p_{n}-d_{n}),0)\Delta t\] (28) \[\epsilon_{na} =\left(\frac{\Theta_{U,na}}{\Theta_{U,a}}\frac{\Theta_{O,na}}{ \Theta_{O,a}}\frac{T_{U,na}}{T_{U,a}}\frac{T_{O,na}}{T_{O,a}}\right)^{\frac{1} {4}} \tag{29}\]