# Deep Learning-Based Autoscaling Using Bidirectional Long Short-Term Memory for Kubernetes
## 6 Experiment and Evaluation
### Evaluation Metrics  
To evaluate the accuracy of the proposed model, we used four metrics as shown in Equations (6)-(9), respectively. These metrics are used to compare the prediction accuracies of the models. We define \(\mathbf{Y_{i}}\) as the actual value, \(\mathbf{\hat{Y}_{i}}\) as the forecast value, and \(\mathbf{\hat{Y}_{i}}\) as the mean value of \(\mathbf{Y}\). In addition, we also compare the average prediction speed of 30 tries that each model takes for prediction. The mean square error (MSE) in Equation (6) calculates the average squared difference between the forecast and the observed values. The root mean square error (RMSE), expressed as Equation (7), represents the square root of two of the differences between the forecast and observed values. The mean absolute error (MAE), expressed as Equation (8), represents the average over the test sample of the absolute differences between the prediction and actual observation, where all individual differences have equal weights. The metrics MSE, RMSE, MAE are used to evaluate the prediction error of the models, where a smaller value indicates a higher prediction accuracy and vice versa. The coefficient of determination, denoted as \(R^{2}\), is shown in Equation (9) as a goodness-of-fit measure for linear regression models. It is the square of the coefficients of multiple correlations between the observed outcomes and the observed predictor values. The higher the \(R^{2}\), the better the model fits the data. \[MSE=\frac{1}{n}\sum_{i=1}^{n}(Y_{i}-\hat{Y}_{i})^{2} \tag{6}\]  
\[RMSE=\sqrt{MSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(Y_{i}-\hat{Y}_{i})^{2}} \tag{7}\]  
\[MAE=\frac{1}{n}\sum_{i=1}^{n}\bigl{|}(Y_{i}-\hat{Y}_{i})\bigr{|} \tag{8}\]  
\[R^{2}=1-\frac{\sum_{i=1}^{n}(Y_{i}-\hat{Y}_{i})^{2}}{\sum_{i=1}^{n}(Y_{i}- \hat{Y}_{i})^{2}} \tag{9}\]  
Furthermore, to evaluate the provision accuracy and elastic speedup of the autoscaler, we used system-oriented elasticity metrics endorsed by the Research Group of the Standard Performance Evaluation Corporation (SPEC) [52], as shown in Equations (10)-(14). These metrics have also been used by other researchers [16; 53]. We denote \(\mathbf{r_{t}}\) and \(\mathbf{p_{t}}\) as the required resource and provide resources at time \(\mathbf{t}\), respectively. \(\mathbf{\Lambda t}\) is the time interval between the last and the current changes in the resource either in the required \(r\) or provided \(p\). For resource provisioning accuracy, the under-provisioning metric \(\mathbf{\Theta_{U}}\) defines the number of missing pods that are required to meet the service-level objective (SLOs) normalized by the measured time. The over-provisioning metrics \(\mathbf{\Theta_{O}}\) identify the number of surplus pods provided by the autoscaler. The range of \(\mathbf{\Theta_{U}}\) and \(\mathbf{\Theta_{O}}\) is from 0 to infinity \(\infty\), where 0 is the best value. \(\mathbf{T_{U}}\) and \(\mathbf{T_{O}}\) correspond to the duration of prediction when the system is under-provisioned and over-provisioned, respectively. The range of \(\mathbf{T_{U}}\) and \(\mathbf{T_{O}}\) are from 0 to 100, and the best value is 0 when there is no under- or over-provisioning during measurement time. Finally, the elastic speedup, \(\epsilon_{n}\), is the autoscaling gain compared to the no autoscaling case, where \(\mathbf{a}\) are cases when using autoscaling and \(\mathbf{n}\) are otherwise. If the \(\epsilon_{n}\) has a value greater than 1, the proposed auto-scaler has an autoscaling gain over no auto-scaler and vice versa.