## 3 Methodology
### Accuracy Metrics  
In this study, to develop an initial predictive model and improve the performance of the initial model, hyper-parameters for each machine learning are selected and set. The predictive model optimization and optimized hyper-parameters were calculated by performance analysis of the predictive model. The predictive model development and performance were analyzed using the parameters applied. The accuracy was evaluated using the coefficient of variation, the Mean Bias Error (MBE), and the Coefficient of Variation (CV) to evaluate the performance of the predictive model. The MBE means the total error of the predicted value, and the coefficient of variation is a method of analyzing the error through the degree of variance. Using the criteria provided by ASHRAE Guideline 14 (MBE of 10% and Coefficient of variation of the Root Mean Squared Error (CvRMSE) of less than 30%), the accuracy of the prediction model was evaluated, as shown in Equations (2)-(4) below [19; 20]. \[\text{MBE}=\frac{\sum_{i=1}^{n}(P_{i}-M_{i})}{n}\times 100 \tag{2}\] \begin{table}
\begin{tabular}{c c c} \hline \hline
**Category** & \(\mathbf{r}\) & \(\mathbf{r^{2}}\) \\ \hline Outdoor air temperature & \(-0.72\) & 0.52 \\ Source side inlet temperature & \(-0.76\) & 0.58 \\ Source side outlet temperature & 0.59 & 0.35 \\ Load side inlet temperature & 0.60 & 0.39 \\ Load side outlet temperature & 0.32 & 0.10 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Correlation analysis between input data and output data. \[\text{RMSE}=\sqrt{\frac{\sum_{i=1}^{n}(P_{i}-M_{i})^{2}}{n}} \tag{3}\]  
\[\text{C}_{\text{v}}\text{RMSE}=\frac{\text{RMSE}}{\overline{\text{P}}}\times 100 \tag{4}\]  
_P_: Predicted value  
_M_: Measured value (or simulation value)  
_n_: Number of measured data  
\(\overline{\text{P}}\): Average of predicted value  
**4. COP Prediction Model Development with Hyper-Parameter Optimization**  
_4.1. Initial COP Prediction Model Development_  
In this study, an initial coefficient of performance prediction model was developed using an ANN and an SVM model to analyze the suitability for predicting the heat pump performance coefficient by machine learning. At this time, each prediction model was developed using the built-in function of R studio. The initial model performance and optimization performance are compared and analyzed. Table 11 shows the statistical analysis of parameters through the random classification of input variables and training data selected for developing the initial coefficient of the performance prediction model [21]. The training data and the validation data were evenly distributed for all parameters. Statistical analysis of the parameters was performed by random classification of the input variables. The training data selected for developing the initial coefficient of the performance prediction model are shown in Table 12. The training and validation data were distributed evenly for all parameters [22]. The initial ANN utilization prediction model was constructed as an ANN model with a structure of one hidden layer and two hidden neurons with input variables. Table 13 lists the structure of the ANN, and Table 14 shows the structure of the initial prediction model of the SVM [23; 24].