## 3 Application and Results
### Evaluation of Water Loss Management Efficiency and Determination of Formulation of Water Loss
The multiple correlation coefficient of MRA analysis is the Pearson correlation coefficient for the relationship between the dependent variable according to the independent variable and the predicted value through the regression model. The average value for the coefficient of determination is 0.55, which is not a high correlation, and it would not be the best data classification for reflecting the characteristics of the data. This result means that it cannot be evaluated that the observed values of the sample are well clustered around the regression line, and that there is a difference between the predicted values and the observed values. Moreover, because the data classification was based on the leakage ratio and the revenue water ratio, it can meet the goal to improve the efficiency of leakage management in the future, but the model formulation predicts the increase in the revenue water ratio and the decrease in the leakage ratio It shows some error with the predicted value. The prediction values of the model formulation show differences in accuracy depending on the classification of the data, even if the same data are used [17]. In this study, the leakage ratio and the revenue water ratio were classified according to the water loss management activity, which means that each variable is independent of each other in the data classification process. However, because the water loss management activities such as pipe and meter replacement reflect the time-sequential characteristics of the revenue water ratio and the leakage ratio after recovery, it is necessary to consider the spatiotemporal characteristics when classifying the data. Therefore, in this study, when classifying the data, the increase in the revenue water ratio and the decrease in the leakage ratio affect the water loss management activity of the immediately preceding year. In this study, the time series and regional characteristics were considered, and a formulation was derived. Table 6 shows the data classified based on spatial and temporal characteristics. As shown in Table 6, there are a total of 22 target areas classified based on spatial and temporal characteristics, and the model formulation was derived for a total of 11 target areas, except for small- and medium-sized cities with less than four datasets. The results are shown in Table 7.  
The multiple correlation coefficient of the model formulation considering spatiotemporal characteristics is 0.87 on average (maximum: 0.99, minimum: 0.75), which shows a higher correlation than the data classified based on the revenue water ratio and the leakage ratio in both analysis approaches (i.e., MRA, DNN). This means that the observations of the sample are well clustered around the regression line, and the regression model predicts the dependent variable well. In particular, BH and JE showed a coefficient of determination of almost 1.00. This means that there are no missing values in the data; these two cities are smaller cities relatively, and the annual water loss management actions were more actively performed than in other areas. Accordingly, it is judged that the improvement of the revenue water ratio and the leakage ratio is larger than others. Moreover, in the case of NJ, the gap in the coefficient of determination presented the largest between MRA and DNN. This is because NJ has a distinct time-series characteristic compared to other cities. Therefore, it is thought that in the case of NJ, data analysis considering the characteristics of time series is more appropriate than independent data analysis. Through the comparison of the above data classification methods, the increase or decrease in the leakage ratio/revenue water ratio by the water loss management activity of local water distribution systems can construct a more effective model for classification, considering both local and temporal characteristics.