## 3 Methods
### Model Validation and Comparison  
The developed machine learning models were evaluated using a one-year-out cross-validation method with receiver operating characteristic (ROC) curves and the area under the ROC curve (AUC) values. A ROC curve is frequently used in forest fire risk and landslide susceptibility mapping studies because of its threshold independent interpretation [52; 53; 54; 55; 56]. Using the false positive rate and the true positive rate, the ROC curve shows the trade-off of the two rates. The prediction accuracy is usually  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Time Scale** & **Fire** & **Non-Fire for the Integrated Model** & **Non-Fire for the Meteorological Model** \\ \hline \hline  & & Hour & \\ \hline
0:00 & 180 & 140 & 771 \\ \hline
2:00 & 168 & 129 & 787 \\ \hline
4:00 & 182 & 129 & 783 \\ \hline
6:00 & 196 & 154 & 782 \\ \hline
8:00  
determined by AUC which can represent the excellent performance if the value is 0.9-1, good for 0.7-0.9, and poor for less than 0.7 [52; 55].  
The small number of forest fire occurrences of a region does not directly mean the low forest fire risk for the region. Thus, after dividing the validation data into four risk classes (low, moderate, high, and very high) based on the equal interval approach except for the low class with the value from 0 to 0.49, the results were additionally evaluated qualitatively through the distribution of risk classes by hourly, seasonally, and yearly. Low class (i.e., very low susceptibility) was assigned for HFRI less than 0.5, referencing the class standard of a national official index (DWI). The rest of the sections were set from 0.5 to 1 as moderate-high-very high by adopting the equal interval method (Table 4). In the previous susceptibility mapping studies, risk classes were usually classified based on the natural breaks method, which results in varied class ranges depending on the training data [7; 10; 57]. On the other hand, in the case of the equal interval, the threshold value does not change by training data, so it is very easy to apply to the timely updated model. HFRI's threshold for class standards is very similar to DWI's original class standards provided by the Korea Forest Service.  
HFRI derived from the ensemble model was compared to the DWI and revised FFMC, which is a FFMC module of the Canadian Forest Fire Information System customized for South Korea [58]. According to [19], FFMC predicts the moisture quantity of fine fuel, and the larger the number, the higher the probability of ignition. In the study area, FFMC was more closely related to forest fires than CFWI [58; 59], and even could be more correlated with forest fires when adjusted considering the environmental characteristics in South Korea (i.e., revised FFMC) [58]. Since DWI data were only available from 2014 to 2017, this period was used to compare two indices based on the percentile rank and distribution of forest fire risk pixels by class. The percentile rank was adopted for a relative comparison because these indices have different ranges. The percentile rank is the percentage of observations that are less than or equal to the specific observation value.