## 4 Statistical Prediction Model
### SWIPE Model Evaluation  
To better optimize the final RF based SWIPE prediction model, the model parameters are tuned iteratively in the SWIPE model training, including the number of trees in the forest (n_estimators), maximum depth of the trees (max_depth), and random split predictor variables (max_features). Figure 4 shows the effect of these parameters on the out-of-box (OOB) score. It indicates that OOB \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Scenario-1** & **Scenario-2** & **Scenario-3** & **Scenario-0** \\  & & & & **(Original)** \\ \hline Weak & 662 & 2388 & 4776 & 79,549 \\ Medium & 662 & 2388 & 2388 & 2388 \\ Severe & 662 & 662 & 662 & 662 \\ Proportion & 1:1:1 & 1:3.6;3.6 & 1:3.6;7.2 & 1:3.6;120 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Numbers of weak, medium and severe convections of four typical sample datasets under different scenarios. Figure 3: The flow chart of SWIPE model training and predicting based on RF algorithm. It contains three steps: First, it tracks potential convective cloud clusters. The second step is to divide the convective storm system dataset into three different types (weak, medium, and severe). Finally, the RF algorithm is used to train and develop a convection intensity classification statistical model — SWIPE. \(\Delta\)T represents the cloud top cooling rate. scores (about 0.96) of all the models hardly change with the variation of the parameters, implying good fitting RF based SWIPE prediction models or low sensitivity of the SWIPE model to parameters. We use the SWIPE model with the n_estimators ranges from 20 to 1000 in this investigation, which is likely to lead to the stable variation of OOB score in Figure 4.  
Generally, some common and important scores must be calculated to evaluate the performance of a prediction model based on the classification confusion matrix. The following ratings in a contingency table are used to access predicted results [24; 47] (see Table 4). \[\text{Probability of Detection, POD}=\text{A}/(\text{A}+\text{B}). \tag{3}\]  
\[\text{False-Alarm Ratio, FAR}=\text{C}/(\text{A}+\text{C}). \tag{4}\]  
\[\text{Critical Success Index, CSI}=\text{A}/(\text{A}+\text{B}+\text{C}). \tag{5}\]  
\[\text{Hit Rate, HR}=(\text{A}+\text{D})/(\text{A}+\text{B}+\text{C}+\text{D}). \tag{6}\]  
To further illuminate the importance of NWP model variables, a new prediction model consisting of only 41 satellite variables is established for comparison purposes, which is marked as Scenario-S (using the same statistical model and the training dataset as Scenario-1, but only satellite parameters  
Figure 4: Effects of total number of trees in the forest (n_estimators), maximum depth of the tree (max_depth), and random split predictor variables (max_features = 7 (upper left), 8 (upper right), 9 (middle left), 10 (middle right), and 11 (lower left)) on OOB scores for the RF classification models of convective storm system. are used as predictors). In this study, in spite of false-alarm detection, we hope the nominally optimal prediction model is able to capture as many severe convective storm samples as possible (meaning a relatively higher POD score). Based on an extra-large amount of training samples for different model parameter tuning, the RF classification model is finally decided using dataset of Scenario-1 with n_estimators = 100, max_depth = 5, and max_features = 10 as the optimal prediction model. Table 5 shows the best performance metrics of convection classification using four independent RF classification models based on four different scenarios of Scenario-1/2/3/0 described above. The specific model parameters are also listed in Table 5 below. From this table, the optimal RF model under Scenario-1 can generate the highest POD scores of 0.66 and 0.70 for severe and medium convective storm cases, respectively.