# Precipitation Estimation Using FY-4B/AGRI Satellite Data Based on Random Forest
## 2 Materials and Methods
### Methods
#### 2.3.4 Model Tuning and Testing  
In the model, there are two important parameters that need to be adjusted: one is the total number of trees 'n' in the forest, which plays a crucial role in the sensitivity of the model's performance. The other is the number of features 'k' available for each node, which more directly affects the performance of the model. It determines the difference between each tree and indirectly affects the stability of the model against a high amount of noise data. Therefore, in the present study, the number of decision trees 'n' ranges from 10 to 1000, and the number of features 'k' ranges from 1 to the maximum number of feature variables of the model (18 during day and 16 at night), respectively, to find the optimal parameters of the model. In RF modelling, there will be some data (about 36.8%) that are never randomly selected, which are called \"out-of-bag data\". These out-of-bag data are not used by the model for training, and sklearn can help us test the model with them. In the classification model, ROC-AUC score of out-of-bag data is used as the evaluation index for parameter adjustments. The ROC-AUC curve is shown in Figure 2. ROC-AUC is one of the most important evaluation indicators to check the performance of a classification model. Receiver operating characteristic (ROC) is a probability curve with false positive rate (FPR) and true positive rate (TPR) as its axes [44]. Area under the curve (AUC) is the area under the ROC curve used to measure the performance of the classifier. The closer the value of AUC is to 1, the better the classifier performance is. In the regression model, mean square error (MSE) of the out-of-bag data is used as the evaluation index for parameter adjustments. MSE is gener  
Figure 2: ROC-AUC curve (The dash line represents the classification ability equal to 0, also known as the pure opportunity line, and the green line represents the ROC curve). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Feature Variables** & **Day** & **Night** \\ \hline \multirow{4}{*}{CWP} & \(VIS_{0,65}\) & \\  & \(VIS_{0,825}\) & \(\Delta T_{3,75}\)–7.42 \\  & \(NIR_{1,61}\) & \(\Delta T_{3,75}\)–10.8 \\  & \(NIR_{2,225}\) & \\  & \(\Delta T_{8,55}\)–10.8 & \(\Delta T_{8,55}\)–10.8 \\  & \(\Delta T_{10,8-12}\) & \(\Delta T_{10,8-12}\) \\  & \(NIR_{1,379}\) & \(NIR_{1,379}\) \\  & \(IR_{13,3}\) & \(IR_{13,3}\) \\ \multirow{2}{*}{WV} & \(W_{6,25}\)–6.95–7.42 & \(W_{6,25}\)–6.95–7.42 \\  & \(W_{6,25}\)–7.42 & \(W_{6,25}\)–7.42 \\ \multirow{2}{*}{Topography} & DEM & DEM \\  & OV & OV \\ Satellite zenith angle & SAZ & SAZ \\ \hline \hline \end{tabular}
\end{table} 
Table 3: _Cont._deviation between the predicted value and the true value of the model. The smaller MSE is, the smaller the error of the model's predictions, which is estimated according to the formula below:  
\[MSE=\frac{\sum_{i=1}^{n}(S_{i}-I_{i})^{2}}{n} \tag{2}\]  
where \(S\) represents model's prediction data, \(I\) represents observation data and \(n\) represents the number of matching points. In addition, in the process of model training, the ratio of non-precipitation and precipitation in the dataset is 4:1, which contains a low number of precipitation pixels. Direct training of randomly selected dataset will lead to serious underestimation of precipitation samples. Therefore, the method of random subsampling is used to reduce the samples of non-precipitation to solve the problem of sample imbalance. The proportion of non-precipitation and precipitation samples is set as 4:1, 3:1, 2:1 and 1:1 for training, to evaluate the effect of the model and select the optimal sample proportion. The precipitation test classification table is used to describe the relationship between model's predictions and observations, as shown in Table 4.  
At the same time, false-alarm ratio (FAR), probability of detection (POD), critical success index (CSI) and equitable threat score (ETS) are introduced to evaluate the accuracy of precipitation identification model [45].