# Spaceborne GNSS-R for Sea Ice Classification Using Machine Learning Classifiers
## 4 Results  
Although the OSISAF SIT products are provided continuously, there is a lack of MYI information from May to October most of the time in the OSISAF SIT products. In addition, the TDS-1 data is very sparse from 2015 to 2017 as the GNSS-R receiver was working only two days in an eight-day cycle. Therefore, the TDS-1 datasets collected in 2018 are selected according to the availability of MYI information of OSISAF SIT products. Figure 9(a) presents the data availability status of both OSISAF SIT products and TDS-1 data that are used in this study. The number of OW, FYI, and MYI samples every month are shown in Figure 9(b). According to the statistics, a total number of 460,685 samples, including 165,434 OW samples, 266,680 FYI samples, and 28,571 MYI samples, are used in this study. \begin{table}
\begin{tabular}{c c} \hline \hline
**Evaluation Metrics** & **Equation** \\ \hline _Accuracy_ & \(\frac{TP+TN}{TP+FP}+FN\) \\ _Precision_ & \(\frac{TP+FP}{TP}\) \\ _Recall_ & \(\frac{TP+FN}{TP}\) \\ _F-value_ & _Recall-Precision_ (\(1+\beta^{\prime}\)) \\ _G-mean_ & \(\frac{TP+TN}{TP+FN}\) \\ _Kappa coefficient_ & \(\frac{2\cdot(TP-TN-FP)}{(TP+FP)\cdot(FP+TN)+(TP+FN)\cdot(FN+TN)}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The definition of evaluation metrics. Figure 9: The definition of the confusion matrix. As illustrated in Section 3, the sea ice classification is implemented using a two-step method. The first step aims to discriminate OW from sea ice using the RF and SVM classifiers. During the RF model training, the number of trees is set to different values ranging from 10 to 200, in order to find the best classifier. The number of estimators is finally set as 70 in this study. After identifying the OW, the sea ice samples are employed to classify FYI and MYI in the second step using the RF classifier. It is worth noting that the number of FYI samples is about nine times that of MYI samples. When randomly selecting samples from the whole FYI and MYI dataset as the training set, only a small amount of MYI samples are selected out. The proportion gap between FYI and MYI samples in the training set is too large and the training model may not grasp the features of MYI. Therefore, 30% of MYI samples are randomly selected as training set and the number of FYI samples is controlled to be three times that of MYI. The confusion matrix of RF and SVM classifiers are presented in Figure 11, which presents the classification results of each class. Table 4 presents the evaluation metrics which are computed according to equations listed in Table 3.  
Figure 11: Confusion matrix of (**a**) random forest (RF) and (**b**) support vector machine (SVM) for open water (OW)-sea ice classification. Confusion matrix of (**c**) RF and (**d**) SVM for first-year ice (FYI)-multi-year ice (MYI) classification. Figure 10: (**a**) The availability of both TDS-1 and OSISAF SIT data, which includes the information of open water (OW), first-year ice (FYI), and multi-year ice (MYI) in 2018. (**b**) Statistics of OW, FYI, and MYI samples every month. Table 4 demonstrates the evaluation metrics through validating with the OSISAF SIT maps using the test dataset. As shown in Figure 11a,b, a total of 322,480 samples, which include 109,559 OW and 212,921 sea ice samples, are applied for testing. The overall accuracy of RF and SVM classifiers for OW-sea ice classification is 98.83% and 98.60% respectively, which are comparable to the results in previous studies [15; 38; 40]. Similarly, the FYI-MYI classification results are evaluated using 20,000 FYI samples and 240,967 MYI samples (Figure 11c,d), which results in an overall accuracy of 84.82% for RF, whereas 71.71% for SVM. The performance of RF and SVM for OW-sea ice classification is comparable, whereas RF outperforms SVM significantly in the FYI-MYI classification. The overall space distribution of classification results is shown in Figure 12, which depicts the distribution of predicted and reference types.