## 3 Results  
As mentioned in Section 2, randomly selected 10% of the data points were used for the model training. The remaining 90% were used as a testing data set to examine the trained models. The overall accuracy of the trained classification model is 97.3% and 99.3% for the trained noise discrimination model. Figures 5 and 6 show the confusion matrix of test data and predicted results using the trained noise discrimination model and classification model, respectively. The noise discrimination model worked relatively well on separating the noise and non-noise data with high accuracy. Only a very few noise data (FNR = 0.3%) were wrongly classified as non-noise data, and most of them were classified as noise again by the classification model. For the non-noise class, 3.1% of them are classified as noise, but they might not all be wrongly classified. In Figure 1d, there are some points with a lower density, which might be noise but mislabeled. Figure 4: A concept diagram of a confusion matrix can be used for evaluating the model prediction. True class is the original labeled class, and the predicted class is the model predicted class. Row summary displays the percentages of correctly and incorrectly classified data for each true class. The column summary displays the percentages of correctly and incorrectly classified data for each predicted class. Figure 5: The evaluation of the noise discrimination model: the confusion matrix chart of the noise discrimination model with the testing data set, which is 90% of all data points. The number in the matrix is the count of data points for each class, blue colors indicate corrected predicted (TP), and red colors incorrectly predicted data (FN). Darker color means a higher percentage accordingly. The row summary is TPR (blue) and FNR (red) for each true class (row), respectively. The column is PPV (blue) and FDR (red) for each predicted class (column), respectively. Figure 6: The evaluation of the classification model: the confusion matrix chart of the classification model with the testing data set, which is 90% of all data points. The number in the matrix is the count of data points for each class, and blanks indicate no such classes are classified. Blue colors indicate corrected predicted (TP), and red colors incorrectly predicted data (FN). Darker color means a higher percentage accordingly. The row summary is TPR (blue) and FNR (red) for each true class (row), respectively. The column is PPV (blue) and FDR (red) for each predicted class (column), respectively. The performance of the classification model varies, depending on the classes. As mentioned above, a small portion of noise data is not identified by the noise discrimination model, but most of them (TPR = 94.6%) are classified as noise by the classification model. The TPR and PPV value of most classes are larger than 93%, which means the model performs quite well in classifying these classes. The rain class is the worst predicted one, with a TPR of 56% but a higher PPV with a value of 70.9%. In other words, the model may underestimate the rain data by around 40%, but for each predicted rain data point, we have around 70% confidence that it is the rain class. Since the training and testing data set could be mislabeled, the results by specific cases need to be assessed. Figure 7 shows the predicted classes of 15 June 2019, along with CNR, \(\beta\) and \(\delta\) values. In this case, the trained model performed well in predicting the classes of clouds and aerosol type I (high \(\beta\) and high \(\delta\)) and the rain that fell around 15:00 UTC. Some data points are classified as aerosol type III (high \(\beta\) and lower \(\delta\)) but are not labeled in Figure 2. We cannot say if these points are wrongly predicted. The depolarization ratio was high in the afternoon, and therefore, these could be some remaining aerosols not cleaned out by the precipitation. However, these data points stayed at a similar height, around 600 m, which is an artifact.