# Neural Network Based Quality Control of CYGNSS Wind Retrieval
## 4 Results
### Algorithm Performance Analysis  
To assess the skill of the quality control algorithm, first the validation metrics, PD and FAR, are examined in Figure 7. These metrics are based on the design parameters discussed in the previous section. The optimal range of operation for this filter is \(\sim\)5 m/s to 17 m/s. In this range the FAR for good samples is consistently \(<\)20% and the PD for outliers is \(>\)75%. The peak performance is between 6-14 m/s where FAR \(<\)10% and PD is \(>\)80%. This is also the region of maximum data density as the wind speed distribution has a peak near 7 m/s.  
Next, the skill of the filter is assessed by looking at the ratio of number of outliers identified by the filter to the total number of outliers for a range of wind speed differences. This is shown in Figure 8. The \(x\)-axis is the difference between CYGNSS wind speed and the GMF line. As per our validation criteria, we have defined any sample as an outlier if the difference is greater than 5 m/s. The 5 m/s threshold is shown in red. It can be observed that \(\sim\)70% of the outliers are rightly identified for wind speed difference \(\sim\) 5 m/s and the filter eliminates close to \(\sim\)100% of outliers with wind differences \(>\)10 m/s.  
Figure 7: PD and FAR metrics for the CYGNSS test dataset.  
To understand Figure 8 better, we look at the distribution of outliers (wind speed difference \(>=5\) m/s) at different MERRA-2 wind speed bins before and after applying the filter. This data distribution is shown in Figure 9. The red distribution shows the density of outliers in the original dataset and the blue shows the distribution of outliers after applying the filter. Firstly, a very significant decrease in the outlier population can be observed after filtering. The filtered dataset has approximately 4 times less outliers. In the original dataset, most of the outliers are present between 5-10 m/s which is also the peak region for wind speed distribution. In this region the filter has been able to remove a large proportion of the outliers. Next, in the filter design section the low PD and high FAR at high winds region was discussed. Though at first, it may appear as if the filter cannot operate in this wind speed region, the distribution of outliers in this region (plot on the top right) shows that the number of outliers is almost an order of magnitude smaller after the filtering process, indicating that the filter can operate efficiently in this region but the low sample density in the region does not reflect this capability of the filter in the PD and FAR metrics.  
Figure 8: Ratio of outliers rightly identified by the filter to the actual no. of outliers vs. wind speed difference.  
Finally, the total wind speed distribution of the dataset before and after applying the filter is plotted in Figure 10. After applying the filter, \(\sim\)20.5% of the data have been removed by the filter as outliers. From Figure 10 it can be observed that the largest difference in density occurs at high wind speeds (>18 m/s). This is partly due to the high FAR of the filter in this region and partly due to large scatter in the data in this region. A substantial difference in density can also be observed at very low wind speed regions (<3m/s), again owing to the high FAR of the filter in this region.