# Neural Network Based Quality Control of CYGNSS Wind Retrieval
## 3 Proposed Quality Control Method
### Neural Network Filter Design  
The NN used for this application consists of a single hidden layer with 10 neurons. The input layer consists of 13 neurons, each for one diagnostic variable and the output layer has one neuron that classifies an input sample as an outlier or a good sample. Only one hidden layer is used as it is a sufficient condition to form any bounded/unbounded convex region in the space spanned by the input [17]. The choice of the number of neurons in the hidden layer is decided by experimentation. In general, a feedforward network can have any shape but the commonly used structure is a pyramidal structure with decrease in number of neurons at each layer away from the input. There is practically no upper limit on the number of neurons to be used in this case as the training population is very high (\(\sim\)4 million). So, 3 different neurons counts are experimented here and the performance plot in terms of PD and FAR at different wind speeds is plotted to make a choice on the hidden layer size. The performance plot is shown in Figure 4.  
The blue curves represent PD and the red curves represent FAR. It can be noticed that all 3 network sizes have a very similar performance in terms of PD and FAR over the entire wind speed range. For this reason, other performance metrics such a computation time, network complexity and % samples removed as outliers are considered when choosing the optimal structure. In terms of computational time, NN size = 15 is the shortest, followed by NN size =10, and the longest is NN size = 5. This is an expected trend, as simpler networks can take larger time for error convergence. Next, in terms of network complexity, NN size = 5 has the least number of tunable parameters, followed by NN size=10 and the largest being NN size =15. The % of samples removed as outliers by NN size = 5 is \(\sim\) 23% of the total data, by NN size =10 is \(\sim\) 20% and by NN size = 15 is \(\sim\)22%. Therefore, after all these considerations, NN size=10 is chosen as the optimal network design for this application. Thus for purposes of quality control application for CYGNSS, the QC filter has two NNs (NN1 and NN2), each trained for a specific wind speed range (0-10 m/s and 10-25 m/s). The NNs are identical in architecture and contain 2 layers with 10 neurons in the hidden layer, as discussed above. The hidden layer is trained with a sigmoid transfer function and a linear transfer function is used in the output. The optimization algorithm used for this is the widely used Levenberg-Marquardt algorithm. \begin{table}
\begin{tabular}{c c} \hline \hline
**Classifier** & **Training Time (secs)** \\ \hline Logistic Regression & 118.43 \\ \hline Decision tree & 78.87 \\ \hline Naïve Bayes & 56.39 \\ \hline K-NN & 49,608 (13.78 h) \\ \hline NN & 6475 (1.8 h) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Training time required for individual classifiers (top). Confusion matrices for different classifiers (bottom). To evaluate the design space of this filter, the definitions of good and outlier samples are varied and the performance metrics are plotted. Understanding the behavior of the filter for different sample definitions can help users understand how the network handles the outliers and choose an optimum definition based on the application requirements. The family of PD and FAR curves are plotted in Figure 5. The blue curves represent PD and red curves represent FAR. The PD metric is affected by the density of outlier samples and the FAR metric is affected by the density of good samples. Changing the wind speed difference thresholds for good and outlier samples will affect the overall performance of this QC filter. For this study, the wind speed difference from the GMF line for a good sample is varied from 1 m/s to 4 m/s and for an outlier is varied from 3m/s to 7 m/s.