# Evaluating Satellite Sounding Temperature Observations for Cold Air Aloft Detection
## 3 Results
### Diagnosing Causes for True and False CAA Detection
NUCAPS uses a threshold of 3.0 K for _eta_rej_ as one of the metrics that informs its retrieval quality flag. The above diagnostic metrics are objectively evaluated against the four possible detection states (true positive, true negative, false positive, false negative) using a logistic regression, which is given by  
\[\log(\frac{p}{1-p})=\beta_{0}+\beta_{1}x_{1}+\beta_{2}x_{2}+\ldots+\beta_{n}x _{n} \tag{1}\]  
where \(p\) is the probability, \(\beta_{i}\) is the regression coefficient, and \(x_{i}\) are the are diagnostic parameters. While useful for fitting data to binary outcomes, the resulting coefficients (\(\beta_{0}\), \(\beta_{1}\), \(\beta_{2-}\)) from the logistic regression describe the log of the odds ratio (\(p/(1-p)\)), which is difficult to interpret. Thus, we exponentiated all terms to obtain the odds ratios:  
\[\frac{P}{1-p}=\exp(\beta_{0}+\beta_{1}x_{1}+\beta_{2}x_{2}+\ldots+\beta_{n}x_{ n}) \tag{2}\]  
The odds ratio in Equation (2) can be interpreted as \"the probability of success to the probability of failure\". Exponential coefficients show the percent increase in the odds of the state being true. So, if an exponential coefficient is greater than one, that variable will \"increase\" the odds, whereas exponential coefficients less than one \"decrease\" the odds of the state being true. For cases where the exponential coefficient is equal to one, there is little impact on the odds of state occurring. To simplify our discussion, we will describe our results in terms of how diagnostic metrics increase the odds of the state being true. The exponential coefficients were tested for statistical significance (\(<\)0.05 of the null hypothesis) using the z score. We use the statistically significant exponential coefficients to examine the relationship between NUCAPS diagnostic metrics and CAA detection states. Figure 6 shows the results of the multinomial logistic regression for the true positive, true negative, false positive, and false negative detection states. The true positive (top left, N = 137) state is more likely for profiles with lower values of _eta_rej_. As described earlier, lower values of _eta_rej_ indicate lower systematic uncertainty due to clouds, which in turn appears to improve NUCAPS' detection of CAA. Larger lapse rate values (_lr250-100_ and _lr500-200_) also increase the odds of the true positive state. This is expected since measurement errors are lower when lapse rates are higher. Finally, low values of _dof_temp_ increase the odds of the true positive state. However, DOF is a full column metric so it is not possible to determine which pressure levels in the column experience a decrease in the vertical resolution. For instance, there could be a decrease in the vertical resolution below 500 hPa, which would not impact CAA detection. So, while the relationship is statistically significant, it is not possible determine why this association is present without examining the full AKM, which is not available in the operational product. Most matchups (N = 1026) fall into the true negative state (top right). Since this is most cases, the true negative state is useful for examining what these diagnostic metrics typically look like for successful retrievals in study region. Increases in the _dof_temp_ metric strongly influences the true negative state. We can in general say that true negative profiles have higher information content than the other states. The next two \"false\" states are cases where NUCAPS failed to accurately detect CAA. High _cloud_frac_ values have roughly a 280% increase on the odds ratio of the false negative state (bottom left, N = 99). However, the odds ratio for the false negative state also increases when _eta_rej_ values are smaller. Thus, high _cloud_frac_ values may impact the upper level detection of air temperature in NUCAPS, even when uncertainty due to the clouds (_eta_rej_) is low. Finally, the false positive (bottom right) state included 49 matchups, the smallest sample of the four states.