# Evaluating Satellite Sounding Temperature Observations for Cold Air Aloft Detection
## 3 Results  
### Evaluating NUCAPS Skill in Detecting CAA  
Figure 4a shows four profiles from the 19 matchups between COSMIC (black dashed line) and NUCAPS (solid blue lines) from 26 February 2018. When compared to Figure 2, COSMIC and NUCAPS temperature profiles are much smoother than radiosondes since both products have a lower vertical resolution. The shaded blue box in Figure 4a indicates the temperature and pressure criteria for CAA. If either the COSMIC profile or any one of the NUCAPS profiles enters the blue box, then CAA is detected. The top two panels show profiles where CAA is detected in both COSMIC and NUCAPS while the bottom two show where CAA is not present in either product. In this study, we examined 1311 NUCAPS and COSMIC matchups, of which 236 COSMIC profiles detected CAA. NUCAPS profile matchup counts vary due to quality control and scan angle, since we only use profiles where the combined infrared and microwave retrievals was successful and there are more potential NUCAPS matches at nadir than at the scan edge (see Section 2.1). It is helpful to understand NUCAPS skill in retrieving air temperature as a function of pressure. As described in Section 2.1, the unitless AKMs help diagnose how much information is contributed to the retrieved variable from the observed radiances. As noted above, low AKM values indicate that the retrieved variable will more closely resemble the a priori. In contrast, higher AKM values indicate vertical levels where the radiances more significantly contribute to the retrieved value. Figure 4b shows the air temperature mean of the AKM diagonal as a function of pressure in the study region on 26 February 2018, which was displayed from an offline calculation of NUCAPS. Through the entire column, NUCAPS has some skill in measuring air temperature. Within the solid red box (500 to 150 hPa), the AKM ranges from 0.15 to 0.20. Note that while AKMs are always calculated in NUCAPS, they are not routinely written to the operational NUCAPS product and are thus not available for use in forecasting. In Section 3.2, we will instead examine the degrees of freedom of the signal, the Trace of the AKM for the entire profile, which is available within the operational NUCAPS product. NUCAPS' skill was assessed using 236 COSMIC profiles that detected CAA using 90 days of matchups from January to March 2018. Table 1 shows the four possible states: True positive, true negative, false positive, and false negative. The first two \"true\" states indicate successful detection while the second two \"false\" states indicate failed detection. Our criteria \"detected\" was met if the NUCAPS air temperature fell below 208 K in any matchup profile for at least one pressure level between 500 hPa and 75 hPa. The pressure range is broader than commercial flight zones to account for daily variability in the tropopause height. COSMIC detected CAA if the air temperature fell below 208 K for at least one vertical level in the same pressure range. If all air temperatures were above 208 K between 500 hPa and 75 hPa, the profile is classified as \"Not Detected.\"  
Figure 5 shows NUCAPS correctly detected CAA 48.1% (true positive) of the time, incorrectly detected CAA 17.2% of the time (false positive), and incorrectly missed the CAA 34.7% (false negative). Most matchups are true negatives but are not included in the figure to better focus on the states that detect CAA. The approximately \(\sim\)50% true positive rate means that NUCAPS has reasonable skill in detecting CAA for the study period. In the next section, we describe the causal relationship between CAA detection and the environmental factors that can impact NUCAPS retrieval quality. \begin{table}
\begin{tabular}{c c c} \hline \hline
**State** & **NUCAPS** & **COSMIC** \\ \hline True Positive & Detected & Detected \\ True Negative & Not Detected & Not Detected \\ False Negative & Not Detected & Detected \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A summary of detection states.