# Evaluating Satellite Sounding Temperature Observations for Cold Air Aloft Detection
## 4 Discussion  
It can be challenging to effectively communicate what, when, and where observations have skill to forecasters [27; 28; 29]. For developers, NUCAPS provides a series of metrics to understand the sources in their uncertainty and diagnose which algorithm steps did not pass QC. A simplified NUCAPS quality control flag was developed for the NWS, which identifies where the combined IR and MW retrieval passed, where the MW-only passed, or where neither passed [15]. This quality control is assigned to the full profile, but primary degradation will occur because of clouds, and thus have a greater impact on the lower troposphere (below 500 hPa). The assessment of CAA may require a different quality indicator altogether, as CAA occurs in the upper troposphere and lower stratosphere (above 500 hPa). From Figure 6 and the corresponding discussion in Section 3, the diagnostic metrics are very different for the CAA detection states and are, in theory, potentially useful for diagnosing the retrieval quality. However, in operations, forecasters do not have time to perform a complex assessment of these diagnostic parameters. Instead, we recommend the development of a simplified quality control for CAA based on the parameters evaluated in this study.  
More orbits may improve both the data quality and utility for forecasters: When combined, the true positive and false positive made up 65.3% of matchups examined. The occurrence of false negative cases (the remaining 34.7%) can be mitigated by including more satellites. For example, if underlying cloud scene changes between overpasses, the retrieval skill may improve and provide updated guidance to forecasters on CAA detection. High latitude regions get more frequent LEO satellite overpasses than midlatitudes, so even one additional satellite will produce significantly more observations to detect CAA [30]. At present, there are web-based tools that blend all available sounders to monitor CAA [31] whereas only a single satellite (Suomi NPP priori to March 2019 and NOAA-20 since August 2019) is available NWS forecasters through the SBN.  
There are some caveats to the analysis presented in this paper. The COSMIC data that is used as truth may not be directly comparable to NUCAPS. For instance, in the tropics, COSMIC measurements are limb views and average over \(\sim\)200 km while the sounder FOR is between 50 and 150 km. Spatial mismatches can produce biases of 1 K between COSMIC and sounder datasets in high latitudes [26]. Furthermore, not all instances of CAA pose an aviation hazard. For example, if the layer of cold air is thin, it poses little risk to aircraft and would be below the detection capabilities of NUCAPS, which has 1-3 km vertical resolution depending on the pressure layer. Furthermore, diagnostics in this study were performed on pressure levels, not flight altitude. The cold air may have existed outside of the commercial flight zone. Nonetheless, 8 out of 18 CWSU forecasters who visually inspected NWP model and in situ data said they had high confidence in NUCAPS for CAA detection [3]. While our evaluation criteria may not exactly match those used in operational forecasting, the \(\sim\)50% correct detection rate in this study is consistent with forecaster evaluation and they found value in using NUCAPS for CAA detection.