## 4 Model Evaluation
### On-Track Comparison of FFNN with the CALIOP and PHYS Model  
To further understand the differences between CALIOP, FFNN, and PHYS, we compared the dust detection from these three products on a pixel-to-pixel basis for the entire testing dataset on the CALIOP track excluding latitudes that are higher than 70\({}^{\circ}\) (Figure 5). For CALIOP and FFNN comparison, we used a CALIOP cloud mask to screen out cloudy pixels. Over land, CALIOP reports an average dust fraction of 51%, which is much higher compared to only 12% over ocean. As explained in Section 2, this is mainly because dust aerosols originate in land and only part of these dust plumes transport over the ocean. The larger portion of over land dust frequency is also partly due to the fact that a dust conservative screening method is used to identify dust plume. In other words, the definition of \"dusty\" in this study requires only one layer of dust to be detected by CALIOP through the entire atmospheric column. Finally, although really rare in the mid-to-high latitude region, it is also possible that CALIOP could misidentifify cirrus clouds as dust if the clouds are too low in altitude. Nevertheless, using CALIOP as the standards, FFNN shows a higher accuracy of prediction dust over both land and ocean. Overland FFNN true positive rate is 83% and false positive rate is 21%. The false positive rate is much lower than the true positive rate, which means that FFNN has the ability to separate surface signals with dust signals. It also could be that, over arid and semi-arid areas, where it is the hardest to detect dust, a high frequency of airborne dust occurs. Over the ocean, FFNN has a lower true positive rate of 33% with a very small false positive rate of 2%. The lower true positive rate indicates that FFNN has trouble separating dust from other aerosol types, which is also included in the clear sky categories. The very low false positive rate is partially due to the fact that, over the ocean, the occurrence of aerosol features might be much smaller than over land. Figure 4: **(a,b)** Spatial dust distribution for all test days. **(a)** CALIOP and **(b)** FFNN shows the dust distribution over CALIOP and VIIRS collocated track. Dust distribution is plotted with \(5\times 5\) lat-lon grid; **(c)** spatial difference between FFNN and CALIOP dust fraction; **(d)** monthly time series of CALIOP and FFNN dust fraction over the entire globe to show the seasonality. When comparing the PHYS model results with the CALIOP classification, there is a larger amount of cloud identified by PHYS when CALIOP products identify as dusty or clear skies (about one third over dusty conditions and about half when it is clear for both overland and ocean). The cloud prediction of PHYS is not generated in the ADP algorithm, but an upper stream cloud mask product [51]. Before the ADP algorithm is attempted, pixels are first filtered to ensure that the observing conditions are suitable for retrieval. Conditions such as clouds, cloud shadows, snow/ice coverage, or extreme viewing/illuminating geometries will be removed before retrieval. The PHYS model only predicts clear sky, dust, smoke, and ash for the remainder of the pixels. One of the known issues of a passive satellite aerosol retrieving algorithm is that optically thick aerosol plumes can be mistaken as clouds due to their similar spectral signals [52]. Another possibility that causes the disagreement in cloudiness between the two methods is the spatial resolution. Although this issue is mysterious, it is beyond the scope of this study. In addition, because Figure 5 only shows the results between 70 degS and 70 degN, another possibility of causing the disagreement, which is due to CALIOP falsely labeling low altitude ice clouds near polar region as dust, is mostly excluded from the statistics. To remove the discrepancies due to cloud masking, we applied the ADP cloud mask to collocated CALIOP and FFNN results. After filtering by the ADP cloud mask, more than one third of CALIOP identified dust is removed over land and ocean.