# Domain Adaptation for Satellite-Borne Multispectral Cloud Detection
## 8 Results
### Performance on Uhotica CogniSAT-XE1  
Once the source and target cloud detectors have been obtained, we executed them on the Uhotica CogniSAT-XE1 [13] and evaluated their performance. The effects of domain adaptation by the proposed methods are visualised in Figure 11. Table 6a shows the ACC and FP results for the Sentinel-2-trained cloud detectors before and after domain adaptation, as well as the memory footprints and inference times of the cloud detectors. Table 6b shows the same types of results as Table 6a but for Landsat-9-trained cloud detectors. Appendix B supplements Table 6 with confusion matrices that facilitate the calculation of additional metrics such as precision, recall and F1-score. Table 7 zooms in on the execution times of the TTA methods for the Sentinel-2-trained cloud detectors, as measured on the Advantech ARK-1123L [82] embedded computer, where 16 test-time samples were used for DUA and a batch size of 1 was used for Tent. Our results confirm the following:  
* For offline adaptation, we can solve Problem 1 in Section 3.3.1 by employing the FISH Mask and, thereby, enabling more sophisticated models to be deployed and updated remotely through the thin uplink channel. As expected, we found that this adaptation approach outperforms the TTA approaches by a large margin. Note that CloudScout and ResNet50 models with a mask sparsity level of 25% and 1%, respectively, were only evaluated here. * For online adaptation, while Tent has a slight performance advantage in ACC over DUA, the advantage does not extend to FP rate, and furthermore, Tent is 60-100% more computationally expensive than DUA. Nevertheless, both TTA methods are evidently viable on satellite-borne edge compute hardware. Other findings from our results are as follows:  
* Quantising the weights from FP32 to FP16 had negligible effects on model performance, as well as the added benefit of reducing the memory footprint by two-fold. * ResNet50 models had a faster inference time (per sample) than CloudScout models, which is surprising since they are \(\approx\)18\(\times\) larger in size. Figure 10: Effects of DUA and Tent on the cloud detectors in Table 5. SOURCE ONLY refers to training cloud detectors only on source data. Error bars represent standard deviations over 10 runs with different random seeds. (**a**) Model performance of the cloud detectors in Table 5a. (**b**) Model performance of the cloud detectors in Table 5b. \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline  & & & \multicolumn{4}{c}{**SOURCE**} & \multicolumn{2}{c}{**FISH Mask**} & \multicolumn{2}{c}{**DUA**} & \multicolumn{2}{c}{**Tent**} \\  & & & \multicolumn{2}{c}{**ONLY**} & & & & & & & \\
**Model Settings** & **Memory** & **Time** & **ACC** & **FP** & **ACC** & **FP** & **ACC** & **FP** & **ACC** & **FP** & **ACC** & **FP** \\  & **Footprint** & & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** & **(\%)** \\ \hline CloudScout-3-S2 & 2.60 & 2252 & 66.40 & 0.80 & 89.60 & 7.20 & 83.60 & 6.40 & 81.20 & 5.20 \\ CloudScout-8-S2 & 2.60 & 2015 & 52.00 & 48.00 & 90.00 & 1.20 & 59.60 & 40.00 & 64.00 & 34.40 \\ ResNet50-3-S2 & 47.00 & 1245 & 90.80 & 1.60 & 88.40 & 1.60 & 88.80 & 2.00 & 90.80 & 5.20 \\ ResNet50-8-S2 & 47.00 & 1346 & 56.80 & 43.20 & 95.20 & 1.20 & 68.40 & 30.80 & 70.40 & 27.60 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Model performance of the source cloud detectors in Table 5 with domain adaptation (FISH Mask, DUA and Tent) and without domain adaptation (SOURCE ONLY) on the Ubotica CogniSAT-XE1. Red text and green text show the negative and positive effects on performance, respectively. Comparison also includes (i) memory footprint (in MB using FP16 weights), and (ii) inference time (in ms per sample). Figure 11: Visualisation of CloudScout-3-S2 performing cloud detection on multispectral data from Landsat 9, where the subcaptions specify the domain adaptation methods for which the subsequent statements are valid.