# Semantic Image Segmentation Using Scant Pixel Annotations
## 6 Conclusions and Future Work  
The SSPA, a novel approach for generating high-performing deep learning models for semantic segmentation, was presented. The SSPA approach seamlessly combines psuedo-segmentation masks are automatically generated using image processing methods with active learning to generate a sequence of segmentation models using minimal manual annotations by experts. The segmentation model output by the SSPA approach is shown to achieve high-quality segmentation while using a fragment of expert annotations in comparison to supervised learning methods. An ensemble of marker-based watershed segmentation (MC-WS) algorithm modules were used to first generate pseudo-segmentation masks because of their superior performance, which were then iteratively refined by experts to generate a series of segmentation models. In each iteration, the model prediction entropy values of images were used to select top-k high entropy (low entropy) images to be given to experts for annotation. The experts were directed to pixels with the most model prediction uncertainty to manually assign labels and use them with other images to generate the next model. The segmentation model generated when no more improvements are feasible is finally output by the SSPA.  
Our experiments verify that our approach achieves superior results on all the datasets considered. Using the SSPA approach with pixel level replacements, we recorded a Dice score of 0.9 and IoU of 0.818 on the EM dataset with only 0.86% expert annotation. We recorded a Dice score of 0.973 and an IoU of 0.824 on the melanoma dataset with just 0.8% expert annotation. We also obtained a Dice score of 0.815 and an IoU of 0.691 with 0.85% expert annotation using only 50% of the original biofilm dataset from [32]. The SSPA approach was effective in determining the boundaries between cross sections or bacterial cells in the EM and biofilm datasets with scant annotations. The approach was also effective in determining the entire lesion of interest in the melanoma datasets with scant annotations. The SSPA approach is a general purpose segmentation approach, and our results suggest that the SSPA approach can be effective for both boundary and semantic segmentation tasks across different applications. The SSPA approach is parameterized based on the uncertainty range of pixels that are targets of directed expert annotations and the number of images that are selected in each iteration, which can be empirically determined for the application at hand. In addition, the SSPA network architecture is modular and can be easily adapted to incorporate a variety of psuedo-segmentation mask generation algorithms and for selecting images in each iteration of active learning. The manual effort involved in directed pixel-level annotations can be further alleviated by the design of a software workbench. For this purpose, in the future we plan to design and implement a workbench to improve the pixel replacement task by the expert and provide a convenient interface as well.  
Conceptualization, A.D.C., M.S. and P.C.; Methodology, A.D.C., M.S. and P.C.; Funding acquisition, P.C. and V.G.; Formal analysis, M.S.; Investigation, A.D.C. and D.A.; Project administration and supervision, M.S. and P.C.; Software, A.D.C.; Writing--original draft, A.D.C. and M.S.; Writing--review and editing, M.S., P.C. and D.A. All authors have read and agreed to the published version of the manuscript.  
[PERSON], [PERSON], [PERSON] are partially supported by NSF EPSCoR RII Track 2 FEC #1920954. [PERSON] acknowledges funding support from NSF RII Track-1 award (1849206) and NSF CAREER Award (1454102).  
Not applicable.