# Semantic Image Segmentation Using Scant Pixel Annotations
## 2 Related Work
### Proposed SSPA Approach  
The _SSPA_ approach seamlessly integrates active learning and semi-supervised learning approaches with pseudo-labels to produce high-performing segmentation models with cost-effective expert annotations. Similar to the semi-supervised approach in [29], the SSPA does not require any expert annotation to produce the base model. It uses an image processing algorithm based on the watershed transform [31] to generate pseudo-labels. The base model generated using these pseudo-labels is then successively refined using active learning. However, unlike the prior active learning approaches used for segmentation, we employ image entropy instead of image similarity to select top-k high entropy or low entropy images for expert annotation. Further, unlike most of the earlier active learning approaches for segmentation (with the exception of [28]), our unit of annotation is a pixel, targeting uncertain pixels only while other pixels are labeled based on the behavior learned by the models (please see Section 3 for more details). Our preliminary work reported as a short paper in [32], explored the viability of using pseudo-labels in place of expert annotations for semantic segmentation. In that paper, we considered datasets where expert annotations are available for the entire dataset and built a benchmark segmentation model using fully supervised learning. We then compared models built using a mixture of pseudo-labels and expert annotated labels with the benchmark model to show that the viability of pseudo-labels for building segmentation models. Requiring the experts to annotate all of the training data a priori and building a fully supervised segmentation model makes our prior work very different from the proposed approach. Further, having the experts deeply annotate each pixel in each image in all of the training data makes our prior approach impractical for several domains, where significant expertise is needed to annotate each image. In contrast, in the _SSPA_ approach, expert annotations are obtained on demand only for the training samples identified in each active learning step. Further, the unit of annotation is a pixel, and the process is terminated when the model performance plateaus or no further refinements are possibly similar to [29]. The _SSPA_ approach outperforms state-of-the-art results in multiple datasets including those used in [32]. The SSPA uses the watershed algorithm to generate pseudo-segmentation masks. This algorithm [33; 34; 35; 31] treats an image as a topographic surface with its pixel intensities capturing the height of the surface at each point in the image. The image is partitioned into basins and watershed lines by flooding the surface from minima. The watershed lines are drawn to prevent the merging of water from different sources. The variant of watershed algorithm used in this paper, the marker-controlled watershed algorithm (MC-WS) [36], automatically determines the regional minima and achieves better performance than the regular one. MC-WS uses morphological operations [37] and distance transforms [38] of binarized images to identify object markers that are used as regional minima. In [PERSON] et al. [39], the authors proposed a ConvNets-based strategy to perform segmentation on medical images. They attempted to reduce the annotation effort by using a partial set of noisy labels such as scribbles, bounding boxes, etc. Their approach extracts and eliminates ambiguous pixel labels to avoid the error propagation due to these incorrect and noisy labels. Their architecture consists of two stages. In the first stage, ambiguity maps are produced by using \(K\) FCNs that perform binary classification for each of the \(K\) classes. Each classifier is given the input of pixels only true positive and true negative to the given class and the rest are ignored. In the second stage, the model trained at the first stage is used to predict labels for missing classes, using a curriculum strategy [40].