# Semantic Image Segmentation Using Scant Pixel Annotations
## 3 The SSPA Algorithm
```
1:\(U=\{u_{1},\ldots,u_{N}\}\)\(GT=\{g_{1},\ldots,g_{M}\},M\leq N\)\(\triangleright\) Optional
2:procedure SSPA (\(T_{i}\), \(M_{i}\), \(L_{i}\), \(\overline{C_{i}}\))
3: Choose \(J\) images with highest entropy (HE)/lowest entropy (LE) from \(T_{i}\)
4:for each\(t_{p}\in J\)do
5: Let \(e_{p}\in\overline{C_{i}}(t_{p})\)
6:\(Target=e_{p}\in(0.5\pm\delta)\)
7:\(Target\gets ExpertAnnotation\)
8: Let each \(p_{xy}\) be a pixel outside \(Target\) in \(t_{p}\)
9:if\(\mu_{i}>\mu_{i-1}\)then
10:\(L_{i+1}(p_{xy})\gets L_{i-1}(p_{xy})\)
11:else
12:\(threshold=\mu(\overline{C_{i}(t_{p})})\)
13:\((\overline{C_{i}(p_{xy})}>threshold)\gets 1\)
14:\((\overline{C_{i}(p_{xy})}<threshold)\gets 0\)
15:endif
16:endfor
17:\(L_{i+1}=\text{Replace }J\) in \(L_{i}\)
18:return\(M_{i+1},\overline{C_{i+1}}\)
19:endprocedure
20:procedureWatershed(\(U\))
21:\(WS=\{L_{ws_{1}},L_{ws_{2}},L_{ws_{3}}\}\) ensemble
22:\(L_{1}\gets MajorityVoting(L_{ws_{1}},L_{ws_{2}},L_{ws_{3}})\)
23:return\(M_{1},\overline{C_{1}}\)
24:endprocedure
25:repeat
26: SSPA (\(T_{i},M_{i},L_{i},\overline{C_{i}}\))
27:until\(\mu_{i+1}>\mu_{i}\)
```  
**Algorithm 1** Segmentation with scant pixel annotations. **Segmentation with Scant Pixel Annotations:** Let \(i\geq 1\). The pseudo label set \(L_{i}\), the corresponding model \(M_{i}\) and the normalized prediction confidence values \(\overline{C_{i}}\) generate the pseudo binary label set \(L_{i+1}\) from \(M_{i}\) as follows. First, choose \(J\) images from \(T_{i}\) with highest entropy (HE) or the lowest entropy (LE) values. Let \(t_{p}\) be one such image chosen and \(L_{i}(t_{p})\) be its label in \(L_{i}\). We construct the training label \(L_{i+1}\) for \(t_{p}\) as follows. Consider all pixels in \(t_{p}\) with prediction confidence values between \((0.5\pm\delta)\) in \(e_{p}\in\overline{C_{i}}\) (pixels whose predictions from model \(M_{i}\) are in the uncertainty range), for expert annotation (lines 8-9). The value of the parameter \(\delta\) is assigned empirically for each dataset. Let \(Target\) be the set of all pixels in \(t_{p}\) that are marked for expert annotation. One way to obtain expert annotation for pixel labels is to manually label each pixel in \(Target\). If \(GT\) is available, we copy the pixel label for each pixel from the ground-truth label of \(t_{p}\) into \(L_{i+1}(t_{p})\). Now consider the pixels that are not in \(Target\). The pixel-level labels for these pixels in \(t_{p}\) can be decided using either the previous model \(M_{i-1}\) or the current model \(M_{i}\) (lines 11-17). Let \(p_{xy}\) be a pixel in \(x^{th}\) row and \(y^{th}\) column of \(t_{p}\). If \(\mu_{i}>\mu_{i-1}\), then the label for \(p_{xy}\) in \(L_{i+1}(t_{p})\) is the same as that in \(L_{i-1}(t_{p})\). Else, label for \(p_{xy}\) in \(L_{i+1}(t_{p})\) is the same as assigning a class 0 or 1 to \(\overline{C_{i}(t_{p})}\) based on a _threshold_. _Threshold_ is calculated as the mean prediction confidence value of \(t_{p}\). Generate the next set of training labels, \(L_{i+1}\) by replacing \(J\) labels in \(L_{i}\) (line 16). Train a segmentation network on pair \(<U,L_{i+1}>\) to obtain next model \(M_{i+1}\). **Termination condition:** At each iteration \(i\), record the mean entropy of \(M_{i}\), \(\mu_{i}\). The algorithm terminates when the mean entropy of \(M_{i+1}\), \(\mu_{i+1}\) is higher than the mean entropy of \(M_{i}\) (lines 29-31). The decrease in model performance indicates that the model is unlearning useful patterns or features during training at the \((i+1)\)th iteration. In the presence of \(GT\) labels, we also record evaluation metrics such as intersection over union (IoU) and Dice score. However, mean entropy as an evaluation metric takes precedence over IoU and Dice score, even in the presence of \(GT\) labels. Refer to Section 4.4 for a detailed discussion of evaluation metrics. Select \(M_{i}\) with the best evaluation metrics as the best model to obtain binary labels, \(L_{i}\) using the least expert intervention.