SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In Fig. 11, we evaluate the accuracy of our final model and dominant class labeling with different budget scenarios: 10K clicks over 5 rounds, 50K clicks over 10 rounds, and 100K clicks over 5 rounds. This analysis is performed on Cityscapes, using a ResNet50 backbone combined with PixBal sampling. As depicted in Fig. 11(a), even in the tiny budget of 10K clicks, the multi-class labeling outperforms the dominant class labeling baseline. This highlights the utility of our method in extremely constrained budget scenarios, where the need for active learning is emphasized. As shown in Fig. 11(b) and (c), when the total budget is kept constant, increasing the frequency of active sampling and model training enhances performance. This improvement can be attributed to the more frequent interactions between the model and the Oracle, leading to more informative active sampling. Multi-class labeling consistently outperforms dominant-class labeling in a wide range of budget sizes, demonstrating the efficacy of our method.
C.2 Comparison with partial label learning loss baselines
In Fig. 12(a), we compare our proposed losses with baseline partial label learning losses: Infimum loss [8], and RC loss [22, 46]. This comparison is performed on Cityscapes, using a ResNet50 backbone combined with PixBal sampling. As shown in Fig. 12(a), the model with proposed losses
Figure 12: Accuracy in mIoU (\%) versus the number of clicks (budget) evaluated on Cityscapes using ResNet50 backbone, with PixBal sampling. (a) The accuracy of our stage 1 model trained with proposed losses  $(\mathcal{L}_{\mathrm{MP}} + \mathcal{L}_{\mathrm{PP}})$ , compared with baseline partial label learning loss: Infimum loss [8], and RC loss [22,46]. (b) The accuracy of stage 1 multi-class labeling model (stage 1), stage 2 multi-class labeling model solely employing intra-region label localization (w/ proto), and a baseline pseudo labeling method without prototype (w/o proto). (c) The accuracy of stage 2 multi-class labeling model with (Local+Expand) and without (Local) label expansion, compared with dominant class labeling model with (Dom+Expand) and without (Dom) label expansion.
Figure 13: Accuracy in mIoU (\%) versus the number of clicks (budget) for multi-class labeling evaluated on Cityscapes equipped with six different acquisition function: Random, KNN, Margin, BvSB, ClassBal, and our sampling strategy (PixBal). The reported accuracy scores are averaged across three trials.
$(\mathcal{L}_{\mathrm{MP}} + \mathcal{L}_{\mathrm{PP}})$  demonstrates superior performance over the models using the baseline losses, particularly in the early rounds.
C.3 Comparison with pseudo labeling baselines
In Fig. 12(b), we conduct an ablation study comparing our proposed intra-region label localization method (denoted as 'w/ proto') with a baseline intra-region pseudo labeling method that assigns the most confident class among multi-class labels as the pixel-wise pseudo label (denoted as 'w/o proto'). This comparison takes place on the Cityscapes dataset, utilizing a ResNet50 backbone in conjunction with PixBal sampling. While both of the intra-region pseudo-labeling methods improve upon the stage 1 model, the proposed prototype-based label localization demonstrates superior performance over the baseline, specifically in the initial round, where the stage 1 model may lack accuracy.
In Fig. 12(c), we compare the performance improvement brought by label expansion when applied to both the multi-class labeling model (denoted as 'Local+Expand') and the dominant class labeling model (denoted as 'Dom+Expand'). This comparison is also conducted on the Cityscapes dataset, using a ResNet50 backbone paired with PixBal sampling. As shown in Fig. 12(c), label expansion proves to be more beneficial when used with multi-class labeling, as it allows the spread of pseudo labels across multiple classes, resulting in a broader expansion of pseudo labels.
C.4 Additional comparison with baseline acquisition functions
In Fig. 13, we evaluate multi-class labeling combined with 6 different acquisition functions: Random, K-Nearest Neighbors (KNN), Margin, Best-versus-Second-Best (BvSB), Class Balanced sampling (ClassBal), and our sampling strategy (PixBal). PixBal consistently outperforms all the others regardless of budget size. Margin shows decent performance with a large budget, e.g., it surpasses ClassBal when using 500K clicks. On the other hand, KNN offers modest improvements, marginally surpassing the performance of Random.
C.5 Qualitative results of our final model
Fig. 14 provides a qualitative result of the predictions of our final model at different rounds. As illustrated in Fig. 14, the quality of the predictions markedly improves as the rounds progress. Notably, the predictions produced by our final model at round 5 exhibit impressive quality, especially when taking into account that it requires only  $9.8\%$  of the labeling cost associated with a fully supervised model.
Figure 14: Qualitative results of our final model on the Cityscapes dataset. (a) Inputs. (b) Prediction in round 1. (c) Prediction in round 3. (d) Prediction in round 5. (e) Ground Truth.