Efficiently Troubleshooting Image Segmentation Models with Human-In-The-Loop
Authors: Haotao Wang
Tianlong Chen
Zhangyang Wang
Kede Ma
Venue:  (2021)

Abstract: Image segmentation lays the foundation for many high-stakes vision applications such as autonomous driving and medical image analysis. It is, therefore, of great importance to not only improve the accuracy of segmentation models on well-established benchmarks, but also enhance their robustness in the real world so as to avoid sparse but fatal failures. In this paper, instead of chasing state-of-the-art performance on existing benchmarks, we turn our attention to a new challenging problem: how to efficiently expose failures of ``top-performing'' segmentation models in the real world and how to leverage such counterexamples to rectify the models. To achieve this with minimal human labelling effort, we first automatically sample a small set of images that are likely to falsify the target model from a large corpus of web images via the maximum discrepancy competition principle. We then propose a weakly labelling strategy to further reduce the number of false positives, before time-consuming pixel-level labelling by humans. Finally, we fine-tune the model to harness the identified failures, and repeat the whole process, resulting in an efficient and progressive framework for troubleshooting segmentation models. We demonstrate the feasibility of our framework using the semantic segmentation task in PASCAL VOC, and find that the fine-tuned model exhibits significantly improved generalization when applied to real-world images with greater content diversity. All experimental codes will be publicly released upon acceptance.

Under review as a conference paper at ICLR 2021
EFFICIENTLY TROUBLESHOOTING IMAGE SEGMENTA- TION MODELS WITH HUMAN-IN-THE-LOOP
Anonymous authors Paper under double-blind review
ABSTRACT
Image segmentation lays the foundation for many high-stakes vision applications such as autonomous driving and medical image analysis. It is, therefore, of great importance to not only improve the accuracy of segmentation models on well- established benchmarks, but also enhance their robustness in the real world so as to avoid sparse but fatal failures. In this paper, instead of chasing state-of-the-ar performance on existing benchmarks, we turn our attention to a new challenging problem: how to efficiently expose failures of perrprrrfomn segmentation models in the real world and how to leverage such counterexamples to rectify the models. To achieve this with minimal human labelling effort, we first automatically sample small set of images that are likely to falsify the target model from a large corpus of web images via the maximum discrepancy competition principle. We then propose weakly labelling strategy to further reduce the number of false positives, before consumonn pixel-level labelling by humans. Finally, we fine-tune the model to harness the identified failures, and repeat the whole process, resulting in an efficient and progressive framework for troubleshooting segmentation models. We demonstrate the feasibility of our framework using the semantic segmentation task PASCAL VOC, and find that the fine-tuned model exhibits significantly improved generalization when applied to real-world images with greater content diversity. All experimental codes will be publicly released upon acceptance.
INTRODUCTION
Image segmentation (i.e., pixel-level image labelling) has recently risen to explosive popularity. due part to its profound impact on many high-stakes vision applications, such as autonomous driving and medical image analysis. While the performance of segmentation models, as measured by excessively reused test sets (Everingham et al. [2000) Lin et al. 2014), keeps improving (Chen et al.) 2018a Badrinarayanan al. 2017: Yu et al. 2018), two scientific questions have arisen to capture the community's curiosity, and motivate the current work:
Q1: Do trop-performing" segmentation models on existing benchmarks generalize to the real world with much richer variations?
Q2: Can we identify and rectify the trained models' sparse but fatal mistakes, without incurring significant workload of human labelling?
The answer the first question is conceptually clearer, by taking reference a series of recent work on image classification (Recht et al.) 2019 Hendrrykks et al., 2019). A typical test set for image classification can only include a maximum of ten thousands of images because human labelling (or verification of predicted labels) is expensive and time-consuming. Considering the high dimensionality of image space and the "human-level" performance of existing methods, such test sets may only spot extremely small subset of possible mistakes that the model will make, suggesting their insufficiency to cover hard examples that may be encountered the real world (Wang et al.) 22220). The existence of natural adversarial examples (Hendryeks et al. 2019) also echos such hidden fragility of the classifiers to unseen examples, despite the impressive accuracy on existing benchmarks.
While the above problem has not been studied in the context of image segmentation, we argue that it would only be much amplisied for two main reasons. First, segmentation benchmarks require

---

Under review as a conference paper at ICLR 2021
pixel-level dense annotation. Compared classification databases, they are much more expensive. laborious, and error- proorr to labe making existing segmentation datasets even more restricted in scale. Second, it is much harder for segmentation data to be class- balanced in the pixel level. making highly skewed class distributions notoriously common for this particular task (Kervadec al.] 20T9 [iicchke et al., 20188) Besides, the "universal" background class (often set cover the distracting or uninteresting classes (Everingham et al., 2010)) adds additional complicacy to image segmentation (Mostajabi et al. 2013). Thus, it remains questionable what extent the impressive performance on existing benchmarks can be interpreted as (or translated into) real-world robustness. If "top-pprffrmmig segmentation models make sparse yet catastrophic mistakes that have not been spotted beforehand, they will fall short of the need by high-stakes applications.
The answer to the second question constitutes the main body of our technical work. In order to identify sparse failures of existing segmentation models, it is necessary to expose them to much larger corpus of real- weal-world labelled images (on the order millions or even billions). This is, however, implausible due the expensiveness of dense labelling in image segmentation. The core question essentially boils down to: how to efficiently decide what to label from the massive unlabelled images, such that a small number of annotated images maximally expose corner-case defects, and can be leveraged to improve the models.
In this paper, introduce a two-stage framework with human-in-the-lonn for effi- ciently troubleshooting im- age segmentation models (see Figure T). The first stage automatically mines, from large pool D of un- labelled images, small im- age set M, which are the most informative in expos- ing weaknesses of the tar- get model. Specifically, in- spired by previous studies on model falsification as model comparison (Wang & Simoncelli 2008 Ma et al. state-of the-ar art methods with different design methodologies, and sample images by MAximizing the Discrepancy (MAD) between the methods. To reduce the number of false positives, we propose weakly labelling method of filtering M to obtain a smaller reñned set S, subject to segmentation by human subjects. In the second stage, we fine-tune the target model to learn from the counterexamples in S without forgetting previously seen data. The two stages may be iterated, enabling progressive troubleshooting of image segmentation models. Experiments 2010 demonstrate the feasibility the proposed method to address this new challenging problem, where successfully discover corner-case errors of perp-efrfoming" segmentation model (Chen ct al. 2017), and fix it for improved generalization in the wild.
Previous training sets
Competing model zoo
Fine- tune
Predict
Target model
Select
Labelled set s
Oradle leg.. human annotators)
Unlabelled set AV
Unlabel
image pool
Figure 1: Proposed framework for troubleshooting segmentation mod- els.
2018 Wang et al. 2020), we let the target model compete with a set of
PASCAL VOC (Everingham et al.)
RELATED WORK
The proposed method takes inspiration from the MAD competition (Wang & Simoncelli 2008 Wang et al. (020) to efficiently spot model failures. Previous works focused on performance evaluation. We take one step further to also fix the model errors detected in the MAD competition. To the best of our knowledge, our work is the first to extend the MAD idea to image segmentation, where labeling efficiency is more desired since pixel-wise human annotation for image segmentation is much more time-consuming than image quality assessment (Wang & Simoncelli 2008) and image classification (Wang et al. 2020) tasks previously explored.
MAD competition
Accordiigg to Everingham et al. (Everingham et al. 2010) and our practice, it can casily take ten times as long segment an object than draw bounding box around it.
2

---

Under review as a conference paper at ICLR 2021
Differential testing Our method is also loosely related to the cross cross-disciplinary field of soft- ware/system testing, especially the differential testing technique (McKeeman) 1998). By providing the same tests to multiple software implementations. differential testing aims to find bug-exposing test samples that lead to different results. Programmers can then dig into these test cases for potential bug fixing. While debugging software is very different from troubleshooting machine learning algorithms, a handful of recent work explored this idea to find pitfalls of deep learning-based autonomous driving systems (Pei et al. 2017 Tian al., 2018, Zhang et al. 2018). Aiming be fully automated without human intervention, these methods have to make strong assumptions such as the ground truth labels can be determined by majority vote or are unchanged under some synthetic image transformations (e.g., brightness and contrast change, or style transfer). Therefore, it is unclear how to generalize the results obtained such noisy and often unrealistic settings, to the real world with both great content fidelity and diversity.
Ad Adversarial examples Introduced by Dalvi et al. (Dalvi et al., 2004) and reignited by Szegedy et al. (Szegedy et al. (2013), most adversarial attacks add small synthetic perturbations to inputs of computational models that cause them to make incorrect decisions. In image classification, Hendrycks et al. (Hendrycks et al., 2019) identified a set of natural images that behave like synthetic adversarial examples, which possess inherent transferability to falsify different image classifiers with the same type of errors. The selected counterexamples by the proposed framework might be treated as new type of natural adversarial examples, that force the two models to make distinct predictions, therefore capable of fooling at least one model.
Similar as natural counterexamples focused in this work, synthetic adversarial examples pose se- curity risks of deploying machine learning algorithms in real-world applications. A large body of research (Madry et al.) 2018 Zhang et al., (219)) delves deep into adversarial training, trying to defend against adversarial perturbations at the expensive cost of sacrificing the generalization on original
test sets without perturbations (Tippaa et al.) 2019 Zhang et al., 2019 Schmidt et al. 2018). This seems to suggest trade-off between generalization to real-world benign examples and robustness to adversarial attacks.
Semantic segmentation with deep learning Fully convolutional network (Long al. 2015 was
among the first deep architectures adopted for high-quality segmentation. Skip connection
(Ron
neberger et al. (2155). recurrent module (Zheng et al. 20155, max index pooling (Noh al. 2015
Badrinarayanan et al. 2017), dilated convolution (Chen et al., 2014, Yu & Koltun| 2016 Chen
jtt 20. and scale training (Chen et al.] 2016] 8a) are typical strategies to boost the
segmentation performance. Conditional random fields (Ladicky et al.) (2010) used to dominate image
segmentation before the advent deep learning were also combined with convolutional networks to model spatial relationships (Zheng et al. 2015). We refer interested readers to (Minaee et al. 2020 for a comprehensive survey of this field.
PROPOSED METHOD
Suppose we have a target segmentation model ft Rhhxw + (1..... "fl....,.]J/xw cchxx, where and w are the height and width of the input image, and c denotes the number of categories. Our goal is to efficiently identify and fix the failure cases of fi encountered in the real world, while minimizing human labelling effort in this process. We start by constructing large image database D. whose collection may be guided by the keywords that represent the categories. Rather than conducting arge- subjective testing to obtain the ground truth segmentation map for each x E D, we choose to create small subset of images M C D, which are strong candidates for revealing corner-case behaviors of ft. To further reduce false positive examples in M, we describe a method to gather weak label for each E M as an overall indicator of segmentation quality. Based on the labels, an even smaller set S C M can be obtained for dense annotation by humans. Last, we fine-tune fe
2 As per (Hendrycks et al. 2019). machine learning algorithms with similar design philosophies tend make common mistakes.
'n many areas of computer vision, methods trained on synthetic data cannot generalize to realistic data, and specialized techniques such as domain adaptation (Zhang et al. 2017 Zhao et al. 2019) have to be used to bridge the performance gap.
3

---

Under review as a conference paper at ICLR 2021
on the combination of S and previously trained data, in attempt to learn from the found failures without forgetting (Li & Hoiem 2017).
3.1 FAILURE IDENTIFICATION
Constructing M Inspired by model falsification methodologies from computational vision (Wagg
& Simoncelli 2008 and software engineering [McKeeman] 1998), we construct the set MM- = { by sampling the most "controversial" images from the large-scale unlabelled database D {a,
where 122 & Specifically, given the target model ft, we let it compete with group of of
the-art segmentation models {g by maximizing the discrepancy (Wang al. 2020 between ft and 9j on D)
a arg max d(f((z). j= RED
(1)
where d(.) a distance metric to gauge the dissimilarity between two segmentation maps (e.g.. negative pixel accuracy, or mean region intersection over union (mloU)). زا4i represents the most controversial image according to and gj. and therefore is the most informative in distinguishing between them. If the competing model Jj performs at high level, and differs from the target model ft in design, .g()) is likely to be failure of ft-
To avoid identifying different instan- tiations of the same underlying root cause (Pei et al., 2017) and to en- courage content diversity of the candi- date images, we describe a "content- aware" method for constructing M. We first partition D into e overlapped subgroups (Di,* based ft's predicted maps, where x E De, least one pixel in (x belongs to the k-th category. After that, we add a content constraint by restricting the size of predicted pixels in the k-th cat- egory, i.... Σ (f)(
U.S
k)/(h x w), within the range of [Pk: ga). This allows excluding images of exceed- ingly large (or small) object sizes, which may be of less practical rele- vance. Moreover, instead of focusing on the most controversial example de- fined in Eq. ①.. we look top-n1 images in Dr,e with ng largest distances computed by
bad
bad
good
good
exoellent:
Figure 2: Purpose of weakly labelling. First row: Candi- date images from M. Counterexamples selected into are highlighted with red rectangles, and the rest false posi- tives. Second and third rows: Predictions by the target and competing models, respectively. See Figure 4 for the color legend.
(a ns
Σ ((ff(E)) L..,,,,, m, I....,c.
arg arg max
(x:) CD,
We then repeat this procedure, but with the roles of ft and gj reversed. That is, we partition D into c subgroups (D,. ) according 9j. and solve the maximization problem over Dj,k- Finally, gather all candidate images to arrive at the set M {x.),, where n2 2mcn3 n
Constructing S Although images in the candidate set M have great potentials being counterex-
amples of fz, some false positives may be included (see Figure 2). especially when gj is inferior fr- In addition, no data screening is involved in the construction of DD increasing chances of including images that are out-of-domain (c.g., falling out of the categories and/or containing inappropriate content). In view of these. we reduce false positives in M via weakly labelling strategy. For each E M, we ask human subjects give discrete score on an absolute category rating (ACR) scale to indicate the segmentation quality of (z) The labels on the scale are "bad", "poor", "fair", "good",
"We have n2 2meng because the images may he optimal in different prohlems specified in Eq. 2
4

---

Under review as a conference paper at ICLR 2021
and "excellent" Fsgure 8 We then rank all images in M by the mean opinion scores, and choose top-n images with the smallest scores to form the counterexample set = {z; ] Finally, we seek pixel-level segmentation results from human annotators for each image in S (see Figure 9b.
3.2 MODEL RECTIFICATION
The proposed framework for ef- ficiently troubleshooting segmentation models Input: An unlabelled image set D, a target
Algorithm 1:
The labelled images in S give us great op- portunity to improve by learning from these failures. In order to be resistant to catastrophic forgetting (McCloskey & Cohen 1989). the fine-tuning can also include labelled data pre- viously used to train ft- We may iterate through the whole procedure of failure identi- fication and model rectification several rounds, leading to progressive two-stage framework for efficiently troubleshooting segmentation for j + to m do models with human comman-in-tiee in-the- tho-loop.
model f( and the dataset ( which it pre-trained, a group of competing models { the maximum number of fine-tuning rounds, hyper-parameters fiz, 71
Output: Improved f(
S < 0
Compute segmentation predictions (9)(2), a D}
When the iterative setting is enabled, the size of S is growing: = Ur_, S(1), where (()) is the counterexample set created in the i-th for i + 0 to
end
do
M(+11 < 0
fine-tuning round. Denoting the initial tar- get model by f!") we fine-tune f! p((-1) on the combination of accumulated in the previous
Compute segmentation predictions (1. (J!0 (x),
i rounds and ((0) used for pre-training
for j + 1 to m do
Compute the distances
We summarize the proposed framework in Al- gorithm
{d(f(" (z), x D}
10
Divide D into e subgroups according f(")
EXPERIMENTS
Filter images by the content constraint
Select top-ng images by solving
In this section, tion task defined in PASCAL. VOC (Evering- ham et al. 2010 as specific application demonstrate the feasibility of our method. It worth noting that the proposed framework can be applied to other segmentation tasks, such 15 as those required in autonomous driving (Ess) eet al., 22099] Cordts et al.] (0166) and medical image analysis (Ronneberger et al. 2015).
the semantic segmenta-
Eq. (2) to form M,
M (++T)
1(++1 UMM
Reverse the roles of p(() and gj. and repeat Steps 10 to 13
end
Source weak human scores for ((+1)) Select top-n4 images with the lowest quality scores and collect level
17
labels from humans to form ((++)
4.1 EXPERIMENTAL SETUPS
S US(++)
18
19 Fine-tune f((() on the combination of S and S(0)
Segmentation models In our experiments, we choose the target model fr to be the state-of-the-art DeepLab V3Plus (Chen et al.) 20186) with DRN (Yu et al. 2017) as the backbone (termed Deeplabb3PP-DRNY We include five competing models: abe with ResNet101 (He et al., (016)) (termed Deeplab DeepLabV3P-RN RN101), DeeplabVV3 (Chen et al. (017)) with ResNet101 (termed DeepLab V3-NNI0I,, DFN (Yu et al.) 2018). Light-Weight RefineNet (Nekrasov ct al. 2018)) with ResNet50 and MobileNetV2 (Sandler et al. 2018) (termed LRefinneNe-RN50 and LRelineNet-MNV2. respectively). Publicly available pre-trained weights on PASCAL VOC 2012 (Everingham et al. 2010) are used for all models. Following (Chen et al. 2017 2018b), images are cropped and resized 513 x 513 before inference.
20 end
Constructing D We first briefly introduce the semantic segmentation database in PASCAL VOC (Everingham et al., 2010). It contains 1, 464 images for training (denoted by ((W) and 1, 149 for validation (denoted by ()) with 20 scene categories (e.g., aeroplane, bicycle and person). We use the 20 class labels and combinations of them as keywords crawl images from the Internet.
5

---

Under review as a conference paper at ICLR 2021
(a)
(b)
Figure 3: Visual comparison of images in (a) S and (b) PASCAL VOC validation set (denoted by (()).
Table I: Segmentation results in terms of mloU on both ( and the unbiased test sets {7(1) Jim
Fine-tuning Round
1
2
Test Set
T
T())
T
T
T
T
DEN
0.8037
0. 1319
0.8037
0.1054
0.8037
0.1365
DeepL.abV3-RNN
0.7795
0. 1589
0.7795
0.1255
0.7795
0.1791
Competing Models
DeepLabV3P-RNIIg
0.7843
0.2555
0.7843
0.1978
0.7843
0.1483
LRefineNNetRNS0
0.7710
0.1740
0.7710
0.1431
0.7710
0,2014
LRefineNettMNNV2
0.7125
0.1325
0.7125
0.1194
0.7125
0.1678
Target Model
DeepLabV3P-DRN
0.7887
0.1759
0.7827
0.2436
0.7828
0.4233
No other constraints are imposed during data collection. As a result, the database 'D includes a total of more than 40, 000 images, which is much larger than PASCAL VOC training and validation sets.
Constructing M We empirically set the two parameters pk and qu in the content constraint to the first and the third quartiles of the k-th category sizes of the training images in PASCAL VOC, respectively. To strike good balance between the human effort of weakly labelling and the content diversity of M, we choose n3 = 25 in Eq. (2). Due to the existence of duplicated images, the actual sizes of {M00} range from 1, 500 to 2, 000 for different rounds.
Constructing We collect weak labels for images in A from three volunteer graduate students, who have adequate knowledge of computer vision, and are told the detailed purpose of the study. Each subject is asked to give an integer score between one and five for each image to represent one of the five categories, with a higher value indicating better segmentation quality. All out-of-domain images are given a score of positive infinity, meaning that any subject is granted to eliminate an image without agreement from the other two. The mean opinion scores averaged across all subjects are used to rank images in M. The hyper papae--parmmeter n, in Algorithm iss set to 100. Representative examples in S are shown in Figure and Figure 6 along with images in (0)). It is clear that images in are visually much harder.
We invite the same three students to provide ground truth segmentation results for images in S, follow- ing the annotation guidance in PASCAL VOC, with the help of the online annotation tool Labelbox Our annotation process includes two stages with the goal of obtaining consistent segmentation maps. In the first stage, each subject is assigned one-third of the images provide pixel-level labels. In the second stage, we carry out cross validation to improve the annotation consistency. Each student takes to check the segmentation maps completed by others, marking the positions and types of possible annotation errors. During cross checking, the subjects can discuss with each other to reach an agreement on small part of ambiguous annotation.
Iterative model correction We perform total of fine-tuning rounds. As suggested in (Chen et al. 2017), fine-tuning for each round is carried out by stochastic gradient descent with momentum
hhups//lahellox.comm/

---

Under review as a conference paper at ICLR 2021
background
actoplane
bicycle
bus
chair
cow covndinin table
horse
person
sheep
train t/monnior
Test image
Predicted by gj T(1)
Predicted by f(
Test image
Predicted by 9j
Predicted by p(1)
Predicted
by f(0)
T(2)
Figure 4: Left panel: Representative images in the test set (()) before fine-tuning the target model
π( From the corresponding predicted segmentation maps, find that the competing models {gj successfully identify the failures of f: Right panel: Representative images in the test set T after the first round fine-tuning, where we see that f(()) achieves noticeable performance improvements by learning from its failures in ((1)
0.9 and weight decay 5 x 10-4, We grid search the initial learning rate from {1, 10) 10 10-5, and choose the one with the highest mloU on ( The learning rate decay is guided by the polynomial policy. We set the mini-batch size and the maximum epoch number to 2 and 80, respectively.
Model evaluation How reliably probe the generalization of computer vision models when deployed in the real world is by itself a challenging problem being actively investigated (Hendrycks) ct al. 2019 Arnab et al. 2018 Recht et al. 2019). Observing progress on T()) is not a wise choice because this set may only contain few catastrophic failures of the target model f(("). It may also be unfair to employ the counterexample set ((++) evaluate the relative progress f(()) against the set of competing models {g, 1 due to the inclusion of the weakly labelling and filtering procedure. Inspired by the maximum discrepancy competition methodology for image classification (Wang et al.) 2020), here we construct new unbiased test set (i+1) с (Mi+1) to compare f(( with {g/ }/= by adding the following constraints. First, similar as the construction of S, all out-of-domain images are filtered out. Second, to encourage class diversity, we retain maximum of four images that contain the same main foreground object (i.e., ( has at most four "car" images). Third, to keep evaluation fair for the competing models, images used fine-tune the target model f((() are not included in (++1), In our experiments, the size of (i+1) is set 30.
4.2
MAIN RESULTS
Quantitative results We use standard evaluation metric mloU quantify the semantic segmen- tation performance. All results are listed in Table [ where we find that, before the first round of fine-tuning, all models achieve competitive results on (0), implying cose-to-saturation performance on PASCAL VOC. However, when tested on T the performance of all models drops significantly,
indicating that many images in (()) able to falsify both the target model f(()) and the associated competing model 9₁- This also provides direct evidence that hard corner cases of existing segmen-
tation models could be exposed. It is also proof-of-concey that the selection procedure is working
7

---

Under review as a conference paper at ICLR 2021
intended. Moreover, the top-1 model on T()) does not necessarily perform the best T(), conforming to the results in (Wang et al. 2020
After the first round of fine-tuning, f(") achieves noticeable improvements on T whereas all competing models experience different degrees of performance drops. This suggests that the target model begins to introspect and learn from its counterexamples in S(1), After the second round of fine-tuning, the mloU of f(2) on ( is boosted by around 18%, surpassing all competing models by larger margin than the previous round. This shows that method successfully learns from and combines the best aspects of the competing models to fix its own defects, with approximately the same performance on T In our experiments, we only perform two rounds of fine-tuning due to limited computation and human resources, while expect further performance gains under the proposed framework if tuned for more rounds.
Qualitative results We show representative test images before and after the first round of fine- tuning in Figure Before fine-tuning, the competing models can effectively find defects of the target model with incorrect semantics and/or boundaries. After fine-tuning, the target model does clearly better job on the corner-case samples. We also visually compare f( and f(") on T(2), where we observe the improved robustness unseen corner-case images. Another interesting finding that our method can fix general class of model mistakes by learning from small set of valuable failures.
Figure shows such an example, where the target model only sees 10 "car" images in ((1), and is able to generalize to images with similar unusual viewpoints.
Figure 5: Representative "car" images from M First row: Test images. Second row: Predictions by f10)_ Third row: Predictions by f(") that is only exposed to small set of "car" images in S(I), The generalization
target model on "car" images with unusual viewpoints largely improved after the first round of fine-tuning. (All images shown in this figure are not in training set of f()))
5 CONCLUSION
We have formulated a new computer vision problem that aims to efficiently expose the failures of nop-performing segmentation models and learn from such failures for improved generalization in the real world. We proposed an efficient numan--n--th--uee models, and demonstrated its promise under realistic settings. We hope this work sheds light on new line of research that requires both machine vision and human interaction. In the future, we plan to explore the idea of leveraging natural failures for improving model robustness in broader vision problems, such as video understanding (Tran al. 2015), computer-aided diagnosis (Mansoor et al. 2015). and computational neuroscience (Golan et al. 2019). Moreover, while the current work remains to focus improving the model's standard generalization (with emphasis natural corner-case samples), our future study will investigate how this could be jointly optimized with adversarial robustness.
framework for troubleshooting segmentation
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Anurag Arnab, Ondrej Miksik, and Philip HS Torr. On the robustness of semantic segmentation models to adversarial attacks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 888-897, 2018.
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. SegNet: A deep convolutional encoder- decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 39(12):2481-2495 2017.
Benjamin Bischke, Patrick Helber, Damian Borth, and Andreas Dengel. Segmentation of imbalanced classes in satellite imagery using adaptive uncertainty weighted class loss. In IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 6191-6194, 2018.
Liang-Chieh Chen, George Papandreou, lasonas Kokkinos, Kevin Murphy, and Alan I. Yuille. Semantic image segmentation with deep convolutional nets and fully connected erfs. arXiv preprint arXiv:1412.7066: 2014.
Liang-Chich Chen, Yi Yang. Jiang Wang, Wei Xu, and Alan L Yuille. Attention to scale: Scale-aware semantic image segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3640-3649, 2016.
Liang-Chich Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv: 170.005777 2017.
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 40(4))834-848, 2018a.
Liang-Chick Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In European Confer- ence on Computer Vision (ECCV), pp. 801-818, 2018b.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The Cityscapes dataset for semantic urban scene understanding. In IEEE Conference on Computer Vision and Pattern Recognition ienittion(CVA (CVPR), pp. 3213-3223, 2016.
Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, and Deepak Verma. Adversarial classification. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp- 99-108, 2004.
Andreas Ess, Tobias Müller, Helmut Grabner, and Luc J Van Gool. Segmentation- based urban traffic scene understanding. In British Machine Vision Conference (BMVC), 2009.
Mark Everingham, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The Pascal visual object classes (VOC) challenge. International Journal of Computer Vision (IJCV), 88(2):00--338 2010.
Tal Golan, Prashant C Raju, and Nikolaus Kriegeskorte. Controversial stimuli: Pitting neural networks against each other as models of human recognition. arXiv preprint arXiv: 1.t::/991100,,,.,.1100000 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pottern Recognition (CVPR), pp- 770-778, 2016.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. arXiv preprint arXiv:1907.07174, 2019.
Ajay J Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classification. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2372-2379, 2009.

---

Under review as a conference paper at ICLR 2021
Hoel Kervadec, Jihene Bouchtiba, Christian Desrosiers, Eric Granger, Jose Dolz, and Ismail Ben Ayed. Boundary loss for highly unbalanced segmentation. In International Conference on Medical Imaging with Deep Learning (MIDL), pp. 285-296, 2019.
L'ubor Ladicky, Paul Sturgess, Karteek Alahari, Chris Russell, and Philip HS Torr. What, where and how many? Combining object detectors and CRFs. In European Conference Computer Viston (ECCV), pp. 424-437, 2010.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pallern Analysis and Machine Intelligence (TPAMI), 40(12))2935-2944 2017.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft COCO: Common objects in context. In European Conference Computer Vision (ECCV), pp. 740-755, 2014.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, 2015.
Kede Ma, Zhengfang Duanmu, Zhou Wang, Qingbo Wu, Wentao Liu, Hongwei Yong, Hongliang Li, and Lei Zhang. Group maximum differentiation competition: Model comparison with few samples. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 42(4):851 864, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Awais Mansoor, Ulas Bagci, Brent Foster, Ziyue Xu, Georgios Z Papadakis, Les R Folio, Jayaram K Udupa, and Daniel J Mollura. Segmentation and image analysis of abnormal lungs at CT: Current approaches, challenges, and future trends. RadioGraphics, 35(4):1056 100110056-177 1076, 2015.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, volume 24, pp. 165. Elsevier, 1989.
William M McKeeman. Differential testing for software. Digital Technical Journal, 10(1):100-107, 1998.
Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos. Image segmentation using deep learning: A survey, arXiv preprint arX5v:000.00556. 2020.
Mohammadreza Mostajabi, Payman Yadollahpour, and Gregory Shakhnarovich. Feedforward seman- tic segmentation with zoom-out features. In IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 3376-3385, 2015.
Vladimir Nekrasov, Chunhua Shen, and Ian Reid. Light-weight RefineNet for real-time semantic segmentation. In British Machine Vision Conference (BMVC), 2018.
Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmentation. In IEEE International Conference on Computer Vision (ICCV), pp. 1520-1528. 2015.
Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. DeepXplore: Automated whitebox testing of deep learning systems. In Symposium on Operating Systems Principles (SOSP), pp. 1-18, 2017.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet classifiers generalize to ImageNet? arXiv preprint arXiv: 108/1, 2019.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer- Assisted Intervention (MICCAI), pp. 231-241, 2015.
10

---

Under review as a conference paper at ICLR 2021
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chieh Chen. Mo- bileNetV2: Inverted residuals and linear bottlenecks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4510-4520, 2018.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data. In Advances in Neural Information Processing Systems (NeurIPS), pp. 5014-5026, 2018.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference Learning Representations (ICLR), 2013.
Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. DeepTest: Automated testing of deep-neural- network-driven autonomous cars. In International Conference Software Engineering (ICSE), pp. 303-314, 2018.
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri, Learning spatiotem- poral features with 3D convolutional networks. IEEE International Conference on Computer Vision (ICCV), pp. 4489 4497, 2015.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Tumerr, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference on Learning Representations (ICLR), 2019.
Haotao Wang, Tianlong Chen, Zhangyang Wang, and Kede Ma. am going MAD: Maximum discrepancy competition for comparing classifiers adaptively. In International Conference on Learning Representations (ICLR), 2020.
Zhou Wang and Eero Simoncelli. Maximum differentiation (MAD) competition: methodology for comparing computational models of perceptual quantities. Journal of Vision, ((12):8.1-813. 2008.
Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Learning a discriminative feature network for semantic segmentation. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1857-1866, 2018.
Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. In Interna- tional Conference on Learning Representations (ICLR), 2016.
Fisher Yu, Vladlen Koltun, and Thomas Funkhouser. Dilated residual networks. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 472-480, 2017.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICML), pp. 7472-7482, 2019.
Mengshi Zhang, Yugun Zhang, Lingming Zhang, Cong Liu, and Sarfraz Khurshid, DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems. In ACM/IEEE International Conference on Automated Software Engineering (ASE), pp. 132-142, 2018.
Yang Zhang, Philip David, and Boqing Gong. Curriculum domain adaptation for semantic seg- mentation of urban scenes. In JEEE International Conference on Computer Vision (ICCV), pp. 0020-2030, 2017.
Sicheng Zhao, Bo Li. Xiangyu Yue, Yang Gu, Pengfei Xu, Runbo Hu, Hua Chai, and Kurt Keutzer. Multi-source domain adaptation for semantic segmentation. In Advances in Neural Information Processing Systems (NeurIPS), pp. 7285-7298, 2019.
Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip HS Torr Conditional random fields as recurrent neural networks, In IEEE International Conference on Computer Vision (ICCV), pp. 1529-1537, 2015.
11

---

Under review as a conference paper at ICLR 2021
A COMPARISON WITH ENTROPY-BASEL METHOD
Entropy has been used as an uncertainty metric to select hard samples (Joshi et al. 2009). The selected hard samples can be further used training data to improve target model performance. In this section, we compare our method with the entropy-based method in (Joshi et al., 2009) to show the advantage of our method than traditional entropy-basee methods. Since our goal is improve model performance
open- world images instead of fixed benchmark datasets, we need to tell which method can achieve models with better performance on open-world images, which can be efficiently done by the MAD competition in (Wang al. 2020).
For both methods, we conduct fine-tuning for one round and all experimental settings (including target model structure, human la- belling budget, etc.) are kept identical. The only difference comes from the sampling strategy: method samples according to Al- gorithm [] while satropy-based method selects images with largest entropy. We then conduct MAD competition Wang al. (2020) on the two models obtained from each method to compare their performance on open-world images. Numerical results are shown in Table 2] As we can see, the model achieved by our method is winning the MAD competition with noticeable mIoU advantage, showing that the model achieved by our method has better performance on spoldworld images than the model fine-tuned by entropy-based method.
Table 2: MAD competition re- sults two models achieved by our method and entropy- base method respectively.
Method
mloU
Entropy-based 19.65
Ours 26.53
B MORE VISUALIZATION RESULTS
(a)
(b)
Figure 6: Visual comparison of images in (a) S and (b) PASCAL VOC validation set (denoted by (())..
C ABLATION STUDY
In order to show the testing results hold as we increase the size of (()) (denoted as k)), we first evaluate the performance of the six models with different number of testing images, and conduct global ranking among the six models. We use top-k ranking to denote the global ranking evaluated with k testing images. Then we calculate the SRCC values between the top-40 ranking (as reference) and other top-k rankings with & {20, 21, ... 393. As shown in Figure 7 the ranking results remain stable when > 25. And our active finetuned target model always achieves the best performance on (1) among the six models, for any & between 20 and 40.
D SUBJECTIVE TESTING GUIS
Graphical user interfaces (GUIs) for our weakly and pixel-wise labelling experiments are shown in Figure g and Figure 9 respectively. For weakly labelling experiments, build our own GUT: an
12

---

Under review as a conference paper at ICLR 2021
1.00
0.95
SRCC
0.85
0.80
0.75
0.70
20
25
30
35
40
K
Figure 7: The SRCC values between the top-40 ranking and other top-k rankings.
image is rendered alongside the prediction made by the target model; scale-and-slider applet is utilized to collect the absolute category rating score of that image as described in Section For pixel-wise segmentation labelling experiments, LabelBox segmentation template to build our GUI.
13

---

Under review as a conference paper at ICLR 2021
background
aecollane
boat
battle
bux
chai
dningtable
dog
honue
matorbrie
pottedplant
sheep
Iemonitor
vord
good
Completed
Next
Figure 8: GUI for our weakly labelling experiments.
TOOLS
Search procs
reroptene
Med
OBJECTS
booungrundd (1)
Figure 9: LabelBox GUT for our pixel-level segmentation labelling experiments.
14


Reviews:
[{"review_id": "znqqZGfHJgB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nIn this work, the authors seek to leverage external sources of data to improve the generalization of segmentation models. In particular, they seek to identify images which generate discordance among models, hypothesizing that they would be well-suited to improve model performance. Once selected, they leverage human annotators to first filter this image set and then segment the images, which are then used to retrain the model. They demonstrate improved performance relative to a batch of competing models which are not updated using this procedure.\n\n------\n\nRecommendation:\nGiven the lack of a competitive baseline, opaqueness around the impact of the algorithm's hyperparameters, and what's likely to be noisy estimates of improvement, I cannot recommend this paper for acceptance as is. See below for greater detail.\n\n------\n\nPositives:\n* The authors recognize that not all images are created equal when looking to improve a model and attempt to tackle this challenging problem. This is particularly relevant in high-stakes environments when failure in rare cases can have a disproportionate impact (e.g., autonomous vehicles, healthcare, etc.).\n\n* The authors identify that scaling human annotation, in particular for image segmentation, can be cost prohibitive and propose a method to optimize this process. If successful, such a method could have significant implications for industries where the cost of annotation is high (e.g., healthcare where highly paid experts are required).\n\n---\n\nConcerns:\n* The authors propose a fairly complex (and costly) pipeline for improving generalization to the unseen dataset. However, they fail to compare against even a simple baseline such as random selection of images from this dataset. Comparing against models which are not updated, in particular when it's clear that none of them generalize, is a weak baseline that could likely be outperformed by far simpler uses of the external data. \n\n* T^(i+1) = 30 is quite small, especially given the number of classes. While it is understandable that such a sample cannot be extraordinarily large, by leveraging such a small value there's significant noise in the evaluation criteria. Given the previous concern, this likely would not affect the reported results in the paper. However, with a more competitive baseline, such noise may make it challenging to identify improvements in the selection of image for finetuning.\n\n* There are a few magic constants throughout the paper. While conducting an ablation study may be cost-prohibitive given the sequential dependencies of the algorithm, it would be helpful to the reader to provide some means of estimating the impact of and sensitivity to these parameters.\n\n---\n\nNits:\n* The comment on maximum test set size on page 1 is a bit strong. Test set sizes are limited by financial incentives. For potentially lucrative endeavors, it would not be surprising to find a test set larger than 10k images.\n\n* There is a typo on page 2: \"not be[en] spotted beforehand\"\n\n* It would be nice to quantify the computational cost of constructing M in each iteration.\n\n* Consider moving the superscript 4 after the period to make it clear that it is a footnote and not exponentiation.\n\n* There is a typo on page 4:(ARC) -> (ACR)\n\n* The F in \"failure\" is erroneously capitalized in the first paragraph of section 3.2\n\n* You may wish to consider citing the field of computer-assisted annotation as relevant work. One such paper (among others) and open-source implementation include:\n** Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++ (Acuna et al)\n** https://developer.nvidia.com/blog/annotation-transfer-learning-clara-train/", "extra_scores": {}, "extra_text": {}}, {"review_id": "PX0m4K1t5PX", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Annotating images for training of segmentation models is time consuming and it can be difficult to annotate enough examples to ensure good performance on the rare difficult examples that often occur when methods are applied to real world data. These cases are referred to as corner-cases. The paper therefore proposes a measure based on the discrepancy of a group of segmentation models to identify more valuable images to annotate and add to the training data in a iterative fashion. The approach is tested on the PASCAL VOC database.\n\nThe problem is important and relevant. I find the motivation clear, however, I disagree somewhat with the reasoning in parts of the introduction (see detailed comments). The approach is reasonable, and there is some evidence that performance is improved on these specific corner-cases by the addition of similarly identified corner-cases to the training set (Table 1). I am not sure how the approach would generalize, however. As I understand it the test-sets where performance is improved consists entirely of corner-cases identified in a similar manner, so whether the approach will generalize, depends entirely on how representable corner-cases identified in this manner are. It would have been much better to use a independent dataset of corner-cases identified manually.\n\n\nDetailed comments\n\n- \"First, segmentation benchmarks require pixel-level dense annotation\", I do not believe this is necessarily true, and there is little need to state this. One could certainly think of useful benchmarks with hard examples only. There are also examples of benchmarks where the groundtruth consists of computer segmentations corrected by humans.\n\n- \"Second, it is much harder for segmentation data to be class-balanced in the pixel level, making highly skewed class distributions notoriously common for this particular task\", \"Besides, the “universal” background class (often set to cover the distracting or uninteresting classes (Everingham et al., 2010)) adds additional complicacy to image segmentation (Mostajabi et al., 2015).\", while this may be true for training datasets, I do not see how this is a problem for benchmarks necessarily.\n\n- I find the description of the construction of the test dataset used in the different iterations unclear. It is my understanding, but I am not actually sure so it would be good to have the approach clarified, that the test datasets ($T^{(1)}$, $T^{(2)}$, and $T^{(3)}$) of iteration 1, 2, and 3 are hard examples, and are thus biased towards the methods involved. That is, they consist of examples that the proposed segmentation model disagrees with the \"competing\" models the most on. It is clear from the Table that these images are selected both based on mistakes of the competing models and mistakes of the target model. After one and two iterations we see that the target model now does much better on the next iteration of hard examples, but we really do not know how representative these hard examples are. If the methods tend to disagree on a limit number of typical cases, then these cases will be added to the training set and it is not so surprising that improvements in the target model is seen. To evaluate how this approach generalizes to the real world, an independent dataset would have to be used.\n\n- \"This also provides direct evidence that existing segmentation models could be particularly weak at certain real-world generalization, which is not surprising because the 1,464 training images are deemed to be extremely sparsely distributed in the space of natural images.\" I am not sure I agree with the evidence part. The dataset in question is selected to be hard (as far as I understand), so it is not surprising that the methods perform worse on it and does not say much about generalization. Essentially this is just evidence that the selection procedure is working as intended.\n\nClarity\nWhile I believe I understand most of the manuscript to an acceptable level, it contains quite a lot of sentences that would benefit from some editting. Some examples below.\n\n- Introduction is wordy, with long and difficult to understand sentences. Words that mean different things than the authors probably intended are also used, see examples \"boosting\", \"spot\", \"cover\" and wrong words are often used.\n\n- \"While the performance of segmentation models, as measured by excessively reused test sets (Everingham et al., 2010; Lin et al., 2014), keeps boosting\", keeps boosting is a bit of a weird phrase here, \"keeps improving\" perhaps?\n\n- \"suggesting their insufficiency to cover hard examples that may be encountered in the real world\", maybe replace \"insufficiency\" and \"cover\" with \"inability\" and \"handle\".\n\n- \"such test sets may only spot an extremely small subset of possible mistakes that the model will make\", \"spot\" is likely the wrong word to use here, maybe contain? But even so, test sets do not contain mistakes, the methods possibly make mistakes on the test set. Consider rewording.\n\n- \"The existence of natural adversarial examples (Hendrycks et al., 2019) also\nechos such hidden fragility of the classifiers to unseen examples\", while I could guess at what the sentence means, it does not really make sense.\n\n- \"which possess inherent transferability to falsify different image classifiers with the same type of errors\", not sure what you mean by this.\n\n- \"It is clear that images in S are visually much harder.\", something is missing.\n\n- \"weakly labelling method of filtering\", what do you mean by this?\n\n- \"Specifically, given the target model $f_t$, we let it compete with a group of state-of-the-art segmentation models {g_j}^m_{j=1} by maximizing the discrepancy (Wang et al., 2020) between f_t and g_j on D.\" They are not really competing are they? The point, as I understand it, is not to select the best model, but to find the most \"controversial\" image.\n\n- I would have prefered legends in each figure, as opposed to having to scroll up and down to find the relevant information.\n\n- \"indicating that many images in $T^{(1)}$ are able to falsify both the target model $f_t$...\", the images are not really falsifying the model.\n\n- \"This suggests that the target model begins to introspect and learn from its counterexamples\", the word introspect appears to be wrongly used here (and elsewhere).\n\n- \"Moreover, the top-1 model on $T^{(0)}$ does not necessarily perform the best on $T^{(1)}$ , conforming to the results in (Wang et al., 2020).\", what results are you talking about specifically? And I guess it should be \"confirming\".\n\nOriginality\nI don't find the work very original and it is not clear to me if the work is very novel. A lot of literature is referenced under related work, but I find it to be mostly tangentially related. It would be good if the authors could describe how this specific problem has been addressed before in image analysis and particularly image segmentation. A google search brings a number of works on hard negative mining. But \"human-in-the-loop\" techniques such as interactive training [1, 2] also enable annotators to focus more time on harder examples. The methodology itself is not groundbreaking. Multiple trained models have been used in combination to assess prediction certainty previously and uncertainty has also been used in active learning setups to focus annotations on difficult regions [3]. I am sure there are even more relevant links, but this is what a couple of minutes googling brought up.\n\nSignificance\nWhile the problem is relevant and the method possibly useful, because of previously mentioned concerns with respect to novelty and generalizability of the results, I do not think it will have a wide ranging significance.\n\n[1] Gonda, Felix, et al. \"Icon: An interactive approach to train deep neural networks for segmentation of neuronal structures.\" 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE, 2017.\n[2] Berg, Stuart, et al. \"Ilastik: interactive machine learning for (bio) image analysis.\" Nature Methods (2019): 1-7.\n[3] Casanova, Arantxa, et al. \"Reinforced active learning for image segmentation.\" arXiv preprint arXiv:2002.06583 (2020).\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "2h8alHDeOBV", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work used a variety of existing segmentation algorithms to discover most \"controversial\" samples from massive online unlabeled images. Those representative controversial samples were believed to have the best chance to confuse the algorithm being trained and to expose its weakness. They are rated by annotators on a spectrum from bad to excellent, and segmentation masks are collected from human annotators for the “worst” images. Several clever measures were taken to reduce human labor. \n\nThe paper addressed an important and somewhat overlooked problem for segmentation and deep learning in the general. Leveraging those counterexamples to improve the segmentation models' generalization performance on unseen images seems to be novel in this field. This looks like a special case of finding natural adversarial examples from unlabeled data. The proposed solution is intuitive and logical, with lots of practical considerations made for its feasibility. The manuscript is also very well written, and the literature review is especially comprehensive. \n\nThis work extends from MAD, ICLR 2020 (https://openreview.net/forum?id=rJehNT4YPr); but it also presents with two nontrivial and interesting innovations: (1) generalizing it to a dense prediction task, which requires revising the human labelling strategy in subjective experiments. Weakly-supervised labeling is more practical for segmentation; and (2) extending to active training/tuning, leveraging the selected hard examples to improve the segmentation model for multiple rounds. The tuned models were shown to improve their robustness remarkably on spotted catastrophic mistakes, while preserving their performance on canonical testing sets. \n\nThis approach is also an instance of the basic active learning paradigm that iterates between spotting hard examples, labeling them, and tuning the model. The method of finding hard examples by competing with a human oracle seems to be novel though. \n\nI noticed in the Appendix, the authors also compared their method with entropy-based active learning – a vanilla baseline needing no competing model. Their model seems to achieve a good mIoU advantage, showing the new proposed way has better performance on open-world images than simple entropy-based fine-tuning. That is perhaps understandable, since the proposed new method also costs more human inspection efforts. However, I’d be interested to see the authors to compare the hard examples selected by their method and by entropy-based one: are there any distinct pattern or notable trend? Is there anything that the entropy-based method can clear miss but your method can pick up?\n\nBesides, this paper uses different deep networks trained as competing models. Why wouldn’t those models more tend to make similar mistakes, due to same training dataset and model type? What if using learning-based but non-deep segmentation models to compete? What if using segmentation models trained on other datasets to compete?\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies how to efficiently expose failures of "top-performing" segmentation models in the real world and how to leverage such counterexamples to rectify the models. The key idea is to discover most "controversial" samples from massive online unlabeled images. The approach is sound, well grounded, and quite logical. Results demonstrate the effectiveness.

However, there exists some limitations coming from R2 and R3, for example, 1) Segmentation benchmarks may not require pixel-level dense annotation. There are also examples of benchmarks where the groundtruth consists of computer segmentations corrected by humans. 2) It is much harder for segmentation data to be class-balanced in the pixel level, making highly skewed class distributions common for this particular task. 3) Citing the field of computer-assisted annotation as relevant work.

In the end, I think that this paper may not be ready for publication at ICLR, but the next version must be a strong paper if above limitations can be well addressed.
Decision: Reject