Efficiently Troubleshooting Image Segmentation Models with Human-In-The-Loop

1 INTRODUCTION . Image segmentation ( i.e. , pixel-level image labelling ) has recently risen to explosive popularity , due in part to its profound impact on many high-stakes vision applications , such as autonomous driving and medical image analysis . While the performance of segmentation models , as measured by excessively reused test sets ( Everingham et al. , 2010 ; Lin et al. , 2014 ) , keeps improving ( Chen et al. , 2018a ; Badrinarayanan et al. , 2017 ; Yu et al. , 2018 ) , two scientific questions have arisen to capture the community ’ s curiosity , and motivate the current work : Q1 : Do “ top-performing ” segmentation models on existing benchmarks generalize to the real world with much richer variations ? Q2 : Can we identify and rectify the trained models ’ sparse but fatal mistakes , without incurring significant workload of human labelling ? The answer to the first question is conceptually clearer , by taking reference to a series of recent work on image classification ( Recht et al. , 2019 ; Hendrycks et al. , 2019 ) . A typical test set for image classification can only include a maximum of ten thousands of images because human labelling ( or verification of predicted labels ) is expensive and time-consuming . Considering the high dimensionality of image space and the “ human-level ” performance of existing methods , such test sets may only spot an extremely small subset of possible mistakes that the model will make , suggesting their insufficiency to cover hard examples that may be encountered in the real world ( Wang et al. , 2020 ) . The existence of natural adversarial examples ( Hendrycks et al. , 2019 ) also echos such hidden fragility of the classifiers to unseen examples , despite the impressive accuracy on existing benchmarks . While the above problem has not been studied in the context of image segmentation , we argue that it would only be much amplified for two main reasons . First , segmentation benchmarks require pixel-level dense annotation . Compared to classification databases , they are much more expensive , laborious , and error-prone to label1 , making existing segmentation datasets even more restricted in scale . Second , it is much harder for segmentation data to be class-balanced in the pixel level , making highly skewed class distributions notoriously common for this particular task ( Kervadec et al. , 2019 ; Bischke et al. , 2018 ) . Besides , the “ universal ” background class ( often set to cover the distracting or uninteresting classes ( Everingham et al. , 2010 ) ) adds additional complicacy to image segmentation ( Mostajabi et al. , 2015 ) . Thus , it remains questionable to what extent the impressive performance on existing benchmarks can be interpreted as ( or translated into ) real-world robustness . If “ top-performing ” segmentation models make sparse yet catastrophic mistakes that have not been spotted beforehand , they will fall short of the need by high-stakes applications . The answer to the second question constitutes the main body of our technical work . In order to identify sparse failures of existing segmentation models , it is necessary to expose them to a much larger corpus of real-world labelled images ( on the order millions or even billions ) . This is , however , implausible due to the expensiveness of dense labelling in image segmentation . The core question essentially boils down to : how to efficiently decide what to label from the massive unlabelled images , such that a small number of annotated images maximally expose corner-case defects , and can be leveraged to improve the models . In this paper , we introduce a two-stage framework with human-in-the-loop for efficiently troubleshooting image segmentation models ( see Figure 1 ) . The first stage automatically mines , from a large pool D of unlabelled images , a small image set M , which are the most informative in exposing weaknesses of the target model . Specifically , inspired by previous studies on model falsification as model comparison ( Wang & Simoncelli , 2008 ; Ma et al. , 2018 ; Wang et al. , 2020 ) , we let the target model compete with a set of state-of-the-art methods with different design methodologies , and sample images by MAximizing the Discrepancy ( MAD ) between the methods . To reduce the number of false positives , we propose a weakly labelling method of filteringM to obtain a smaller refined set S , subject to segmentation by human subjects . In the second stage , we fine-tune the target model to learn from the counterexamples in S without forgetting previously seen data . The two stages may be iterated , enabling progressive troubleshooting of image segmentation models . Experiments on PASCAL VOC ( Everingham et al. , 2010 ) demonstrate the feasibility of the proposed method to address this new challenging problem , where we successfully discover corner-case errors of a “ top-performing ” segmentation model ( Chen et al. , 2017 ) , and fix it for improved generalization in the wild . 2 RELATED WORK . MAD competition The proposed method takes inspiration from the MAD competition ( Wang & Simoncelli , 2008 ; Wang et al. , 2020 ) to efficiently spot model failures . Previous works focused on performance evaluation . We take one step further to also fix the model errors detected in the MAD competition . To the best of our knowledge , our work is the first to extend the MAD idea to image segmentation , where labeling efficiency is more desired since pixel-wise human annotation for image segmentation is much more time-consuming than image quality assessment ( Wang & Simoncelli , 2008 ) and image classification ( Wang et al. , 2020 ) tasks previously explored . 1According to Everingham et al . ( Everingham et al. , 2010 ) and our practice , it can easily take ten times as long to segment an object than to draw a bounding box around it . Differential testing Our method is also loosely related to the cross-disciplinary field of software/system testing , especially the differential testing technique ( McKeeman , 1998 ) . By providing the same tests to multiple software implementations , differential testing aims to find bug-exposing test samples that lead to different results . Programmers can then dig into these test cases for potential bug fixing . While debugging software is very different from troubleshooting machine learning algorithms , a handful of recent work explored this idea to find pitfalls of deep learning-based autonomous driving systems ( Pei et al. , 2017 ; Tian et al. , 2018 ; Zhang et al. , 2018 ) . Aiming to be fully automated without human intervention , these methods have to make strong assumptions such as the ground truth labels can be determined by majority vote2 or are unchanged under some synthetic image transformations3 ( e.g. , brightness and contrast change , or style transfer ) . Therefore , it is unclear how to generalize the results obtained in such noisy and often unrealistic settings , to the real world with both great content fidelity and diversity . Adversarial examples Introduced by Dalvi et al . ( Dalvi et al. , 2004 ) and reignited by Szegedy et al . ( Szegedy et al. , 2013 ) , most adversarial attacks add small synthetic perturbations to inputs of computational models that cause them to make incorrect decisions . In image classification , Hendrycks et al . ( Hendrycks et al. , 2019 ) identified a set of natural images that behave like synthetic adversarial examples , which possess inherent transferability to falsify different image classifiers with the same type of errors . The selected counterexamples by the proposed framework might be treated as a new type of natural adversarial examples , that force the two models to make distinct predictions , therefore capable of fooling at least one model . Similar as natural counterexamples focused in this work , synthetic adversarial examples pose security risks of deploying machine learning algorithms in real-world applications . A large body of research ( Madry et al. , 2018 ; Zhang et al. , 2019 ) delves deep into adversarial training , trying to defend against adversarial perturbations at the expensive cost of sacrificing the generalization on original test sets without perturbations ( Tsipras et al. , 2019 ; Zhang et al. , 2019 ; Schmidt et al. , 2018 ) . This seems to suggest a trade-off between generalization to real-world benign examples and robustness to adversarial attacks . Semantic segmentation with deep learning Fully convolutional network ( Long et al. , 2015 ) was among the first deep architectures adopted for high-quality segmentation . Skip connection ( Ronneberger et al. , 2015 ) , recurrent module ( Zheng et al. , 2015 ) , max index pooling ( Noh et al. , 2015 ; Badrinarayanan et al. , 2017 ) , dilated convolution ( Chen et al. , 2014 ; Yu & Koltun , 2016 ; Chen et al. , 2018a ) , and multi-scale training ( Chen et al. , 2016 ; 2018a ) are typical strategies to boost the segmentation performance . Conditional random fields ( Ladickỳ et al. , 2010 ) used to dominate image segmentation before the advent of deep learning were also combined with convolutional networks to model spatial relationships ( Zheng et al. , 2015 ) . We refer interested readers to ( Minaee et al. , 2020 ) for a comprehensive survey of this field . 3 PROPOSED METHOD . Suppose we have a target segmentation model ft : Rh×w 7→ { 1 , · · · , c } h×w , where h and w are the height and width of the input image , and c denotes the number of categories . Our goal is to efficiently identify and fix the failure cases of ft encountered in the real world , while minimizing human labelling effort in this process . We start by constructing a large image database D , whose collection may be guided by the keywords that represent the c categories . Rather than conducting large-scale subjective testing to obtain the ground truth segmentation map for each x ∈ D , we choose to create a small subset of imagesM ⊂ D , which are strong candidates for revealing corner-case behaviors of ft. To further reduce false positive examples inM , we describe a method to gather a weak label for each x ∈ M as an overall indicator of segmentation quality . Based on the labels , an even smaller set S ⊂M can be obtained for dense annotation by humans . Last , we fine-tune ft 2As per ( Hendrycks et al. , 2019 ) , machine learning algorithms with similar design philosophies tend to make common mistakes . 3In many areas of computer vision , methods trained on synthetic data can not generalize to realistic data , and specialized techniques such as domain adaptation ( Zhang et al. , 2017 ; Zhao et al. , 2019 ) have to be used to bridge the performance gap . on the combination of S and previously trained data , in an attempt to learn from the found failures without forgetting ( Li & Hoiem , 2017 ) . 3.1 FAILURE IDENTIFICATION . ConstructingM Inspired by model falsification methodologies from computational vision ( Wang & Simoncelli , 2008 ) and software engineering ( McKeeman , 1998 ) , we construct the setM = { xi } n2i=1 by sampling the most “ controversial ” images from the large-scale unlabelled database D = { xi } n1i=1 , where n2 n1 . Specifically , given the target model ft , we let it compete with a group of state-ofthe-art segmentation models { gj } mj=1 by maximizing the discrepancy ( Wang et al. , 2020 ) between ft and gj on D : x̂ ( j ) = argmax x∈D d ( ft ( x ) , gj ( x ) ) , j = 1 , . . . , m , ( 1 ) where d ( · ) is a distance metric to gauge the dissimilarity between two segmentation maps ( e.g. , negative pixel accuracy , or mean region intersection over union ( mIoU ) ) . x̂ ( j ) represents the most controversial image according to ft and gj , and therefore is the most informative in distinguishing between them . If the competing model gj performs at a high level , and differs from the target model ft in design , x̂ ( j ) is likely to be a failure of ft. To avoid identifying different instantiations of the same underlying root cause ( Pei et al. , 2017 ) and to encourage content diversity of the candidate images , we describe a “ contentaware ” method for constructing M. We first partition D into c overlapped subgroups { Dt , k } ck=1 based on ft ’ s predicted maps , where x ∈ Dt , k if at least one pixel in ft ( x ) belongs to the k-th category . After that , we add a content constraint by restricting the size of predicted pixels in the k-th category , i.e. , ∑ 1 [ ft ( x ) == k ] / ( h × w ) , within the range of [ pk , qk ] . This allows excluding images of exceedingly large ( or small ) object sizes , which may be of less practical relevance . Moreover , instead of focusing on the most controversial example defined in Eq . ( 1 ) , we look at top-n3 images in Dt , k with n3 largest distances computed by { x̂ ( j , k ) i } n3 i=1 = argmax { xi } n3 i=1∈Dt , k n3∑ i=1 d ( ft ( xi ) , gj ( xi ) ) , j = 1 , . . . , m , k = 1 , . . . , c. ( 2 ) We then repeat this procedure , but with the roles of ft and gj reversed . That is , we partition D into c subgroups { Dj , k } ck=1 according to gj , and solve the maximization problem over Dj , k . Finally , we gather all candidate images to arrive at the setM = { xi } n2i=1 , where n2 ≤ 2mcn3 n1.4 Constructing S Although images in the candidate setM have great potentials of being counterexamples of ft , some false positives may be included ( see Figure 2 ) , especially when gj is inferior to ft . In addition , no data screening is involved in the construction of D , increasing chances of including images that are out-of-domain ( e.g. , falling out of the c categories and/or containing inappropriate content ) . In view of these , we reduce false positives inM via a weakly labelling strategy . For each x ∈M , we ask human subjects to give a discrete score on an absolute category rating ( ACR ) scale to indicate the segmentation quality of ft ( x ) . The labels on the scale are “ bad ” , “ poor ” , “ fair ” , “ good ” , 4We have n2 ≤ 2mcn3 because the same images may be optimal in different problems specified in Eq . ( 2 ) . and “ excellent ” ( see Figure 8 ) . We then rank all images inM by the mean opinion scores , and choose top-n4 images with the smallest scores to form the counterexample set S = { xi } n4i=1 . Finally , we seek pixel-level segmentation results from human annotators for each image in S ( see Figure 9 ) . 3.2 MODEL RECTIFICATION Algorithm 1 : The proposed framework for efficiently troubleshooting segmentation models Input : An unlabelled image set D , a target model f ( 0 ) t and the dataset S ( 0 ) on which it is pre-trained , a group of competing models { gj } mj=1 , the maximum number r of fine-tuning rounds , hyper-parameters n3 , n4 Output : Improved f ( r ) t 1 S ← ∅ 2 for j ← 1 to m do 3 Compute segmentation predictions { gj ( x ) , x ∈ D } 4 end 5 for i← 0 to r − 1 do 6 M ( i+1 ) ← ∅ 7 Compute segmentation predictions { f ( i ) t ( x ) , x ∈ D } 8 for j ← 1 to m do 9 Compute the distances { d ( f ( i ) t ( x ) , gj ( x ) ) , x ∈ D } 10 Divide D into c subgroups according to f ( i ) t 11 Filter images by the content constraint 12 Select top-n3 images by solving Eq . ( 2 ) to formMj 13 M ( i+1 ) ←M ( i+1 ) ⋃ Mj 14 Reverse the roles of f ( i ) t and gj , and repeat Steps 10 to 13 15 end 16 Source weak human scores forM ( i+1 ) 17 Select top-n4 images with the lowest quality scores and collect pixel-level labels from humans to form S ( i+1 ) 18 S ← S ⋃ S ( i+1 ) ; 19 Fine-tune f ( i ) t on the combination of S and S ( 0 ) 20 end The labelled images in S give us a great opportunity to improve ft by learning from these failures . In order to be resistant to catastrophic forgetting ( McCloskey & Cohen , 1989 ) , the fine-tuning can also include labelled data previously used to train ft. We may iterate through the whole procedure of failure identification and model rectification several rounds , leading to a progressive two-stage framework for efficiently troubleshooting segmentation models with human-in-the-loop . When the iterative setting is enabled , the size of S is growing : S = ⋃r i=1 S ( i ) , where S ( i ) is the counterexample set created in the i-th fine-tuning round . Denoting the initial target model by f ( 0 ) t , we fine-tune f ( i−1 ) t on the combination of S accumulated in the previous i − 1 rounds and S ( 0 ) used for pre-training . We summarize the proposed framework in Algorithm 1 .

Summary: This work used a variety of existing segmentation algorithms to discover most "controversial" samples from massive online unlabeled images. Those representative controversial samples were believed to have the best chance to confuse the algorithm being trained and to expose its weakness. They are rated by annotators on a spectrum from bad to excellent, and segmentation masks are collected from human annotators for the “worst” images. Several clever measures were taken to reduce human labor.