TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks

1 Introduction . Deep learning ( DL ) systems are prone to errors due to many factors , such as the biased training/validation dataset , the limitations of the model architecture , and the constraints on training cost . It is essential to conduct high-quality testing before DL models are deployed in the field ; otherwise , the behaviors of DL models can be unpredictable and result in severe accidents after deployment . However , the cost of building test oracles ( i.e. , the ground-truth output ) by manually labeling a massive set of test instances is prohibitive , especially for tasks requiring experts for accurate labeling , such as medical images and malware executables . To tackle the above problem , various test input prioritization techniques [ Feng et al. , 2020 , Byun et al. , 2019 , Shen et al. , 2020 ] are proposed to identify ‘ high-quality ’ test instances from a large amount of unlabeled data , which facilitates revealing more failures ( e.g. , misclassification ) of the DL model with reasonable labeling effort . These methods try to derive the failure-revealing capability of a test instance with its intrinsic attributes extracted from the responses of the model under test ( e.g. , the softmax-based probabilities given by the target DL model to this specific input ) . DeepGini [ Feng et al. , 2020 ] feeds the unlabeled data to the target DL model and calculates confidence-related scores based on the model ’ s output probabilities to rank the unlabeled test cases . Test cases with nearly equal probabilities on all output classes are regarded as less confident ones and are likely to reveal 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . model failures . Similarly , [ Byun et al. , 2019 ] use the uncertainty score obtained from MC-Dropout for test input prioritization . Multiple-boundary clustering and prioritization ( MCP ) [ Shen et al. , 2020 ] considers both the output probabilities and the balance among each classification boundary when selecting test cases . All existing works try to identify instances near the decision boundary and prioritize them . However , we argue that near-boundary instances are not necessarily failures , especially for well-trained classifiers with high accuracy . Also , as failures can be far from the decision boundary , existing methods could fail to reveal these remote failures . To estimate a test instance ’ s capability in revealing failures , in addition to the intrinsic attributes mentioned above , there is another type of information : the known classification correctness of labeled samples ( i.e. , training samples and previously tested samples ) and their relationship to the unlabeled instance . This information provides extra insight into the target model ’ s behavior . Such data is already known , and it provides contextual information that reflects the corresponding inference behaviors of the target model for a set of similar instances . This work presents a novel test input prioritization technique , namely TestRank , for DL classifiers . TestRank exploits both intrinsic and contextual attributes of test instances to evaluate their failurerevealing capabilities . Based on the intuition that similar inputs are usually associated with the same classification results , we propose to use graph neural networks ( GNNs ) [ Kipf and Welling , 2017 ] to summarize the neighboring classification correctness for each unlabeled instance into contextual attributes . GNNs have been well-studied and valued for their relational inductive bias for extracting graph information . Our method , TestRank , constructs a similarity graph on both unlabeled and labeled instances and apply the semi-supervised GNN learning to extract the contextual attributes . After that , we aggregate intrinsic ( such attributes are extracted from the input samples without considering their neighbors ) and contextual attributes with a neural-network-based binary classifier for test input prioritization . The contributions of our work are as follows : • To the best of our knowledge , TestRank is the first work that takes the contextual information from the target DL model into consideration for test input prioritization . • We propose constructing a similarity graph on both labeled and unlabeled samples and training a graph neural network to extract useful contextual attributes from the contextual information for these unlabeled instances . We also present approximation techniques to reduce its computational complexity with minor impact on the performance of TestRank . • We propose a simple yet effective neural network that combines the intrinsic attributes and contextual attributes of unlabeled test instances for their failure-revealing capability estimation . We empirically evaluate TestRank on three popular image classification benchmarks : CIFAR-10 , SVHN , and STL10 . The results show that our method outperforms the state-of-the-art methods by a considerable margin . 2 Test Input Prioritization . Let us use f : X → Y to represent the given target DL model , where X and Y are the input and output space , respectively . For effective testing1 , the debugging center needs to perform test input prioritization , i.e. , select a certain number of test instances from the large unlabeled test instance pool that can reveal as many failures as possible . Later , these failures are fed back to the training center for failure analysis and model repair . We define the model failures as follows : Definition 1 . DL Model Failure . A failure of the DL model can be uncovered by the test instance x if the predicted label f ( x ) is inconsistent with its ground truth label yx , namely f ( x ) , yx . Formally , the debugging center selects and labels b test cases XS ( |XS | = b ) from the unlabeled test instance pool XU . The objective of test input prioritization is to maximize the detected failures : max | { x| f ( x ) , yx } | , where x ∈ XS and |XS | = b . ( 1 ) 1Please note that , we focus on testing the functional correctness of the DL model , and we assume the collected testing data are clean samples instead of maliciously generated ones . Different solutions are proposed to quantify the failure-revealing capability of unlabeled instances . DeepGini [ Feng et al. , 2020 ] proposes to evaluate a single test instance via the DL model ’ s final statistical output : f ( t ) = 1 − ΣNi=1 p2t , i , where pt , i is the predicted probability that the test case t belongs to the class i . Given the sum of pt , i is 1 , impurity function f ( t ) is maximal when all pt , i values are equal . DeepGini also adopts the neuron coverage criteria proposed in DeepXplore [ Pei et al. , 2017 ] and DeepGauge [ Ma et al. , 2018 ] in test input prioritization , and the result shows that the impurity-based selection is much better than coverage-based selection . Instead of evaluating the overall likelihood of failure for all classes , Multiple-Boundary Clustering and Prioritization ( MCP ) proposes to evaluate it for each pair of classes individually [ Shen et al. , 2020 ] . In this way , test instances can be evenly selected for each class pair and the failure cases are investigated at the finer granularity . Besides these metrics , [ Byun et al. , 2019 ] also propose to measure the likelihood of incorrect prediction by the uncertainty of the model ’ s output , which reflects the degree to which a model is uncertain about its prediction . In practice , evaluating uncertainty requires the task DL model to be a Bayesian Neural Network [ Richard and Lippmann , 1991 , Neal , 2012 ] or containing a dropout layer for approximation [ Gal and Ghahramani , 2016 ] . Besides examining the DL model ’ s final outputs , [ Kim et al. , 2019 ] proposes two surprise adequacy ( SA ) criteria that make use of the target DL ’ s internal outputs ( e.g. , the activation traces ) . They are Likelihood-based Surprise Adequacy Coverage ( LSA ) and Distance-based Surprise Adequacy Coverage ( DSA ) . LSA and DSA measure the likelihood or distance of an unlabeled instance to the training instances , respectively . Test samples with higher SA values are preferred in testing . To sum up , all existing methods use the target model ’ s outputs to one input , i.e. , its intrinsic attributes , for its failure-revealing capability estimation . In contrast , we make use of both intrinsic and contextual attributes of an instance for better estimation ( see later sections for details ) . 3 TestRank 3.1 Motivation The failure-revealing capability of an unlabeled test input is closely related to its attributes for the DL model under test . In this work , we distinguish two kinds of attributes for an unlabeled instance : the intrinsic attributes and the contextual attributes . We define the intrinsic attributes of an input as the output responses assigned by the target DL model to this specific input . It could be , for example , the predictive output distribution of the input from the target DL model , reflecting the sentiment derived from the computation performed by the target model [ Byun et al. , 2019 ] . This kind of attributes is adopted by existing test input prioritization approaches [ Feng et al. , 2020 , Shen et al. , 2020 , Byun et al. , 2019 ] . Note that we define such attributes as ‘ intrinsic ’ because they are extracted from inputs without considering their context , i.e. , the classification correctness of its similar instances . In contrast with the intrinsic attributes , the contextual attributes provide a deeper insight into the target model for the unlabeled samples : the contextual attributes for an unlabeled sample summarize the classification correctness of similar and labeled samples . For a particular test instance , such contextual attributes are useful and complementary to the intrinsic attributes . An illustrative example is shown in Figure 1 , wherein we visualize the behavior of a two-class classifier on the unlabeled test data and historically labeled data distribution . The blue region includes the instances that are near the decision boundary . Intuitively , the classifier is uncertain about the data when data is near the decision boundary and is likely to misclassify it . Existing works [ Shen et al. , 2020 , Feng et al. , 2020 , Byun et al. , 2019 ] propose various indicators ( e.g. , confidence/uncertainty/surprise scores ) to help identify the near-boundary instances . However , the near-boundary instances are not necessarily failures , and some of them can be correctly classified by a well-trained classifier . What is worse , such testing approaches fail to capture the failures lying far from the decision boundary ( i.e. , remote failures , shown in the red region in Figure 1 ) , because DL models usually output high confidence ( or low uncertainty ) for these inputs . These failures may be caused by limited model capacity , insufficient training data , etc . Our key insight is that we can use the contextual information ( e.g . the classification correctness of similar labeled samples ) to help locate both near-boundary and remote failures . The usefulness of the contextual information is due to the local continuity property [ Bishop , 2006 ] , which means that inputs close in the feature space share similar prediction behavior , e.g. , classification results from the target model . As shown in Figure 1 , some already labeled data , whose classification correctness is already known , surround the unlabeled data . If an unlabeled instance is close to already falsely classified data , under the local continuity property , it is likely that this instance is also a model failure . This property motivates us to extract the contextual attributes for an unlabeled instance from its neighboring labeled data . By combining the extracted contextual attributes with the intrinsic attributes , we expect to achieve better failure-revealing capability estimation .

Summary: This paper proposes a new test sample prioritization strategy to test deep learning systems. The objective is to maximize the number of failure-inducing samples selected from unlabeled samples. The authors propose to leverage both the samples' intrinsic properties (from the model's prediction behavior on them) and contextual properties (from other labeled samples close to them) as the criteria to select samples. By learning another binary classifier that takes the two properties as input, TestRank produces probabilities to rank the test samples. Empirical results demonstrate that TestRank outperforms existing test prioritization techniques.