TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks

1 Introduction . Deep learning ( DL ) systems are prone to errors due to many factors , such as the biased training/validation dataset , the limitations of the model architecture , and the constraints on training cost . It is essential to conduct high-quality testing before DL models are deployed in the field ; otherwise , the behaviors of DL models can be unpredictable and result in severe accidents after deployment . However , the cost of building test oracles ( i.e. , the ground-truth output ) by manually labeling a massive set of test instances is prohibitive , especially for tasks requiring experts for accurate labeling , such as medical images and malware executables . To tackle the above problem , various test input prioritization techniques [ Feng et al. , 2020 , Byun et al. , 2019 , Shen et al. , 2020 ] are proposed to identify ‘ high-quality ’ test instances from a large amount of unlabeled data , which facilitates revealing more failures ( e.g. , misclassification ) of the DL model with reasonable labeling effort . These methods try to derive the failure-revealing capability of a test instance with its intrinsic attributes extracted from the responses of the model under test ( e.g. , the softmax-based probabilities given by the target DL model to this specific input ) . DeepGini [ Feng et al. , 2020 ] feeds the unlabeled data to the target DL model and calculates confidence-related scores based on the model ’ s output probabilities to rank the unlabeled test cases . Test cases with nearly equal probabilities on all output classes are regarded as less confident ones and are likely to reveal 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . model failures . Similarly , [ Byun et al. , 2019 ] use the uncertainty score obtained from MC-Dropout for test input prioritization . Multiple-boundary clustering and prioritization ( MCP ) [ Shen et al. , 2020 ] considers both the output probabilities and the balance among each classification boundary when selecting test cases . All existing works try to identify instances near the decision boundary and prioritize them . However , we argue that near-boundary instances are not necessarily failures , especially for well-trained classifiers with high accuracy . Also , as failures can be far from the decision boundary , existing methods could fail to reveal these remote failures . To estimate a test instance ’ s capability in revealing failures , in addition to the intrinsic attributes mentioned above , there is another type of information : the known classification correctness of labeled samples ( i.e. , training samples and previously tested samples ) and their relationship to the unlabeled instance . This information provides extra insight into the target model ’ s behavior . Such data is already known , and it provides contextual information that reflects the corresponding inference behaviors of the target model for a set of similar instances . This work presents a novel test input prioritization technique , namely TestRank , for DL classifiers . TestRank exploits both intrinsic and contextual attributes of test instances to evaluate their failurerevealing capabilities . Based on the intuition that similar inputs are usually associated with the same classification results , we propose to use graph neural networks ( GNNs ) [ Kipf and Welling , 2017 ] to summarize the neighboring classification correctness for each unlabeled instance into contextual attributes . GNNs have been well-studied and valued for their relational inductive bias for extracting graph information . Our method , TestRank , constructs a similarity graph on both unlabeled and labeled instances and apply the semi-supervised GNN learning to extract the contextual attributes . After that , we aggregate intrinsic ( such attributes are extracted from the input samples without considering their neighbors ) and contextual attributes with a neural-network-based binary classifier for test input prioritization . The contributions of our work are as follows : • To the best of our knowledge , TestRank is the first work that takes the contextual information from the target DL model into consideration for test input prioritization . • We propose constructing a similarity graph on both labeled and unlabeled samples and training a graph neural network to extract useful contextual attributes from the contextual information for these unlabeled instances . We also present approximation techniques to reduce its computational complexity with minor impact on the performance of TestRank . • We propose a simple yet effective neural network that combines the intrinsic attributes and contextual attributes of unlabeled test instances for their failure-revealing capability estimation . We empirically evaluate TestRank on three popular image classification benchmarks : CIFAR-10 , SVHN , and STL10 . The results show that our method outperforms the state-of-the-art methods by a considerable margin . 2 Test Input Prioritization . Let us use f : X → Y to represent the given target DL model , where X and Y are the input and output space , respectively . For effective testing1 , the debugging center needs to perform test input prioritization , i.e. , select a certain number of test instances from the large unlabeled test instance pool that can reveal as many failures as possible . Later , these failures are fed back to the training center for failure analysis and model repair . We define the model failures as follows : Definition 1 . DL Model Failure . A failure of the DL model can be uncovered by the test instance x if the predicted label f ( x ) is inconsistent with its ground truth label yx , namely f ( x ) , yx . Formally , the debugging center selects and labels b test cases XS ( |XS | = b ) from the unlabeled test instance pool XU . The objective of test input prioritization is to maximize the detected failures : max | { x| f ( x ) , yx } | , where x ∈ XS and |XS | = b . ( 1 ) 1Please note that , we focus on testing the functional correctness of the DL model , and we assume the collected testing data are clean samples instead of maliciously generated ones . Different solutions are proposed to quantify the failure-revealing capability of unlabeled instances . DeepGini [ Feng et al. , 2020 ] proposes to evaluate a single test instance via the DL model ’ s final statistical output : f ( t ) = 1 − ΣNi=1 p2t , i , where pt , i is the predicted probability that the test case t belongs to the class i . Given the sum of pt , i is 1 , impurity function f ( t ) is maximal when all pt , i values are equal . DeepGini also adopts the neuron coverage criteria proposed in DeepXplore [ Pei et al. , 2017 ] and DeepGauge [ Ma et al. , 2018 ] in test input prioritization , and the result shows that the impurity-based selection is much better than coverage-based selection . Instead of evaluating the overall likelihood of failure for all classes , Multiple-Boundary Clustering and Prioritization ( MCP ) proposes to evaluate it for each pair of classes individually [ Shen et al. , 2020 ] . In this way , test instances can be evenly selected for each class pair and the failure cases are investigated at the finer granularity . Besides these metrics , [ Byun et al. , 2019 ] also propose to measure the likelihood of incorrect prediction by the uncertainty of the model ’ s output , which reflects the degree to which a model is uncertain about its prediction . In practice , evaluating uncertainty requires the task DL model to be a Bayesian Neural Network [ Richard and Lippmann , 1991 , Neal , 2012 ] or containing a dropout layer for approximation [ Gal and Ghahramani , 2016 ] . Besides examining the DL model ’ s final outputs , [ Kim et al. , 2019 ] proposes two surprise adequacy ( SA ) criteria that make use of the target DL ’ s internal outputs ( e.g. , the activation traces ) . They are Likelihood-based Surprise Adequacy Coverage ( LSA ) and Distance-based Surprise Adequacy Coverage ( DSA ) . LSA and DSA measure the likelihood or distance of an unlabeled instance to the training instances , respectively . Test samples with higher SA values are preferred in testing . To sum up , all existing methods use the target model ’ s outputs to one input , i.e. , its intrinsic attributes , for its failure-revealing capability estimation . In contrast , we make use of both intrinsic and contextual attributes of an instance for better estimation ( see later sections for details ) . 3 TestRank 3.1 Motivation The failure-revealing capability of an unlabeled test input is closely related to its attributes for the DL model under test . In this work , we distinguish two kinds of attributes for an unlabeled instance : the intrinsic attributes and the contextual attributes . We define the intrinsic attributes of an input as the output responses assigned by the target DL model to this specific input . It could be , for example , the predictive output distribution of the input from the target DL model , reflecting the sentiment derived from the computation performed by the target model [ Byun et al. , 2019 ] . This kind of attributes is adopted by existing test input prioritization approaches [ Feng et al. , 2020 , Shen et al. , 2020 , Byun et al. , 2019 ] . Note that we define such attributes as ‘ intrinsic ’ because they are extracted from inputs without considering their context , i.e. , the classification correctness of its similar instances . In contrast with the intrinsic attributes , the contextual attributes provide a deeper insight into the target model for the unlabeled samples : the contextual attributes for an unlabeled sample summarize the classification correctness of similar and labeled samples . For a particular test instance , such contextual attributes are useful and complementary to the intrinsic attributes . An illustrative example is shown in Figure 1 , wherein we visualize the behavior of a two-class classifier on the unlabeled test data and historically labeled data distribution . The blue region includes the instances that are near the decision boundary . Intuitively , the classifier is uncertain about the data when data is near the decision boundary and is likely to misclassify it . Existing works [ Shen et al. , 2020 , Feng et al. , 2020 , Byun et al. , 2019 ] propose various indicators ( e.g. , confidence/uncertainty/surprise scores ) to help identify the near-boundary instances . However , the near-boundary instances are not necessarily failures , and some of them can be correctly classified by a well-trained classifier . What is worse , such testing approaches fail to capture the failures lying far from the decision boundary ( i.e. , remote failures , shown in the red region in Figure 1 ) , because DL models usually output high confidence ( or low uncertainty ) for these inputs . These failures may be caused by limited model capacity , insufficient training data , etc . Our key insight is that we can use the contextual information ( e.g . the classification correctness of similar labeled samples ) to help locate both near-boundary and remote failures . The usefulness of the contextual information is due to the local continuity property [ Bishop , 2006 ] , which means that inputs close in the feature space share similar prediction behavior , e.g. , classification results from the target model . As shown in Figure 1 , some already labeled data , whose classification correctness is already known , surround the unlabeled data . If an unlabeled instance is close to already falsely classified data , under the local continuity property , it is likely that this instance is also a model failure . This property motivates us to extract the contextual attributes for an unlabeled instance from its neighboring labeled data . By combining the extracted contextual attributes with the intrinsic attributes , we expect to achieve better failure-revealing capability estimation .

Summary: - Given a dataset of labeled and unlabeled examples, authors build a   data point prioritization technique, which helps choose data points   to be labeled next, with a goal of revealing failures as soon as   possible. The technique should help prioritize labelling instances   which currently trained model would misclassify. - They use intrinsic and contextual attributes to build a model   performing data point prioritization. Intrinsic attributes are those   that are obtained by observing outputs of the deep learning model   for a given input. Contextual capture information about similarity   to the other instances -- if a given instance is similar to   misclassified instances, that would be a strong signal that the   instance is problematic. - For intrinsic attributes, they use the output layer logits (before   the softmax activation) of the DL model to construct a vector to be   used for a final model. Second, they use a graph neural network to   create contextual representation vector out of both labeled and   unlabeled data. The aforementioned vectors are input to an   multi-layer perceptron (MLP) model. The output of the MLP is a   probability that a given input would be misclassified by the DL   model. - They evaluate their model against other data point prioritization   techniques measuring number of failure revealing instances for a   given labelling budget.