TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks
Authors: YU LI
Min Li
Qiuxia LAI
Yannan Liu
Qiang Xu
Venue: NeurIPS 2021 Poster (2021)
Keywords: Deep Neural Networks
Debugging
Testing
Graph Neural Network
Test input prioritization

Abstract: Deep learning (DL) systems are notoriously difficult to test and debug due to the lack of correctness proof and the huge test input space to cover. Given the ubiquitous unlabeled test data and high labeling cost, in this paper, we propose a novel test prioritization technique, namely TestRank, which aims at revealing more model failures with less labeling effort. TestRank brings order into the unlabeled test data according to their likelihood of being a failure, i.e., their failure-revealing capabilities. Different from existing solutions, TestRank leverages both intrinsic and contextual attributes of the unlabeled test data when prioritizing them. To be specific, we first build a similarity graph on both unlabeled test samples and labeled samples (e.g., training or previously labeled test samples). Then, we conduct graph-based semi-supervised learning to extract contextual features from the correctness of similar labeled samples. For a particular test instance, the contextual features extracted with the graph neural network and the intrinsic features obtained with the DL model itself are combined to predict its failure-revealing capability. Finally, TestRank prioritizes unlabeled test inputs in descending order of the above probability value. We evaluate TestRank on three popular image classification datasets, and results show that TestRank significantly outperforms existing test prioritization techniques. 
  

TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks
Yu Li', Min Li', Qiuxia Lai', Yannan Liu", and Qiang Xu' CURE Lab, The Chinese University of Hong Kong Wuhenn Lab, ByteDance [yuli, mli, qxlai qcueecse. cuhk. edu. hk
Abstract
Deep learning (DL) systems notoriously difficult test and debug due the lack of correctness proof and the huge test input space to cover. Given the ubiquitous unlabeled test data and high labeling cost, in this paper, we propose a novel test input prioritization technique, namely TestRank, which aims at revealing more model failures with less labeling effort. TestRank brings order into the unlabeled test data according to their likelihood of being failure, i.c., their failure- revealing capabilities. Different from existing solutions, TestRank leverages both intrinsic and contextual attributes of the unlabeled test data when prioritizing them. To be specific, we first build similarity graph on both unlabeled test samples and labeled samples (e.g., training previously labeled test samples), Then, we conduct graph-based semi-supervised learning to extract contextual features from the correctness of similar labeled samples. For particular test instance, the contextual features extracted with the graph neural network and the intrinsic features obtained with the DL model itself are combined to predict its failure-revealle revealing capability. Finally, TestRank prioritizes unlabeled test inputs descending order of the above probability value. We evaluate TestRank on three popular image classification datasets, and results show that TestRank significantly outperforms existing test input prioritization techniques. Our code is available https://gitub..com/curc la---etttakkk
Introduction
Deep learning (DL) systems are prone to errors due to many factors, such as the biased train- ing/validation dataset, the limitations of the model architecture, and the constraints training cost. is essential to conduct high-quality testing before DL models are deployed in the field; otherwise, the behaviors of DL models be unpredictable and result in severe accidents after deployment. However, the cost of building test oracles (i.c., the ground-truie -tuth output) by manually labeling a massive set of test instances is prohibitive, especially for tasks requiring experts for accurate labeling, such as medical images and malware executables.
To tackle the above problem, various test input prioritization techniques (Feng et al.) 2020 Byun cet al. 2019 Shen et al. 2020) are proposed to identify "high-quality' test instances from a large amount of un unlabeled data, which facilitates revealing more failures (e.g., misclassification) of the DL model with reasonable labeling effort. These methods try to derive the failure-revealing capability of test instance with its intrinsic attributes extracted from the responses of the model under test (c.g.. the softmax-bas based probabilitics given by the target DL model this specific input). DeepGini [Feng scores based on the model's output probabilities to rank the unlabeled test cases. Test cases with nearly equal probabilities on all output classes regarded as less confident ones and are likely reveal
al. [2020] feeds the unlabeled data to the target DL model and calculates confidence-relatee
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

model failures. Similarly, [Byun et al. 2019 use the uncertainty score obtained from MC- MC-Dropout for test imput prioritization. Muliple-boundary clustering and prioritization (MCP) [Shen al. 2020) considers both the output probabilities and the balance among each classification boundary when selecting test cases. All existing works try to identify instances near the decision boundary and prioritize them. However, we argue that near- harr-boundary instances are not necessarily failures, especially for well-trained classifiers with high accuracy. Also, as failures can be far from the decision boundary, existing methods could fail to reveal these remote failures.
To estimate test instance's s capability in revealing failures, in addition to the intrinsic attributes mentioned above, there is another type of information: the known classification correctness of labeled samples (i.e.. training samples and previously tested samples) and their relationship to the unlabeled instance. This information provides extra insight into the target model's behavior, Such data is already known, and it provides contextual information that reflects the corresponding inference behaviors of the target model for set of similar instances.
This work presents a novel test input prioritization technique, namely TestRank, for DL classifiers. TestRank exploits both intrinsic and contextual attributes of test instances to evaluate their failure- revealing capabilities. Based on the intuition that similar inputs are usually associated with the same classification results, we propose to use graph neural networks (GNNs) Kipp and Welling 2017] to summarize the neighboring classification correctness for each unlabeled instance into contextual attributes. GNNs have been well-studied and valued for their relational inductive bias for extracting graph information. Our method, TestRank, constructs a similarity graph on both unlabeled and labeled instances and apply the semi-supervised GNN learning to extract the contextual attributes. After that, we aggregate intrinsic (such attributes are extracted from the input samples without considering their neighbors) and contextual attributes with a neural-networr- binary classifier for test input prioritization.
The contributions of our work are as follows:
To the best of our knowledge, TestRank is the first work that takes the contextual information from the target DL model into consideration for test input prioritization.
We propose constructing a similarity graph on both labeled and unlabeled samples and training a graph neural network to extract useful contextual attributes from the contextual information for these unlabeled instances. We also present approximation techniques to reduce its computational complexity with minor impact on the performance of TestRank.
We propose simple yet effective neural network that combines the intrinsic attributes and contextual attributes of unlabeled test instances for their failure-revealing capability estimation.
We empirically evaluate TestRank on three popular image classification benchmarks: CIFAR- 10. SVHN, and STL10. The results show that our method outperforms the state-of-the-arr methods by a considerable margin.
2 Test Input Prioritization
Let us use f : X + y to represent the given target DL model, where X and y are the input and output space, respectively. For effective testing") the debugging center needs to perform test input prioritization, i.., select certain number of test instances from the large unlabeled test instance pool that can reveal as many failures as possible. Later, these failures are fed back to the training center for failure analysis and model repair. We define the model failures as follows:
Definition 1. DL Model Failure. A failure of the DL model can be uncovered by the test instance x if the predicted label f(x) is inconsistent with its ground truth label yxo namely f(x) # yx-
Formally, the debugging center selects and labels b test cases Xs (Xxs b) from the unlabeled test instance pool Xy- The objective of test input prioritization is to maximize the detected failures:
max [[x] fx) # yxll, where x E X5 and [x| =
Please note that, we focus on testing the functional correctness of the DL model, and we assume the collected testing data are clean samples instead of maliciously generated ones.

---

Different solutions are proposed to quantify the failure-revealing capability of unlabeled instances. DeepGini [Feng et al. 2020 proposes evaluate single test instance via the DL model's 's final statistical output: f(1) = = where Pr. is the predicted probability that the test case belongs to the class Given the sum of P.a 1, impurity function f(() is maximal when all Pri values equal. DeepGini also adopts the neuron coverage criteria proposed in DeepXplore [Pei et al.) 2017] and DeepGauge [Ma et al. 2018) in test input prioritization, and the result shows that the impurity-based selection is much better than coverage-based selection.
Instead of evaluating the overall likelihood of failure for all classes, Multiple-Boundary Clustering and Prioritization (MCP) proposes to evaluate it for each pair of classes individually [Shen et al. 2020]. In this way, test instances can be evenly selected for each class pair and the failure cases are investigated at the finer granularity. Besides these metrics, [Byun et al. 2019] also propose to measure the likelihood of incorrect prediction by the uncertainty of the model's output, which reflects the degree to which model is uncertain about its prediction. In practice, evaluating uncertainty requires the task DL model to be a Bayesian Neural Network [Richard and Lippmann 1991 Neal 2012] or containing dropout layer for approximation [Gal and Ghahramani 2016].
Besides examining the DL model's final outputs, [Kim al. 2019 proposes two surprise adequacy (SA) criteria that make use of the target DL's internal outputs (e.g., the activation traces). They are Likelihood- based Surprise Adequacy Coverage (LSA) and Distance-basee Surprise Adequacy Coverage (DSA). LSA and DSA measure the likelihood or distance of an unlabeled instance to the training instances, respectively. Test samples with higher SA values are preferred in testing.
To sum up, all existing methods use the target model's outputs to one input, i.e., its intrinsic attributes, for its failure-revealing capability estimation. In contrast, make use of both intrinsic and contextual attributes of an instance for better estimation (see later sections for details).
TestRank
3.1 Motivation
The failure-revcaling capability of an unla- beled test input is closely related to its at- tributes for the DL model under test. In this work, we distinguish for an unlabeled instance: the intrinsic at- tributes and the contextual attributes.
DL model boundary
Near-boundary failure
kinds of attributes
We define the intrinsic altributes of an in- put as the output responses assigned by the target DL model to this specific input. It could be, for example, the predictive out- put distribution of the input from the target DL model, reflecting the sentiment derived from the computation performed by the tar- get model [Byun al., [219]] This kind of attributes is adopted by existing test input prioritization approaches (Feng et al. 2020
Remote failure
Near-boundary failure
la la Unlabeled data which uncover model failures
Labeled data
A Unlaheled data in test set
Figure 1: motivational example.
Shen et al. 2020 Byun et al. 2019]. Note that we define such attributes 'intrinsic"
because they are extracted from inputs without considering their context, ie., the classification correctness of its similar instances.
In contrast with the intrinsic attributes, the contextual attributes provide a deeper insight into the target model for the unlabeled samples: the contextual attributes for an unlabeled sample summarize the classification correctness of similar and labeled samples. For a particular test instance, such contextual attributes useful and complementary to the intrinsic attributes.
An illustrative example is shown in Figure I wherein we visualize the behavior of a two-class classifier on the unlabeled test data and historically labeled data distribution. The blue region includes the instances that are near the decision boundary. Intuitively, the classifier is uncertain about the data when data is near the decision boundary and is likely to misclassify it. Existing
3

---

intrinsle attributes
Unlabeled Duta Pool X
DL Model under Test (f)
Labeling budget
Positive
Negalive Unlabeled
Similarity graph
em
Faihmr-eee Capabclin
Selected Test Inputs X₃
Labeled Data (X,, v,
Feature Extractor
Feature
Graph-based Semi-
cc
MLP
Rank
(ftee)
veciors
supervised Model
Extract contextual attributes
Figure 2: The overview of TestRank.
works [Shen et al. 2020 Feng et al. 2020 Byun et al. 2019 propose various indicators (e-g-. confalenee/nccerantty/uuupp
scores) help identify the near-boundary instances. However, the near- hear-boundary instances are not necessarily failures, and some of them can be correctly classified by well-trained classifier. What is worse, such testing approaches fail to capture the failures lying far from the decision boundary (i.e., remote failures, shown in the red region in Figure I. because DL models usually output high confidence (or low uncertainty) for these inputs. These failures may be caused by limited model capacity, insufficient training data, etc.
Our key insight is that we similar labeled samples) to help locate both near- boundryy and remote failures. The usefulness the contextual information is due to the local continuity property [Bishop [006]. which means that inputs close in the feature space share similar prediction behavior, e.g., classification results from the target model. As shown in Figure 1 some already labeled data, whose classification correctness is already known, surround the unlabeled data. If an unlabeled instance is close to already falsely classified data, under the local continuity property, it likely that this instance also a model failure. This property motivates us to extract the contextual attributes for an unlabeled instance from its neighboring labeled data. By combining the extracted contextual attributes with the intrinsic attributes, expect to achieve better failure-revealing capability estimation.
the contextual information (e.g. the classification correctness of
3.2 Overview
Figure 2 shows the overview of TestRank, which consists of two attribute extraction paths for the final faiTure-revealing capability estimation:
1. Path A: intrinsic attributes extraction. Given a pool unlabeled inputs X5, we use the target DL model f to extract the intrinsic attributes em for each input. More precisely, we collect the output logits (i.e., vectors before the so softaax layer) from the DL model as ew-
2. Path B: contextual attributes extraction. First, we use feature extractor to map the original data space into a more compact feature space with good local continuity property. Then, we create a similarity graph (i.e., k-Nearest Neighbor Graph) based on the obtained feature vectors and their corresponding classification correctness, if any, from both unlabeled data pool X and labeled data pool X (e.g. training set and previously labeled test samples). Last but not least, the graph-based representation learning technique is applied to extract the contextual attributes e, for each unlabeled instance. The details are elaborated Sec. 33.
After attributes c. and are extracted, combine them together via Multi-Layer Perceptron (MLP) (see Sec. for details). The MLP responsible for predicting the failure-revealin ability for unlabeled test instances. At last, these instances ranked according to their failure failure-revealing capability, and the top ones are selected under the given labeling budget.
As intrinsic attributes extraction is straightforward, we discuss the path B and how to combine path A and B in detail in the following subsections.
3.3 Contextual Attributes Extraction
We represent the contextual information from the DL model as a set of labeled inputs X, and the corresponding classification correctness Y, E (0, 1), where correctly classified inputs are labeled as 0 and misclassified ones are labeled as 1. Given the contextual information, our goal is to extract the contextual attributes for each unlabeled test instance x E X However, extracting contextual attributes from labeled and unlabeled data is a non-trivial task because of the following reasons.

---

Algorithm 1: GNN-based Contextual Attributes Extraction
Input: Input samples X5 U X2, Correctness of labeled samples Yz. Number of neighbors k, Feature extractor fers number of GNN layers M, number of training epochs.
Output: Contextual altributes E. for Xy
X
(No U X,) # Extract compact representation;
Edge kwm_grrph(X.k) # KNN Graph construction;
3 A = Edge Ix. D= E-=,AA, A,,
H =x:
for number training epocks do
for I 0.
H ④D $ AD + H'O',
8
end
loss CrossEntropyLoss(tt" Yz): Update O:
10
end
12 E. = H "[index X,] extract the representation from the M 1a GNN layer;
13 return E,;
First, it well-known that the real data, especially image data, usually locates in high- dimenssinnii space, wherein the underlying data distribution will live complex and non-linear manifold em- bedded within the high-dimensional space [Bishop 2006]. Therefore, constructing the relationships between different instances is difficult. To address the challenge, we adopt the representation learning process [Grill et al. 2020 van den Oord et al.) 2018 Chen et al. 2020 He al., 2020], which map the raw data into compact feature space with better local continuity property, such that inputs close in the feature space share similar classification results. Thus, in the feature space, the proximity between inputs can be measured with simple distance metrics (e.g., L2, cosine).
Second, manually designing protocols summarize the neighboring classification results is subject to the imperfection of local continuity. Namely, the result is easily affected by the noisy data in the neighboring space. To solve this challenge, we construct similarity graph based on the labeled and unlabeled data, and then apply the more powerful graph learning technique graph neural networks (GNN) for contextual attribute extraction.
The GNN empowers graph embedding learning, as it employs a learnable aggregation and transform procedure (Kipf and Welling 2017], which exploits the relational inductive-bias that exhibits in the graph structure. It generates a umnbeddiingrreeresenttiin that summarizes the "contextual information" for each input sample, making easier separate the correct and misclassified The attributes extracted by the graph neural network can then be combined with the intrinsic attributes to conduct the better Gailure-revealing capability estimation (See Sec. 3.4). The contextual attributes extraction process is formally depicted by Algorithm
The contextual
Feature Vector Representation (Line 1). As the target model is be tested, its feature extraction quality is not guaranteed. Out of this concern, and to make full use the labeled and unlabeled data, choose to out-of-shelff unsupervised model for feature space construction.
Among the unsupervised learning techniques, the BYOL (Grill et al. 2020) explicitly introduces local continuity constraint into the learned feature space and shows good results on various downstream tasks. Therefore, we train a BYOL model fer to extract the features from the raw input images. The data used to train the BYOL model includes both labeled and unlabeled data: (Xv U X,)), and the resulting feature matrix is denoted as X. Please note that the feature extractor can be replaced by any other well-trained feature extractor with the local continuity property (e.g., SimCLR [Chen al.) 2020) and MoCo He et al., 2020)).
Similarity Graph Construction and Approximation (Line 2). After the extraction of feature representation, we use the simple distance metric (i.c., cosine) to measure the similarity between any two test instance and in X: Dist(i, j) = cosine(x; x,). Based on the distance matrix Dist, we construct N-NN Graph G. wherein each sample connected its top-k most similar samples. The connection represented by an adjacency matrix RAXY where N is the number of sample in X. The entry A. equals 1 if node j is within the & nearest neighbors of node and 0 otherwise. The
5

---

edge weight matrix of the similarity graph is denoted as Edge, wherein each edge weight in Edge, if exists, is inversely proportional to the corresponding distance Dist:
Edge; 1/Dist(i..) J)
Aij = Aij =
Lis (O.....
This means that the connection between two nodes, exists, is weaker if their proximity is large.
Constructing such a k-NN graph is, however, computationally expensive. This is because, calculating the distance between each pair of test instances requires computational complexity of O(N2), which is prohibitive to scale up to the current massive unlabeled test instances in real applications. Therefore, we propose an approximation method for k-NN graph construction. Our intuition that, since the target of graph construction is to exploit the failure patterns of the nearby labeled instances for the unlabeled instances, the connections between unlabeled data are less meaningful. Therefore, we propose to only consider the connections among labeled data X,, and the connections between labeled X to unlabeled data Xy. This approximation reduces the cost from O(N²) to O(P² + PQ), where and Q stand for the number of data in X and Xv, respectively. Usually, in the real-world scenario, much smaller than Q, thereby we could obtain a near-linear graph construction algorithm with complexity O(PQ).
GNN-based representation Learning (Line 3-12). To apply the GNN algorithm, we first initialize the input node representation matrix H in the similarity graph as X. Recall that in each GNN layer, the node representations are propagated between neighbors and aggregated together. Thus, we can obtain the representation the next GNN layer by:
H = aD -D-!AA AD H'O')
(3)
where Á = Edge IN. IN is the identity matrix, D = Σ, A.j. O' is the trainable weight matrix for the 1th layer, is activation function and H is the output representation matrix. The propagation and aggregation are repeated for M layers, with the output dimension of the M, layer is (for binary classification purpose).
Then, for any labeled node X,,, we could obtain a cross entropy loss between the GNN output M and the expected label y E Yz (e.g. misclassified or not): Lee = =-ylog((h*) (1 y)gg(( - )))), )) where hM denotes probability that X, is misclassified. The model is trained via minimizing the loss for some training epochs (we set it as 600 in our experiment). After that, we apply the trained GNN model (except the last layer) on Xy to obtain the E, (line 12). In this way, the correctness of the neighboring samples could be effectively summarized for each node.
3.4 Failure-revealing Capability Estimation
To properly combine both the intrinsic attributes em and contextual attributes for collaborative failure-revealing capability estimation, we formulate the combination function as a simple binary classifier (e.g. ML.).. Specifically, the input to the MLP is concatenation of em and ee, and the output the failure-revealing estimation for an test instance. The final failure-revealing probability is produced by applying a sigmoid function (r) labeled instances (X,, Yr.) to train the MLP in a supervised manner, with an objective of minimizing the binary Cross-Entrope Entropy loss. After training, the MLP shall re- weight the importance of intrinsic and contextual attributes and make a final decision by assigning a high probability to a test instance it is likely to reveal failure.
I+ on the MLP model's output. We use the
Finally, we rank the unlabeled test instances in a descending order based on their failure-revealing capability. Under the given budget, we select the top ones to label and test.
4 Experiment
4.1 Setup
Datasets. We evaluate the performance of TestRank on three popular image classification datasets: CIFAR-10 (Krizhevsky et al. 2009]. SVHN [Netzer et al., 2011]. and STL.10 (Coates et al. 2011] as shown in Table] I More elaboration is shown in the Appendix.

---

#Casss
Ollicial Train/ Test/rra Split SpINKTC/DCHHD)
Architocture
sci( (A/B/C)
CIFAR-ID
10
60K
50K/10K/X
20K/39K/IK SUKA9K/SSKK 4K775K0055KK
ResNet-18
70.1/66.4/68.3
SVHN
630K
73K/26K/S3IK
Wide-ResNet ResNet-34
14.2/92.5/816
STL.0
5K/8K/x
54/8/44153.6
Table 1: The Dataset and DL Models.
There are mainly two parties involved in the model construction process: the training center and the debugging center. Hence, we manually split the dataset into the training center dataset (see the TC column in Table [ and the debugging center dataset (see the DC column in Table [ To mimic the practical scenario where the unlabeled data is abundant, we move a portion of training data to the debugging center to create this scenario. In the debugging center, we let a set of test data as labeled ones to represent the historical test oracles, and they are 8K/10K/1.5 (OKK/ for CIFAR-10 10, SVHN, and STL10, respectively, which are used to train the GNN and MLP model. The rest of the data in the debugging center are left unlabeled. Also, we spare a hold-out dataset (see the I1O column), which is used for evaluating the model accuracy.
Target DL model (model under test). As shown in Table we use the popular ResNet and WideResNet architectures as the backbonn models (He et al. 2016 Zagoruyko and Komodakis 2016]. To simulate models of different qualities, for each dataset, we train three DL models with different randomly drawn sub-sets from the training set owned by the training center. For model B and C, the training set are drawn with in-equivalent class weights. After training, we report the accuracy of models on the debugging center's hold-out dataset To in Table
Evaluation metric. We propose new evaluation metric for test input prioritization techniques: Test Relative Coverage (TRC). TRC defined as the number of detected failures divided by the number of budget or the number of total failures identified by the whole unlabeled test set, whichever is minimal:
#Detected Failures
TRC- min(# Budget otal Failures)
(4)
When # budget S # total failures, the maximum number of failures can be identified equals to the budget. When # budget 2 # total failures, the maximum number of failures can be detected equals to the total number of failures. Therefore, TRC measures how far test input prioritization technique to the ideal case.
In practice, under the massive unlabeled data, the performance under small budget is considered more important than that under a large budget. To provide an insight on the quality of one test input prioritization technique under a small budget, we also provide an ATRC metric: ATRC measures the average TRC values for budget values less than the total failures:
1 N-1 ATRC
= Σ TRC, N
(5)
where TRC, stands for the TRC value under budget b,, b, # bj. and b, S number of total failures.
The proposed metrics enhance the ones used by Feng et. al. (Feng et al. 2020 and Byun et. al. [Byun et al. 2019). They use the percentage of detected failures against the percentage of budget (and an APFD [Do and Rothermel] 2006] value derived based on it) for evaluation. Their metric would produce a small value under a small budget, regardless of how good the prioritization technique is. For example, let's assume that there are 10,000 unlabeled data, and 2,000 of them can detect model failures. If the budget 100, the best percentage of detected failures is 5%, and the worst is 0%. Thus, under their metric, the gap between the best and the worst is only 5%. By contrast, TRC enlarges this gap to 100% to better differentiate the ability of different test input prioritization techniques.
4.2 Comparison of TestRank with Baselines
We evaluate TestRank against five representative baselines: Random, DeepGini (Feng et al. 2020] (the state-of-the-ard MCP [Shen et al. 2020), DSA (Kim et al., 2019], and Dropout-uncertaie [Byun et al., 2019]. The details of each baseline are illustrated in the Appendix. For the dropout uncertainty method, we run 1000 times inferences with a default dropout rate of 0.5 (the dropout rate is consistent with the one used in [yynn et al. 2019). ). For the DSA method, collect the activation
7

---

TestRank Contextual-Omly
Dataset
Model ID
Random
MCP
DSA
Uncertainty
DecpGini
TestRank
A
30.15
58.25
60,93
58,09
67.47
51.39
76.56
CIFAR-10
B
34.18
46.46
62.34
61.85
67.80
58.85
87.87
34.27
65.25
64.47
63.10
71.15
75.33
85.53
A
10.16
39.98
55.47
58.29
63.47
44.16
66.06
SVHN
B
11.85
38.07
57.96
58,06
63.85
51.26
76.36
C
23.41
65.33
69.34
71.80
81.68
93.99
95.32
A
39.25
66.62
64,56
64.30
69.70
60.09
79.00
STLIO
42.60
69.97
67.12
65.30
72.89
71.90
80.96
70.34
79.55
88.67
Table 2: Comparison of TestRank with baseline methods with ATRC values (%).
100
100
100
90
80
00
Our
Our
40
DettCon
60
50
D5A
Uncertantty
Uncartarit
uncersarry
Random
Sandom
Sandom
60
80
100
40
80
100
60
100
(0)
CIEAR-10
(b)
SVIN
(c)
STL10
Figure 3: The TRC values against all budgets. X-axis: the budget (%). Y-axis: the TRC value. Note: this figure is generated with model B on each dataset.
traces of the final convolution layer to calculate the surprise score. For our method, we set the number of neighbors for constructing the KNN graph as 100. Also, we use a two-layer GNN with a hidden dimension of 32. More ablation studies are in Section 4.3
Table S compares TestRank with baselines using the ATRC metric. From this table, we have several observations. First, compared with the baselines. TestRank can achieve the highest ATRC values on all evaluated datasets and models. For instance, on CIFAR-10, TestRank can achieve 9.09%, 20.07%, 14.38% higher ATRC values than the best baseline DeepGini for model A, B, C, respectively, Therefore, our method can distinguish the failure-revealing capability of the unlabeled test inputs much more accurately. Second, the Coaounk-Cotexttal-OOO0
column shows the result using only the contextual attributes. We observe that the contextual attributes alone can achieve higher effectiveness than random prioritization. For example, for model on CIFAR- 10, the effectiveness of random prioritization is 30.15% while that of the context-only method is 51.39%. We manually check the distribution of failures of model C and find that many failures are centralized on two classes, where the training data is insufficient. This kind of failure is casily detected by the contextual attributes -base method. Hence, the contextual information not sufficient. The combination of intrinsic and contextual attributes is essential in achieving high accuracy failure-revealing capability estimation.
helpful. But still, the context attributes alone are
To show more detailed results, we present the TRC value against every labeling budget in Figure 3 We observe that the TRC values for most curves decrease in the beginning and then increase. The turning point is when # budget = # total failures. When # budget < # total failures, the TRC values decrease because we rank the test instances according to their cellure-revealing probabilitics. With the budget increases, the selected test cases have lower average failure- ability, thus the decreased TRC value. When budget > # total failures, according to the definition TRC (see Equation 4 the denominator is fixed. The increase in budget will improve the number of detected failures, and hence the TRC value will increase.
Figure 3 shows that our method consistently outperforms baselines, especially when the budget is small. For example, Figure 3 (a). TestRank improves the prioritization efficiency by around 20% compared to the best baseline when the budget is around %. When the budget is rather high (e.g. budget 80%), the difference between different methods less obvious because most failures can be selected under the large budget.
8

---

1000
Dataset
Model
TeeRank
(96
TesRank wio approx. (%)
TargesModel(
TestRank
(1221
(-7.71)
ATPP values (96)
80-
CIFAR-TO
B
87.87
87.70 (-0.17) 17)
81 46.46 (-6.40)
85.53
88.1D (42.57)
71.73 (-7.79)
SVHN
76.36
82.01 (+5.68)
95.32
96.62 (+1)30)
79.00
Moddl Model
STI.10
80.96
78. (-1 1.8))
14.43 (-6.52)
88.67
89.32 (+0.65)
72.43 (-16.23)
Model
Influence (%)
+0.95
-6.23
Number Neighbors
Table 3: The performance (ATRC values) of Tes- (Rank under different configurations.
Figure 4: The impact of the number of neighbors k (STL10 dataset).
4.3 Impact of TestRank Configurations
Feature Extractor. TestRank uses an unsupervised BYOL model trained on both labeled and unlabeled data to extract their features. One may wonder if can be replaced by supervised model (e.g,, the target DL model). To investigate this, we replace the feature extractor in TestRank with the front layers (we remove the last few linear layers) of the target DL model. The result is shown in the TestRank- TargetModel column in Table Comparing with the original TestRank, the average ATRC value on the reported datasets and models reduces by 6.23%, which is significant. As the quality of the given model is to be examined, its feature extraction performance may not be reliable. Also, the dimension of the hidden layer could huge, making it computationally expensive to calculate similarity values between input samples (This is why we do not report results on the SVHN dataset). In contrast, the separate model is more controllable, enabling extract better features for these data. Hence, using a separate feature extractor is necessary.
k-NN graph approximation. To reduce the computation complexity. TestRank uses approximation techniques when constructing the k-NN graph (see Section 3.3). The Tes/Raan-wwjo-approx column in Jable]showw, the result when we use the original k-NN graph without approximation. It indicates that the average influence of the approximation is small (c.g. 0.95%). Therefore, if there is a computation resource limit and the unlabeled test instances are massive, we recommend using the approximation to save computation with negligible performance loss greatly.
Number of nearest neighbors k. When constructing the k-NN graph, the number of neighbors k decides the range of the context one node can reach. In previous experiments, the is set to 100. We enlarge this range to (20 800) study the influence. The result is shown in Figure 4.3 One can observe that the prioritization effectiveness will decrease when & is too small or too large. When too small, the context information available one instance is limited, making it difficult for the GNN to extract valuable contextual attributes. On the other hand, when k is too large, the GNN may grasp irrelevant/noisy information. Still, TestRank can achieve good performance k values. For example, for model A, TestRank is better than the best bascline 69.70% (see Table when & larger than 20. Hence, selecting the number of nearest neighbors & is relatively flexible.
wide range of
Conclusion
We propose TestRank, a novel test input prioritization framework for DL models. To estimate test instance's is failure-revealing capability, TestRank only leverages the intrinsic attributes of an input instance obtained from the target DL model, but also extracts the contextual attributes from the DL model's historical inputs and responses. Our empirical results show that TestRank outperforms existing solutions significantly. At the same time, this paper considers each test case equally and aims identify as many raiuure-rveaaling test cases as possible. In practice, the impact of each test case could be different. We leave the study of such impact for future work.
Acknowledgment
This work is supported in part by General Research Fund (GRF) of Hong Kong Research Grants Council (RGC) under Grant No. 14205018 and No. 14205420.

---

References
Christopher M Bishop. Pattern recognition and machine learning. springer, 2006.
Taejoon Byun, Vaibhav Sharma, Abhishek Vijayakumar, Sanjai Rayadurgam, and Darren D. Cofer. Input prioritization for testing neural networks. In IEEE International Conference On Artificial Intelligence Testing, AlTest 2019, Newark, CA, USA, April 4-9, 2019, pages 63-70. IEEE, 2019.
doi: 10.1 Att.0////.t........0000
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations, In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 1597-1607. PMLR, 2020.
Adam Coates, Andrew Y. Ng, and Honglak Lee. An analysis of single-layer networks in unsuper- vised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Stafistics, AISTATS 2011, Fort Lauderdale, USA, April 11-13, 2011, volume 15 of JMLR Proceedings, pages 215-223. JMLR.org. 2011.
Hyunsook Do and Gregg Rothermel. On the use of mutation faults in empirical assessments of test case prioritization techniques. IEEE Trans. Software Eng., 32(9):733-752, 2006. doi: 10.1109/TSE.200..99
Yang Feng, Qingkai Shi, Xinyu Gao, Jun Wan, Chunrong Fang, and Zhenyu Chen. Deepgini: prioritizing massive tests to enhance the robustness of deep neural networks. In ISSTA '20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, USA, July 18-22, 2020, pages 177-188. ACM, 2020. doi: 10.1145/399536333397555
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. CoRR, 0ts//9933.2225 2019.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty deep learning. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pages 1050-1059. JMLR.org, 2016
Jean-Base Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent new approach self-supervised learning. In Advances Neural Information Processing Systems, volume 33, pages 21271-21288 212284, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegus, NV, USA, June 27-30, 2016, pages 770-778. IEEE Computer Society, 2016. doi: N0.1109/CVPR.22166.39
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEEJCVF Conference Computer Vision and Pattern Recognition, pages 9729-9738, 2020.
Jinhan Kim, Robert Feldt, and Shin Yoo. Guiding deep learning system testing using surprise adequacy. In Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019, pages 1039-1049, IEEE ACM, 2019. doi: 10. 10910//............// 208800018...
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
10

---

L. Ma, Felix Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, C. Chen, Ting Su, L. Li, Y. Liu, Jianjun Zhao, and Yadong Wang. Deepgauge: Multi- grauularanllritt testing criteria for deep learning systems. 2018 33rd IEEF/ACM International Conference Automated Software Engineering (ASE), pages 120-131, 2018.
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703, 2019.
Kexin Pei, Yinzhi Cao, Junfeng Yang. and Suman Jana. DeepXplore: Automated Whitebox Testing of Deep Learning Systems. Proceedings of the 26th Symposium on Operating Systems Principles (SOSP), 2017. doi: (..114/333324473313222
Michael D. Richard and Richard P. Lippmann. Neural network classifiers estimate bayesian posteriori probabilities. Neural Comput., 3(4):461-483 1991. doi: (0.1162/neco.1991.3
Weijun Shen, Yanhui Li, Lin Chen, Yuanlei Han, Yuming Zhou, and Baowen Xu. Multiple-boundary clustering and prioritization promote neural network retraining. In 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020, Melbourne, Australia, September 21-25, 2020, pages 410-422. IEEE, 2020. doi: 10.1 10.1145533248843441666
Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. CoRR, abs/ 807.007.00 00748, 2018.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19-22, 2016, BMVA Press, 2016.

---

A Appendix
A.1
Experiment details
Our code implemented with the open-source Py PyTorch (under BSD license) Paszke et al., 2019) and PyG (under MIT license) [Fey and Lenssen 2019) ML. libraries. All of experiments performed on single TITAN V P
A.1.1 Datasets
CIFAR-10 is officially composed of 50,000 training images and 10,000 test images, and it has ten classes natural images. The Street View House Numbers (SVHN) dataset contains house numbers from Google Street View images. It contains 73,257 training images and 26,032 testing images. Besides, the SVHN dataset also has an extra set of 531,131 images. The STL10 dataset contains ten classes of natural images. In each class, there 500 training images and 800 test images.
Please note that the datasets we used are open-sourced and available for research purposes Also, since the data we used is about numbers and the animals, we assume there are personally identifiable information or offensive content.
A.1.2 Baselines
Given a DL model and specific budget, the goal of our method is to select test cases from an unlabeled data pool to discover the failures of the given DL model. We compare our work with the following test input prioritization techniques:
DeepGini [eng et al. 2020] DeepGini is the state-of-art test case selection technique.
DeepGini ranks unla unlabeled test cases by a score defined based on the output confidence.
MCP [Shen al., 2020]: In addition the output confidence, MCP also considers the balance among different class boundaries of the selected test inputs. Specifically, MCP groups test cases into different clusters, where each cluster stands for distinct classification boundary, and equally choose low confidence test cases from each cluster.
DSA [Byun et al., 2019] Byun el. al. propose to use the distance- based surprise score (DSA) as a test input prioritization metric, which was initially proposed in [Kim et al.) 2019). The surprise score measures the distance between the test case to the training set. Samples with higher surprise scores are prioritized.
Uncertainty [Byun et al., 2019]: The uncertainty is calculated as the entropy on the averaged output probabilities by running the model multiple times (e.g. times) with specific dropout rate.
Random:
Test inputs are randomly drawn from all unlabeled samples.
A.2 Evaluation More Model Structures
We evaluate our TestRank on two extra architectures: ShuffleNet and MobileNet. The results are shown in Table 4 The model ID indicates the same meaning (e.g., model training with different parts of the training set) as our main paper (see Section 4.1-Target DL model). As we can observe, TestRank generalize to other models and architectures.
A.3 Limitation
Run-time overhead. We compare TestRank with baseline methods terms of the run-time overhead in Table However, despite the considerable performance improvement in test input prioritization, TestRank introduces longer run-time than most baselines. The main reason is due to the calculation of the distances between input features for k-NN graph construction. This limitation hinders the application of TestRank in large datasets, and we will solve our future work.
htttp://www.nvidi.. www.eeoooooottttaaaaeeee
hhat::/ufidl stanford. edd/houseeumbers// "hupss//cs.stanford.co stanford.edu/ acoates/stl 10/ "htps://www. coronto.edul kriz/cifar.html
12

---

Dalasel
Architecture
ID
Accuracy
Random
MCP
DSA
Uncertainly
DeepGini
Ours
19.13
$7.16
$6.69
64.33
66.76
ShuffleNet
79.1
23.02
56.82
59.03
37.55
65.95
76.15
C
75.6
24.93
60.23
62.54
51.23
69.64
79.55
CIFAR-10
A
79.5
21.94
58.08
36.99
53.86
64,83
66.85
MobieNet
73.5
26.21
56.28
62.45
49.92
64.72
80.71
72.9
28.08
62.61
67.18
68.11
73.09
80.84
6.11
53.58
$2.32
5.12
59.18
ShufileNet
96.56
6.97
54.76
55.95
5.84
59.74
64.32
95.68
8.55
61.90
58.34
7.01
68.62
78.03
SVHN
A
96.55
7.08
53.94
55.87
12.98
62.55
61.91
MobieNet
95.46
8.93
55.43
52.57
14.32
62.44
68.63
58.65
71.50
15.24
81.60
25.9
56.86
57.42
67.42
ShufileNet
69.8
27.15
59.50
54.68
62.87
64.5
68.71
63.4
32.63
59.19
58.02
66.13
66.3
84.40
STL10
62.8
32.28
58.40
58.65
66.69
67.6
72.22
MohieNet
60.8
35.71
61.48
67.62
67.75
66.52
72.73
38.7
64.16
70.59
Table 4: Additional comparison of TestRank with baseline methods ATRC values (%).
Dutaset
Random
MCP
DSA
Uncertainty
DeepGini
Ours
CIFARIO
779
103
747
STLIO
9
627
108
142
SVHN
16
33
7931
1413
18
1020
Table 5: Run-time overhead comparison of TestRank with baseline methods (s).
Failure diversity. Besides the number of detected failures, failure diversity is another important factor for model debugging. this work, we have the implicit assumption that the historical labeled data can cover the input distribution, and under such circumstances, TestRank can prioritize unlabeled tests effectively. TestRank, we should analyze the test pool and try to fill this gap first. Otherwise, the detected failures lack diversity. We shall consider this problem in our future research.
however, the historical test data severely biased, before prioritizing tests with
A.4 Broader Impacts
This work targets building an efficient and effective test input prioritization technique for deep learning models, which can help ensure the security of deep learning models after deployment. Various safety-critical tasks, such as autonomous vehicles, industrial robotics, and medical diagnosis, can benefit from test input prioritization techniques. We consider the scenario in which the unlabeled data is abundant, and subset of unlabeled data is selected for labeling and testing. While building the unlabeled data by collecting them from the Internet or other sources, may have chance to include some unauthorized private data. Also, the collected unlabeled data could be biased, resulting in the testing being incomplete. To avoid such cases, we should always guarantee that the collected data will not violate any privacy or rights and ensure that the collected data cover instances many as possible.
13


Reviews:
[{"review_id": "x85QyoENCW6", "reviewer": "Reviewer_7iHb", "summary": "- Given a dataset of labeled and unlabeled examples, authors build a\n  data point prioritization technique, which helps choose data points\n  to be labeled next, with a goal of revealing failures as soon as\n  possible. The technique should help prioritize labelling instances\n  which currently trained model would misclassify.\n- They use intrinsic and contextual attributes to build a model\n  performing data point prioritization. Intrinsic attributes are those\n  that are obtained by observing outputs of the deep learning model\n  for a given input. Contextual capture information about similarity\n  to the other instances -- if a given instance is similar to\n  misclassified instances, that would be a strong signal that the\n  instance is problematic.\n- For intrinsic attributes, they use the output layer logits (before\n  the softmax activation) of the DL model to construct a vector to be\n  used for a final model. Second, they use a graph neural network to\n  create contextual representation vector out of both labeled and\n  unlabeled data. The aforementioned vectors are input to an\n  multi-layer perceptron (MLP) model. The output of the MLP is a\n  probability that a given input would be misclassified by the DL\n  model.\n- They evaluate their model against other data point prioritization\n  techniques measuring number of failure revealing instances for a\n  given labelling budget.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- I find the problem being solved an important one as manual labelling\n  costs can become prohibitive. The problem being solved should help\n  maximize the gains from manual labelling efforts providing a better\n  quality model with less data overall.\n  \n- Considering that the overall goal is to maximize the quality of the\n  data we label and generality of the model we train using that data,\n  I wonder if data point prioritization techniques should be run on a\n  single model or multiple models trained. We only know that the\n  instances prioritized are troublesome for the current model, but are\n  they really special? What if we trained another model with the same\n  data we have, would we end up prioritizing different inputs?\n\n- I would like to see evaluation on more models. More precisely, I\n  would like to see for a single dataset having couple of model\n  architectures, and for each architecture K models trained. Then,\n  evaluate data point prioritization techniques on K models for each\n  architecture. This is to gain more confidence that the performance\n  gains generalize across models and architectures.\n\n- I find it slightly difficult to read the paper. For example, on line\n  256 it is written: \"8K/10K/1.5K\", but it is not written what the\n  splits are for. Then, the following sentence sounds incomplete:\n  \"when ..., the maximum number of failures can be identified by a\n  test prioritization technique equals to the budget\". Also, following\n  one: \"which exploits the relational inductive-bias that exhibits in\n  the graph structure\". Also, following sentence: \"we assume that\n  there are 10,000 unlabeled data, and 2,000 of them can detect model\n  failures\", should probably start with: \"let's assume\".\n\n- I see many mentions of \"test prioritization\" in the\n  paper. Personally, I would find \"data point prioritization\", or\n  \"instance prioritization\" a more natural name than the \"test\n  prioritization\". The reason is that in software engineering, in the\n  case of test case prioritization, there is already an oracle\n  predefined for the tests being prioritized, while in this case we\n  are about to create the oracle (label a prioritized instance). There\n  are also mentions of \"test instance prioritization\" and \"test input\n  prioritization\" which sound more natural. What might help paper read\n  better is to use the naming consistently.\n\n- Figure 3. It's hard to distinguish colors of different approaches in\n  the legend.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Regarding the limitations, authors recognize that not each data\ninstance prioritized is of the same importance, which they did not\ntake into consideration. I think another important point to mention is\nthat while the technique learns to prioritize instances for a single\nmodel, it is unclear if these instances are in any way special, or\nthey are just specific to the model trained. This is important if our\ngoal is to produce the best quality dataset. Alternatively, if the\nidea is to keep retraining the model with the additional data it would\nbe good to evaluate how model benefits when augmented with newly\nlabeled data points.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "h7HY34xmTel", "reviewer": "Reviewer_d2bi", "summary": "This paper proposes a new test sample prioritization strategy to test deep learning systems. The objective is to maximize the number of failure-inducing samples selected from unlabeled samples. The authors propose to leverage both the samples' intrinsic properties (from the model's prediction behavior on them) and contextual properties (from other labeled samples close to them) as the criteria to select samples. By learning another binary classifier that takes the two properties as input, TestRank produces probabilities to rank the test samples. Empirical results demonstrate that TestRank outperforms existing test prioritization techniques.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "####Stregths:\n- Clear description of the technique.\n- Using contextual features generated by KNN+GNN seems novel.\n\n####Weakness:\n- Lack of justification on why samples close to incorrectly-labeled samples should be prioritized.\n- Unclear why GNN is needed to extract contextual property -- how it compares to simple alternatives.\n\n####Comments:\n\nOverall, the paper is well-written. The motivation and the methodology are clearly described and very easy to understand. Having said that, my main concern is that I am not convinced by the criterion that the authors choose to prioritize test samples. I understand the authors' motivation is that test samples close to known error-inducing samples should be prioritized, as these samples are more likely \"failure-revealing .\" However, this assumption raises two questions:\n\nFirst, if failure-revealing is the only criterion, numerous adversarial testing papers have been shown very effective in constructing error-inducing samples. The generated samples can also be far away from the decision boundary. Besides, the generated samples can be regularized to match real-world image distribution (using GANs or hard constraints). Given that adversarial testing can generate an unbounded number of real-world failure-revealing test samples, I do not see the clear use-case or unique advantage of test prioritization using only existing (limited number of) samples.\n\nSecond, I think the ultimate goal of testing is to reveal model violations of some abstract properties. For example, for computer vision models, we want to test whether the model is robust to a certain class of transformation (lighting conditions), labels, background, object poses, etc. Therefore, the testing should uncover diverse samples that can reveal different property violations instead of keeping generating samples violating the same properties. The idea is similar to traditional software testing, where we are interested in constructing tests that trigger different code paths. So why prioritize those samples that we already know the model is highly likely to fail?\n\nAs of methodology, I do not find the justification for using GNN (in Section 3.3) convincing. I understand that measuring the proximity of images is challenging, but why not directly leverage the target model, e.g., using its last convolution layer (often used to produce image representations), to generate the representation and compute the distance? Since the contextual property identifies the close samples from the target model's perspective, it seems natural to use its internal representation of the inputs to compute distances. This also saves the scalability issue of GNNs when working on a large number of nodes.\n\n###Minor:\n\nSometimes \"TestRank\" is named as \"TextRank\". Are they indicating the same thing?\n\nDoes training the binary classifier (for failure-revealing) also update the weight parameters of the GNN? It is interesting to study which strategy works better - updating GNNs or keeping them frozen.\n\nApproximate nearest neighbor (ANN) might be a very good alternative to KNN as the authors are approximating the computation anyway (ignoring unlabeled samples). ANN is much more efficient than KNN.\n\n\"et. al.\"->\"et al.\" ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "F22OGOujoBx", "reviewer": "Reviewer_i1xT", "summary": "This paper introduce a way to test the generalization performance of DNNs using labeled together with unlabeld datasets via less labeling efforts. Authors proposed  TestRank -- an integrated approach that \n(1) adopts an semi-supervised learning algorithm to \"label\" every unlabeled sample using a vector of discriminative features using GNN and label data.\n(2) retrieves features of every unlabeled sample from the DNN model under testing, and\n(3) trains an MLP using both GNN-predicted features and DNN-extracted features as inputs to predict failures of the DNN model as targets.\nSome evaluation has been done using three image classification datasets. ", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Testing generalization performance of DNNs with unlabeled datasets is an important problem of deep learning. I am wondering whether the proposed problem formulation \"test prioritization\" really makes sense in ML societies (I noted the most of cited papers were from the software engineering domain). \n\nActually, there are a series of challenges and works on predicting the generalization gap using unlabeled datasets or validation-free information, such as NIPS 2020 Challenge \"Predicting Generalization Gap\" . Furthermore, there are also some papers in this domain such as \"In Search of Robust Measures of Generalization\" at NIPS 2020, and \"Predicting the Generalization Gap in Deep Networks with Margin Distributions\" at ICLR 2019. \n\nTwo simple yet effective baseline to compare with TestRank are as follows.\n(1) Using random data augmentation to generate pseudo testing samples using the training dataset with labels preserved, and predicting the generalization gap using the pseudo testing samples and the training samples. The gap of classification error between original samples and augmented samples could simply predict the generalization performance.\n \n(2) Using random data augmentation to generate an additional set of unlabeled dataset using the original unlabeled datasets, and verifying whether the outputs of the DNN model based on every pair of images (the original image and the augmented image) are consistent. Such inconsistency should also characterize the generalization gap. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This paper introduces a way to test the generalization performance of DNNs incorporating unlabeled datasets (validation-free information).", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "2hrs"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers appreciated both the interesting problem of prioritizing unlabelled data points as well as the graph-based similarity technique to predict failure-revealing capabilities of data points. There were some concerns regarding baselines, additional evaluation tasks, and motivation of design choices, but the responses clarified many of these concerns. It would be great if authors can incorporate the additional clarifications and experimental results in the final version.
Decision: Accept (Poster)