Explanation-based Data Augmentation for Image Classification
Authors: Sandareka Wickramanayake
Wynne Hsu
Mong-Li Lee
Venue: NeurIPS 2021 Poster (2021)
Keywords: Explainability
Data Augmentation
Image Classification

Abstract: Existing works have generated explanations for deep neural network decisions to provide insights into model behavior. We observe that these explanations can also be used to identify concepts that caused misclassifications. This allows us to understand the possible limitations of the dataset used to train the model, particularly the under-represented regions in the dataset. This work proposes a framework that utilizes concept-based explanations to automatically augment the dataset with new images that can cover these under-represented regions to improve the model performance. The framework is able to use the explanations generated by both interpretable classifiers and post-hoc explanations from black-box classifiers. Experiment results demonstrate that the proposed approach improves the accuracy of classifiers compared to state-of-the-art augmentation strategies.

Explanation-based Data Augmentation for Image Classification
Sandareka Wickramanayake Mong Li Lee
Wynne Hsu
School of Computing National University of Singapore
{sandaw, leeml, whsu)@comp. nus edu.sp
Abstract
Existing works have generated explanations for deep neural network decisions to provide insights into model behavior. We observe that these explanations can also be used to identify concepts that caused misclassifications. This allows us to understand the possible limitations of the dataset used to train the model, particularly the under-representte regions in the dataset. This work proposes a framework that utilizes concept-basee explanations to automatically augment the dataset with new images that cover these under-represented regions to improve the model performance. The framework is able to use the explanations generated by both interpretable classifiers and post-hoc explanations from black-box classifiers. Experiment results demonstrate that the proposed approach improves the accuracy of classifiers compared to state-of-tee ofte--oftthearr art augmentation strategies.
1 Introduction
Machine learning models learn decision boundaries based on the training dataset. Ideally, a training dataset should provide sufficient variations in each class that the model can learn the correct decision boundaries. However, when there are under-repressented regions in the training dataset, the model will learn sub-optimal decision boundaries, leading misclassification of data points 2 One way to address this problem is to augment the training dataset. A common data augmentation approach is to apply different transformations the training samples 3 or to mix existing samples 4. 5]. However, such data augmentations only explore the neighborhood of these samples and may not cover the under-represented regions. Another approach is use annotations such as part landmarks to obtain images from image resources such as Flickr or Google [6]. This approach may introduce
images and degrade the classifier performance.
Identifying the appropriate samples to add to the training dataset is crucial. Here, we propose to use model decision explanations facilitate the search for samples from image repositories and augment the training dataset with samples from the under-represented regions to enable the model to learn the correct decision boundaries. Figure Ta shows the image of juvenile Black Tern that has been misclassified as White Breasted Nuthatch. A closer examination of the training dataset reveals that it contains very few images of juvenile Black Terns out of 30 images). These juvenile birds have white heads with black skullcap and black earmark, unlike the adult Black Terns [Z] (see Figure [b). In fact, the juvenile Black Tern in Figure Ialis more similar to White Breasted Nuthatch as shown in Figure Ic leading to the misclassification. Clearly, there is need to augment the training data to include more images juvenile Black Ter order learn the correct decision boundary between Black Terns and White Breasted Nuthatch.
Methods explain model decisions include pixel-level attributions g 9 10 and high-level semantic concepts such as coarse-grand gaarseerainedd saliency maps [1] 12). interpretable basis decomposition [3],
linguistic explanations [14 [5] and prototypes [6] The work in 17 designs Comprehensive
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

(a) Juvenile black temn
(b) Images of adult black tern. (c) Images white breasted nuthatch.
Figure 1: Example of juvenile black tern that is misclassified as white breasted nuthatch.
Convolutional Neural Network called CCNN that explains the model decision in terms of word phrases, e.g., the bird in Figure Ta is classified as While Breasted Nuthatch due to the concepts 'white breast', 'white belly'. 'long beak'.
This paper introduces a framework called BRACE (BetteR Accuracy from Concept-based Explana- tion), which utilizes concept-basse explanations identify candidate samples from image repositories for data augmentation. The key idea is to identify concepts that have led to misclassifications and augment the dataset with images that contain these concepts. We design utility function that computes the degree of match between the visual features in an image and the set of concepts in the under-represented regions. The proposed approach enables us to select samples with high utility scores and increase the representativeness of the training dataset. Experiment results on multiple datasets demonstrate that BRACE outperforms state-of-the-art augmentation methods.
Related Work
Data augmentation is a widely used pre-processing step in training deep neural networks. The objective is to inflate the training dataset with label preserving small transformations to avoid over- fitting. Techniques include geometric transformations, color space augmentations, kernel filters, mixing images, feature space augmentation, adversarial training, generative adversarial networks, neural style transfer, and meta-learning (3. However, these augmentation methods only explore the neighborhood of existing images and are not able to add samples from under-represented regions [8]
The work in O taps into the large online image repositories to augment the training dataset. Object bounding boxes and part landmarks annotations in the original dataset are used to select relevant and diverse images. Since the images retrieved are typically noisy, [9 trains two models, to select in-distribution samples and to re-label the noisy images the selected set. Both these works may not increase the representativeness of the dataset as they do not consider the under-represented regions in the dataset when deciding which samples to add.
Existing explanation methods for deep neural network based image classifiers include pixel-level attribution methods and high-level semantic explanation methods. Pixel-level attribution methods 8 10 assign an importance score to each pixel in the image based on the contribution of that pixel towards the model decision. On the other hand, semantic explanation methods explain model decisions in terms of human
human-friendly friendly concepts and can be categorized into post-hoc methods versus fully interpretable classifiers. Post-hoc methods include Grad-CAM [I], FLEX [55, IBD [3] TCAV (20). and ACE (21). The first three approaches are image-level explanation mechanisms explaining the model's 's prediction of the given input. In contrast, TCAV and ACE are class-level ex- planation mechanisms that explain the characteristics of the entire class. Fully interpretable classifiers such as ProtoPNet [66] and CCNN (17) explain model decisions in terms of prototypical parts and word phrases, respectively. The proposed BRACE framework is able to utilize explanations from both post-hoc methods and fully interpretable classifiers to obtain new samples for data augmentation.
Proposed Framework
Machine learning models often make wrong classifications for classes that have insufficient variance in training data. Adding more diverse samples to the mix will enable the model to improve its accuracy. Intuitively, want give more training samples to classes with higher misclassifications so that the model is able to learn the real-world variations.
Let C' be the set of class labels, and D = {(ar, y:}} be the original training set, where yı E C is the class label of image Zę. For each class e E C, we want to augment D with a set of new samples that is proportional to the number of misclassifications for that class. Given deep neural network model

---

M, we compute the ratio of misclassifications re for class on the validation set D as follows:
Tc = E(E,c)ED [(M (z) = =c)
(1)
m
where m is the number of validation samples with class label e, and I is a function that returns when its argument is true and O otherwise. We will augment D with [ee] number of new samples where De C D is the set of images with class label c.
Let X be the set of samples obtained using class as the keyword query on image repositories. This set is typically noisy, which may degrade model performance. As such, we will assess whether the new image is in the under-represented region and whether it contains concepts that led to the misclassification. We say that an under-represented sample if it has visual features of class but the model's confidence that x belongs to a different class ē * is high. We compute the rudee--epressntatioon score as follows:
fa-fe ((d())
B(2,c,0) |Ifxllllfell
where far is the feature vector of x: and fe is the average feature vector of all the images in class c, P(člz) is the predicted probability of x belonging to class č. Feature vectors are extracted from model M. A high 3 value indicates that æ is similar to some images in class c, but the model has classified it as class Such samples are more informative, and using them for training may help M generalize better.
Further, the sample x should not only possess features of class e but should also contain specific concepts that caused the model M to misclassify it as class č. Adding images with these specific concepts will help the model better differentiate class from class C. Let Sa-v be the set of concepts that caused M to misclassify images of class e into ē, and C = U². Let (S x:) be the function that computes the degree of match between the visual features in an image x and the concepts in S. The derivation of S and (.) differ depending on the explanation mechanism used (see Section 4). The utility score of a sample æ is computed as follows:
utility(x)
Σ [B(x,c,0) A(S--1,z)) ZEC
Then Del X Te samples with the highest utility scores are added to the original train dataset. We repeat this process for all the classes in D. We fine-tune the weights of M with the augmented training dataset. Note that adding new samples may result in data imbalance, and this is resolved by using weighted cross- cooos-y loss during fine-tuning where the weight of class c is given by [D][[++r.) With this, there will be minimal change to the weights of the classes where M already performs well. Algorithm gives the details of the BRACE framework.
4 Derivation of S.4z and A()
In this section, we describe how we derive the concepts that cause the misclassification, Sc-sa for two popular explanation mechanisms: the fully interpretable classifier with linguistic explanation and the black-box classifier with post-hoc explanation using heat maps. We also show how to compute the degree of match A(.) these cases.
4.1 Fully Interpretable Classifier with Linguistic Explanation
The work in [17] introduces an additional concept layer to the CNN-based architecture to guide the learning of the associations between visual features and word phrases extracted from image descriptions. This Comprehensible Convolutional Neural Network (CCNN) explains its decisions in word phrases corresponding to different visual concepts. The training objective function considers concept uniqueness which encourages each learned concept to correspond to only one word phrase, and mapping consistency which aims to preserve the distance between the learned concept and its corresponding word phrase in joint embedding space. Together with classification accuracy loss, CCNN is currently the otate-of-the-art fully interpretable classifier. Here, we discuss how the proposed BRACE framework can further improve CCNN classification accuracy.
3

---

Algorithm 1: BRACE
input :Classification model M, Original training datasct D, Set of class labels C'
output: Funetteeinetuneed weights 0
0 <- Weight initialization.
b Initialize with pre-trained weights of M
for ater E [0, maaz [ineziitration]]] do
D' D
for e e do
Te f Calculate miss-classification ratio Eq.
X. + Obtain images given the class name of c as the query term.
for E X. do
for do
c( Calculate inder-representation score as in Eq.
Se + Derive concepts caused misclassifications ASS 2) &- Calculate availability of concepts S
end
utility(x) Exec (,( A(Sc-c:)))
end
D' - D x r. samples with the highest utility scores]
end
0 finetune weights(M, D') end
Let V = {ui...., J be the set of visual features extracted by the concept layer of CCNN for an image x. Each 8, E V corresponds concept described by a word phrase p E p= {PI.... Pn CCNN employs Global Average Pooling (GAP) layer to express its classification decision weighted sum of the learned concepts. Suppose the output of the GAP layer {01, .... 0n where corresponds to concept i in z. Then the contribution of i to the class decision e is given by oz x Wi.e W, where W E RmxC is the weight matrix of the fully connected layer CCNN. We apply the softmax function to obtain the percentage contributions and select the set of concepts whose contribution greater than some threshold. We will take the top-5 there are more than five concepts that satisfy the criteria. Here, set the threshold to be J. For instance, CCNN classifies the bird in Figure 2 as Nashville Warbler and explains that the concepts "Yellow breast", "Grey crown" and "White eyering" accounted for 47%, 19%, and 14% of the decision, respectively.
Predicted class: Nashville Warbler Concepts contributed:
Yellow breast (0.47),
Grey crown 19).
White eyering (0.14)
Figure 2: Concepts contributed to CCNN's predictions.
Given Xe-se the set of images with class label that have been misclassified e, compute the number of images in X--sa that contains the concept i. The top concepts with the highest number form the set So-z. We set le = 3 in experiments. Recall, each visual feature in CCNN concept layer is mapped to a word phrase ps describing a concept, and v; is activated only when the concept occurs in the image. As such, given the set of concepts S.-e, we utilize their corresponding visual feature activations to determine the degree of match of a concept i in an image x as follows:
1
a, = log 1 +
where o; is the normalized o, obtained by subtracting the mean activation of the GAP layer. The negative log function gives more weight to an image with one highly activated concept than one with multiple weakly activated concepts. With this, the degree of match between the set S and image x is given by:
A ASS x) =
Ok
(5)
1=1

---

4.2 Post-hoc Explanation
BRACE requires image-level explanation identify concepts that caused misclassification. One widely accepted image-level post-hoc explanation mechanism is Grad-CAM [IT]. Grad-CAM computes gradients of the predicted class score with respect to the last convolutional layer feature maps. The gradients are averaged channel-wise and multiplied with the respective feature maps to create a coarse- -aanseeg saliency map. Since feature maps in the last convolutional layer correspond we consider the image regions highlighted by GradCAM to cover semantic concepts. our implementation, we normalize the pixels in the saliency map to have values between 0 and 1, and regions whose values are greater than the threshold of 0.5 correspond to semantic concepts that are responsible for the model decision.
to semantic concepts [22 23
Let Xe-+z be the set of images whose class label has been misclassified from to č. Then S.-+ is the set of regions Xe-+z obtained from the saliency maps of Grad-CAM. The regions Se+t are passed through M, and the output of the last layer before the classification layer form their visual features, denoted as U. Given an image I, we use a state-of-the-e salient region proposals that correspond to an object or object part. These region proposals are passed through M to obtain the corresponding visual features denoted as W. The degree of match between So-s. and is given by the similarity between visual features in U and W.
RCNN model [24] to derive
A(Sc-E, = Σ ZM NEU
where Zu = max WEW
(- -llog 1
u.w
+ (
(6)
| ||ll
An image with high A(S x) indicates that it contains many concepts that led the model to misclassify. Such images help the model to learn correct decision boundaries and are selected to augment the training dataset. For each concept u that causes misclassifications, 2, indicates the degree of match between u and some region w in œ. We take the region with the maximum similarity as it indicates the highest probability of being present in the image. Using a negative log allows us to choose an image containing at least one high confidence concept that causes misclassification over another image with multiple low confidence concepts. This approach can be utilized when we use BRACE with any explanation method that provides super-pixel-based explanations, e.g., ACE [2] and IBD [3.
Performance Evaluation
We carry out experiments to show that BRACE can effectively improve the accuracy of general image classification tasks as well as fine-grained image classification. We also demonstrate the broad applicability of BRACE on wide range of network architectures such as ResNet and DenseNet (6) All the codes are implemented using PyTorch, and the experiments are run on NVIDIA Tesla V100 GPUs The following datasets are used:
1. Caltech UCSD Birds (CUB) [27]. This dataset has 11,788 images of birds belonging to 200 classes. The dataset is divided into a train set of 3994 images, a validation set of 2000 images, and test set of 5794 images. Each image has ten sentences describing the bird collected by (28].
2. CUB-Families [29]. This dataset groups the 200 species of birds in CUB into families comprised of multiple bird species. We create under-represented regions by removing some species in the test split of [30] from each family in the training dataset. No species are removed in the validation and test sets. The resultant dataset contains 4585 training images, 2343 validation images, and 4860 test images.
3. Tiny ImageNet. Tiny ImageNet contains 110,000 images with 200 classes. In the standard dataset split, each class has 500 training images (400 for training the model and 100 for tuning the paper-parameters and 50 validation images. We report our results on the validation set.
In our experiments, we use the samples collected by [31 for CUB and CUB-Families. Since (31 has not collected samples for Tiny ImageNet, we use Flicker API to obtain 250 new samples per class using the class name as the search query. These samples are used by all the methods in our comparative study.
5

---

The number of samples added to each class c is based on the ratio of misclassification Te (recall Section 3. To ensure that no two images are highly similar, we remove images similar the ones in the training image set and duplicates obtained from the online repositories.
Since CCNN requires indicator vector per image for training, we extract 398 and 378 word phrases from the image descriptions of CUB and CUB-Families, respectively, to construct these vectors depicting the concepts present in each image. The union of the indicator vectors in the same class are called the class indicator vectors. For each dataset, we set the number of nodes in the concept layer to be the same as the number of extracted word phrases and train CCNN using the same hyper-paramee parameters settings in [17]. For new samples without image descriptions, we use the corresponding class indicator vector as the indicator vectors.
5.1 Comparison with Data Augmentation Methods
We compare BRACE and the following state-of- of-thh-artt data augmentation methods:
CutMix [5] This mixed-based data augmentation method cuts out one image patch, pastes it on another image, and mixes their labels according the area proportion.
SnapMix [4]. This is another based data augmentation method. It combines images similar to CutMix but uses the semantic composition of the resultant image to derive the label.
WS-DAN 32), WS-DAN is an attention-based approach that uses attention-cropping and diteetiio-ddopping. Attention-cropping refers cropping the region of the image attended by the model to create a new sample. Attention-dropping refers to erasing the attended region create new sample.
66 This method obtains samples from image repositories by using object bounding boxes and part landmark annotations.
Part-based
Metaset-based [9] This method also obtains samples from image repositories. It trains two models to ensure the samples are in-distribution and to correct noisy labels.
We train black-box classifier using ResNet-34 architecture. Table I shows the results CUB, CUB Famiiiais,,s and Tiny ImageNet. We observe that training using the BRACE-augmented datasets consistently achieves the highest accuracy for all datasets. In contrast, Cut-mix and Snap-mix fall short as they only combine existing images without considering the under-representee regions. Though WS-DAN uses the attention weights to determine the regions for cropping/dropping, the samples created may not cover the under-represented regions, leading to a lower accuracy compared BRACE. For the CUB dataset, Part-based augmentation added a total of 106,000 samples while Metaset-based added 18, 18,000 samples. Despite the large number of new samples, the improvements lower compared BRACE, indicating the importance of adding only samples that are likely to cover the under-represented regions.
Table 1: Performance of ResNet-34 black-box classifier with GradCAM.
Method
CUB
CUB-Families
Tiny ImageNet
Original dataset
85.5
82.6
76.1
Cut-mix
85.7
85,9
77.0
Snap-mix
87.1
86.3
77.5
WS-DAN
87.2
83.5
76.6
Part-based
86.3*
Metaset-based
86.8
89.3
73.1
BRACE
87.7
90.0
78.3
*taken from corresponding references.
Next, we examine the performance of a fully interpretable classifier using these augmentation techniques. We train a CCNN based on the ResNet-34 architecture. Note that when mixed-based augmentations are used with CCNN, the indicator vector of the original image might not match the resultant image. Hence, we combine the class indicator vectors of mixed classes according to their proportions and apply threshold to create the indicator vector of the resultant image. Further, CCNN requires image descriptions that are available in the CUB and CUB-Fanilies datasets. We omit the Tiny ImageNet dataset as it does not have image descriptions. Table 2 shows the results on CUB and

---

Table 2: Performance of fully interpretable CCNN classifier based on ResNet-34.
Method
CUB
CUB- Fammiliis
Original dataset
84.3
83.8
Cut-mix
80.6
79.0
Snap-mix
82.4
79.9
WS-DAN
81.6
81.8
Metaset-based
85.1
88.1
BRACE
88.7
CUB-Families. Again, we see that BRACE- augmented datasets achieve the highest accuracy. The improvement is bigger in CUB-Familics where there are more inder-represented regions.
5.2 Comparison of Sample Selection Methods
BRACE ranks the samples based on the utility scores and selects those with top scores to augment the training dataset. In this set of experiments, we compare BRACE with the following methods:
Random. Here, the samples are selected randomly.
Confidence. The samples obtained each class are passed through the classifier. We rank
the samples based the classifier's scores, and the top scored samples selected. Core-set [331. k representative samples are selected using the K-Center- Center-Greddy algorithm.
L-loss (34]. The classification loss of sample is predicted, and samples with the top-k losses are returned.
Table 3 shows the classification accuracies for 1esNet-101 and DenseNet-161 based classifiers using post-hoc explanation. We see that Brace(utility) has the greatest improvement accuracy in all datasets. The results for L-loss and Core-set are mixed when compared to Random and Confidence methods. This because both L-loss and Core-set tend to select more noisy images (e.g., out-of- distribution images) as compared to Random and Confidence leading to lower performance accuracy. See Figure
Table 3: Comparison of classification accuracies using post-hoc hoc explanation.
Method
CUB
CUB-Families
Tiny ImageNet
Dense- 161
Res-101
Dense- 161
Res-101 101
Dense-161 161
Res-101
Original dataset
86.6
87.4
86.5
85.4
80.1
81.3
Core-set
84.8
85.9
87.2
88.6
80.2
77.8
L-loss
85,2
84.8
86.5
88.4
78.3
77.4
Random
86.5
86.4
87.0
80.3
Confidence
87.3
87.0
86.5
86.6
80.6
81.3
BRACE(utility)
88.0
89.2
93.0
91.2
81.1
81.7
Table 4: Comparison of classification accuracies using linguistic explanation mechanism.
Method
CUB
CUB- Families
Dense-161
Res- 101
Dense-161 161
Res-101
Original dataset
84.5
86.6
85.8
85.7
Core-set
85,0
84.5
87.1
86.1
L-loss
82.0
84.7
84.7
85.5
Random
85.8
87.0
88.6
88.0
Confidence
85.4
86.7
85.8
85.8
BRACE(utility)
86.8
88.4
91.9
Table 4
4shows shows the classification accuracy when we use the linguistic explanation mechanism. We observe that the proposed BRACE(utility) consistently gives the highest accuracy. We also investigate the performance of BRACE(utility) in the under-represented classes by selecting five classes in the CUB-Families dataset that have the lowest classification accuracy. Table that BRACE(utility) significantly improves the classification accuracies of these classes compared to other methods. Note that Confidence has lower accuracy compared Random. One possible reason is that the samples selected by Confidence tend to be similar to the training images resulting in model over-fitting.
7

---

Table 5: Class-level classification accuracies on different categories in CUB- (UB-Families.
Method
Hirundinidae
Mimidae
Cuculidae Fringillidae
Icteridae
Original dataset
41.1
46.7
52.3
53.1
55.4
Core-set
54.7
67.6
72.7
56.3
67.7
L-loss
52.8
66.7
68.2
49.2
72.1
Random
67.1
70.2
63.9
63.3
70.8
Confidence
32.9
44.4
56.9
53.1
50.0
BRACE(utility)
72.6
71.1
75.6
82.2
Don Uniltty
Corridence Randoon
Dxsdins LiElly
Confilence Rancom
Coreest 14855
(a) CUB
(b) CUB-Families
Figure 3: Classification accuracy of DenseNet-based CCNN.
Category
Removed images
Samples selected by method
Utility
Random
Confidence
Core-set
L-loss
Hirundinidae
CAT
Totalia
Fringillidae
Figure 4: Comparison of samples selected by different methods.
Figure 3 shows the classification accuracy of the DenseNet- Mel-Neet-111-basee CCNN for the different itera- tions as we add samples to the training dataset. We see that with just one iteration, BRACE(UUility) is able to achieve higher accuracies compared to the other methods. This indicates that the utility score can identify the under-represented, and hence more informative samples are added to the training dataset, thus enabling the learning of such under-represented class boundaries. This performance gap dataset.
widens the
Finally, we visualize the samples selected for the "Hirundinidae" and "Fringillidae" categories in CUB-Families. Figure that the samples selected by BRACE(utility) are the most similar to the images in the subcategories that have been removed. This assures us that BRAC BRACE(utility) can select samples containing concepts the under-represented regions. This allows the model to learn the correct boundaries for these under-represented regions, leading higher accuracies.

---

5.3
Ablation Study
Next, we examine the effects of 3 and by implementing two variants BRACE with CCNN:
BRACE-A does not take into account the degree of match with the concepts that caused misclassification. In other words, the utility score is given by Eeed (1,,, c,
BRACE P does not consider if a sample falls in the under-representedd region and computes the utility score EEEo A(S
Table shows shows the classification accuracy on CUB and CUB-Families datasets. We observe that the highest accuracy achieved when both 3 and A are used in the computation of utility score. This is because 3 ensures that the samples selected fall into the under-represented regions, while A ensures that the concepts that caused misclassifications are present in the selected samples. Further, we see that A plays greater role in improving the classification accuracy compared to 3.
Table 6: Results of ablation study.
CUB
CUB-Familiss
Method
Dense- 161
Res-101
Dense- 161
Res-10
CCNN
84.8
86.6
85.8
85.7
BRACE 85.9
87.1
91.2
91.4
BRACE
86.3
87.6
91.3
91.6
BRACE
86.8
88.4
91.9
92.2
Class
Sample images
Images removed
BRACE
BRACE
BRACE
Alcedinidae
Concepts: blue crown, white neck, red crown
Stercorariidac
Concepts: large
wingspan, black crown, grey wing
Figure Comparison of samples selected by variants of BRACE.
Figure5 shows samples selected by BRACE variants for two classes in CUB-Families given the top-3 concepts responsible for the misclassifications for each class. We observe that BRACE-A has selected images from under-represennted regions. They either belong to the subcategories that were

---

excluded from the training dataset, have unfamiliar backgrounds, or the discriminative features of the class are hidden due the pose or occlusions. However, the images selected by BRACE A do not include the concepts caused for misclassifications. On the other hand, BRACE- selects images clearly showing one or more concepts that have led to misclassifications, e.g., Alcedinidaes with white neck and Stercorariidae with large wingspan. However, some of these images do not fall into under-represented regions. contrast, BRACE has selected images from the under-represented regions that contain concepts contributed for misclassifications Such images are informative and help the model to learn correct boundaries, leading to higher accuracies.
We compute the "overlapping rate". defined as the intersection over the union of sets of the samples selected by BRACE variants. The overlapping rate of BRACE À and BRACE 8 0.09. indicating that the two variants tend to select different image sets. In addition, the overlapping rate between BRACE and BRACE- A is 0.20, which lower than the 0.37 overlapping rate between BRACE, and BRACE- This suggests that A has a greater influence on the set of images selected by BRACE.
5.4 Generalizability with BRACE
A representative dataset facilitates the learning of more generalizable features. As a result, a classifier trained on such representative dataset can recognize wide variety of instances of class even when some of those variations were not part of the training data for that class. We demonstrate the generalizability of BRACE in this final set of experiments.
We use ResNet-34 trained on CUB-Families and test on a subset of NAbirds [35. This subset consists of classes that are found in
These classes have wide variety of sub-categories that may or may not be present in the CUB-Families. We call this subset NAbirds-Sub. For ResNet-34 trained on Tiny ImageNet, we test it on the subset of images from ImageNet-V2 (36) This subset consists of classes that are in Tiny ImageNet, and we call it ImageNer-V2-Suu. Table 7 shows that BRACE(utility) achieves the highest accuracy, confirming that BRACE enables the classifier to learn features that are generalizable to handle more diverse images. Note that when collecting new images for Tiny ImageNet using Ficklr API, we set the API parameters different from those used in 336 to ensure there is no overlap between the images we collect and ImageNet-V2 V2.
Table 7: Comparison on generalizability.
Method
NAbirds Sub
ImageNet- V2-Sub
Original dataset
81.5
54.4
Cut-mix
72.1
37.2
Snap-mix
76.0
42.6
WS-DAN
67.0
56.1
Metaset-based
83.5
42.4
Core-set
83.0
61.9
L-loss
79.7
63.9
Random
82.4
65.1
Confidence
81.5
67.6
BRACE(utility)
84.9
70.0
6 Conclusion
In this work, we have examined the potential of utilizing explanation mechanisms to augment training datasets to increase classification accuracy. In contrast to existing data augmentation approaches, we have proposed a framework called BRACE that uses the explanations from model decisions to understand the limitations in the training dataset and select informative images that are in the under-represented regions and contain visual concepts that have led to misclassifications to fine-tune the classifier. The assumption here is that the image repositories contain a sufficiently large number of diverse images. For applications with limited images (e.g., images of rare diseases), the performance of BRACE may be affected. Nevertheless, the experiment results multiple datasets demonstrate that our proposed solution is able to learn features that are generalizable and improve the classification accuracy over state-of-the-art augmentation methods.
Acknowledgement. This research is supported by the National Research Foundation, Singapore under its Al Singapore Programme (Award No: AISG-GC-2019-001, AISG-RP-2018--00
10

---

References
[1] A. Torralba and A. A. Efros, "Unbiased look at dataset bias," in CVPR 2011, 2011.
[2] R. Taori, A. Dave, V. Shankar, N. Carlini, B. Recht, and L. Schmidt, "Measuring robustness to natural distribution shifts in image classification," NeurIPS, 2020.
[3] C. Shorten and T. M. Khoshgoftaar, "A survey on image data augmentation for deep learning," " Journal of Big Data, 2019.
[4] X. W. Shaoli Huang and D. Tao, "Snapmix: Semantically proportional mixing for augmenting fine-grained data," " in AAAI, 2021.
[5] S. Yun, D. Han, S. Oh, S. Chun, Choe, and Y. Yoo, "Cutmix: Regularization strategy to train strong classifiers with localizable features," in ICCV, 2019.
[6] Z. Xu, S. Huang, Y. Zhang, and D. Tao, "Webly-supervised ine-grained visual categorization via deep domain adaptation," IEEE 2016.
IEEE transactions on pattemn analysis and machine intelligence,
[7] "All about birds." httpp://www.lllllabbbbrrrrrrr//a 2021-01-06.
Accessed:
[8] K. Simonyan, A. Vedaldi, and A. Zisserman, "Deep inside convolutional networks: Visualising image classification models and saliency maps," arXiv, 2014.
[9] A. Binder, S. Bach, G. Montavon, K. R. Muller, and W. Samek, "Layer-wise relevance propaga- tion for deep neural network architectures," in Information Science and Applications, pp. 913- 922. Springer, 2016.
[10] M. Sundararajan, A. Taly, and Q. Yan, "Axiomatic attribution for deep networks," in ICML, 2017.
[11] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-cam: Visual explanations from deep networks via gradient-based localization." in ICCV, 2017.
[12] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discrininative localization,"" " CVPR, 2016.
[13] B. Zhou, Y. Sun, D. Bau, and Torralba, "Interpretable basis decomposition for visual explanation," in ECCV, 2018.
[14] L. A. Hendricks, Z. Akata, M. Rohrbach, J. Donahue, B. Schiele, and T. Darrell, "Generating visual explanations," in ECCV, 2016.
[15] S. Wickramanayake, W. Hsu, and M. L. Lee, "Flex: Faithful linguistic explanations for neural net based model decisions," " in AAAI, 2019.
[16] C. Chen, O. Li, D. Tao, Barnett, C. Rudin, and K. Su, "This looks like that: deep learning for interpretable image recognition," in NeurIPS, 2019.
[17] S. Wickramanayake, W. Hsu, and M. L. Lee, "Comprehensible convolutional neural networks via guided concept learning," " in IJCNN, 2021.
[18) T. Tran, T. Pham, G. Carneiro, L. Palmer, and I. Reid, "A bayesian data augmentation approach for learning deep models," in NeurlPS, 2017.
[19] C. Zhang, Y. Yao, X. Shu, Z. Li, Z. Tang, and Q. Wu, "Data-driven meta-set based fine-grained visual recognition." in ACM-MM, 2020.
[20] B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, el al., "Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)," ICMI,, 2018.
[21] A. Ghorbani, Wexler, Y. Zou, and B. Kim, "Towards automatic concept-based explanations."
in NeurlPS, 2019.
[22] M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," in ECCV, 2014.
[23] D. Bau, B. Zhou, A. Khosla, A. Oliva, and A. Torralba, "Network dissection: Quantifying interpretability of deep visual representations," in CVPR, 2017.
[24] Ren, K. He, Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," NewrIPS, 2015.

---

[25] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition." in CVPR, 2016.
[26] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in CVPR, 2017.
[27] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, "Caltech-uscl 200-2011 dataset," California Institute of Technology. 2011.
[28] S. Reed, Z. Akata, H. Lee, and B. Schiele, "Learning deep representations of fine-grained visual descriptions," " in CVPR, 2016.
[29] T. Chen, W. Wu, Y. Gao, L.. Dong, X. Luo, and L. Lin, "Fine- Frrineggrined representation learning and recognition by exploiting hierarchical semantic embedding," in ACM MM, 2018.
[30] Y. Xian, C. H. Lampert, B. Schiele, and Z. Akata, "Zero-shot learning- a comprehensive evaluation of the good, the bad and the ugly," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
[31] C. Zhang, Yao, H. Liu, G.-S. Xie, X. Shu, Zhou, Zhang, F. Shen, and Z. Tang, "Web- supervised network with softly update-drop training for fine-grained visual classification,"" in AAAI, 2020.
[32] Hu, H. Qi. Q. Huang, and Y. Lu, "See better before looking closer: Weakly supervised data augmentation network for fine-grained visual classification," arXiv preprint arXiv: (eeee1101999999 2019.
[33] O. Sener and S. Savarese, "Active learning for convolutional neural networks: A core-set approach," in ICLR, 2018.
[34] D. Yoo and Kweon, "Learning loss for active learning," " in CVPR, 2019.
2019.
[35] G. Van Horn, Branson, R. Farrell, S. Haber, J. Barry, P. Ipeirotis, P. Perona, and S. Belongie, "Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection," in CVPR, 2015.
[36] B. Recht, R. Roelofs, L. Schmidt, and V. Shankar, "Do imagenet classifiers generalize imagenet?," in ICMI, 2019.
12


Reviews:
[{"review_id": "qFPyDkBt_8D", "reviewer": "Reviewer_4Jog", "summary": "This paper proposes a form of data augmentation that uses explanations to select external images from poorly performing classes.  Specifically, it looks for images that:  \n- are \"under-represented\" meaning they are similar (according to the model) to images of the target class but are predicted as being part of a different class\n-  have the same concepts that are often present when the model mistakes images of the target class as being part of this different class", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n# Originality \n\nThe methodology is novel and the idea of using explanations to guide data augmentation is an interesting one.  The connection between this work and prior work is clear.  \n\n\n# Quality\n\nMost of the claims are well-supported, but there are a few things that need clarification.  \n\n*L128:  \"We apply a softmax function to obtain the percentage contributions and select the set of concepts whose contribution is greater than some threshold.\"*\n\nHow is this threshold chosen?   \n\n\n*L150: \"We consider the image patches whose pixels have importance scores greater than some threshold as the concepts that have contributed to the model decision.\"*\n\nHow is this threshold chosen? \n\n  \n*L175: \"We report our results on the validation set.\"*\n\nBased on the \"Implementation Details\" in the Supplement, it seems like the hyper-parameters for this method were tuned while those from the baselines were not.  How were the hyper-parameters tuned?  If the validation set was used, this places the results on this dataset into doubt.  This is especially concerning because CUB(-Family) is a small dataset, where we would expect augmentation to be more helpful than on larger datasets, and TinyImageNet is the only larger dataset tested.  \n\n\n*Table 2.*\n\nWhy are the results for the other \"external source\" data augmentation methods not shown?\n\n\n*5.2 Comparison of Sample Selection Methods*\n\nWhy are these results based on ResNet101/DenseNet161 while the previous sections' results are based on ResNet34?\n\n\n# Clarity\n\nMost of the paper is clearly written.  Detailed comments follow\n\n*2.  Related Work*\n-  It would be nice if this section was clearly divided into \"Augmentation\" and \"Explanation\" based sections\n-  It would be nice if relationship between [6] and [19] was clarified \n-  It is a little unusual to describe methods like GradCAM as concept-based because they just point to parts of the image when methods like TCAV/ACE/CCNN are directly based on concepts.   Relatedly, it is surprising that only GradCAM and CCNN are used in Section 4.  \n\n*3.  Proposed Framework*\n-  \"A high β value indicates that x is similar to some image in class c but the model M has classified it as class c ̄. \" This doesn't seem quite right because Equation 2 is based on similarity with the mean of the class representation rather than the max of the similarity images of the class.  \n-  Generally, a simple illustration would help explain the intuition behind the beta term (which otherwise requires some effort to realize why it is a \"under-represented region\") \n\n*4.1 Fully Interpretable Classifier with Linguistic Explanation*\n-  This section relies heavily on the reader being familiar with CCNN\n\n*4.2 Post-hoc Explanation*\n-  It would be nice if the intuition behind Equation 6 was explained the same way it was for Equation 4\n\n*L170: \"We create under-represented regions by removing some species in the test split of [28] from each family in the train dataset.\"*\n\nIt should be made clear that the purpose of this is to do something like zero-shot learning.  Otherwise, this experiment looks like one that is clearly designed for this method to work.  \n\n\n*Table 5*\n\nThis caption should be explicit about the fact that these are classes where the training data was deliberately removed.  \n\n# Significance\n\nIf the concerns over hyper-parameter selection for TinyImageNet and why the Part/Meta-based augmentation strategies were excluded from Table 2 are addressed, this does seem like a significant contribution.  \n\n# Update based on discussion\n\nThe reviewer's main concerns have been addressed and the rating has been revised upwards by a point.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "gV3vjzx9R25", "reviewer": "Reviewer_NzM8", "summary": "This paper proposes an explanation-based data augmentation method for image datasets. Technically, this method defines a utility function to choose the data samples collected from the Internet. The utility function is defined to select the samples that could better reflect the under-represented region in the data distribution. The authors compare the proposed method with some existing data augmentation techniques on three datasets and showcase the effectiveness of the proposed method to helping achieve higher finetuning accuracies.  ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. The paper mentioned two types of existing data augment methods: the first type of methods conduct certain transformations to input samples to explore the local regions around the given inputs. It is relatively straightforward to understand that these methods cannot explore the under-represented region because of the local exploration nature. The second type of methods collect new data from other sources and add those data into the training set. The paper first argues that these methods will introduce noise or OOD samples, which is again quite obvious. However, the authors then argue in the L59 these methods can neither explore the under-represented region. I am kind of doubting this argument. I could understand that the authors want to argue that these methods do not explicitly include a mechanism to explore the under-represented region. But this does not mean that they cannot achieve this goal. In fact, collecting diverse samples from other sources may also serve the purpose of exploring under-represented regions. I would suggest the authors provide some empirical supports on this argument other than the finetuning accuracy. For example, the authors could show the utility scores of samples selected by these methods to show whether they could uncover the under-represented region. \n\n2. My second comment is that the first and second terms in the utility function may have a similar effort and select a similar set of samples. My hypothesis is that most misclassified samples are wrongly labeled because they contain the specific features that cause the model to make mistakes. To this end, It seems that the second term (explanation term) only helps to filter out the highly confusing samples that are caused by randomness. However, since the first terms select the samples that are most deviated from the class center of their true class. It is less likely that their misclassifications are caused by randomness. I appreciate that the authors provide an ablation study to showcase the effectiveness of each term. I would suggest the authors also report the difference in the sets of samples selected by each term and showcase some examples. IMHO, this will help better understand the different effects of the two terms and why the second term works better than the first one. \n\n3. Regarding the proposed utility function, I found the following questions to be addressed to make it clearer. \n(1) What is the GAP layer, and what is $o$_{i} in line 126? (2) For the post-hoc explanation models, why not directly using Grad-CAM to generate explanation? What is the insight of choosing R-CNN? Why not selecting other explanation methods? (3) In Eqn. (6), what is the insight of adding the first term (i.e., max_{w \\in W}).\n\n4. Regarding the evaluation, I have the following concerns: (1) Major concern: the performance improvement in most setups is marginal. I also checked the results with stds shown in the supplement and found out that in some cases, there are overlaps between the accuracies obtained by baselines and the proposed method. To better demonstrate the superiority of the proposed method, I would suggest the authors conduct some statistical testings (e.g., paired t-test) on the results; (2) I found some inconsistencies among the experiments. More specifically, I would suggest the authors comment what's the reason for only applying two baselines in the self-explainable model experiment. (3) As I mentioned earlier, besides the finetuning accuracy, I would also suggest reporting the results of other metrics to showcase the effectiveness of the proposed method (e.g., the utility scores of samples selected by each method and the sample overlapping rate in the ablation study, the error collection rate (the number of originally errors that are corrected after finetuning) and the number of newly introduced errors after finetuning). (4) I would suggest adding a hyper-parameter sensitivity test on \\epsilon and the explanation method used in the black-box case. \n\n5. I would suggest the authors run a grammar check on the current version. I found some typos and grammar errors (e.g., Line 141 convepts -> concepts).\n\n==== After rebuttal. ====\nThe rebuttal responses addressed most of my comments (especially the potential overlap between the $\\beta$ and $\\Delta$ and the statistical tests). With that, I am happy to raise my score to above the acceptance threshold. I would further appreciate that if the authors could include the new experiments and other parts of the responses in the revised paper (Or the camera-ready version if get accepted).\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This paper discusses one possible limitation in the conclusion section. I would appreciate it if the authors could include a more comprehensive discussion of the limitations and future works, for example, the scalability of the proposed method, the generalizability of the proposed method in other application domains, etc. Regarding the social impact, the authors could discuss the potential threats of the proposed method if attackers use it to generate poison data. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "about 7 hours"}, "extra_text": {}}, {"review_id": "ckdbkbitX7q", "reviewer": "Reviewer_zD5d", "summary": "This paper proposed a utility function to rank new training samples with respect to their potential contributions of improving the model performance. The proposed utility contains two parts, one is for the under-represented region, another is based on explanation algorithms for finding the concepts that cause the misclassification. Experiments proved the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall I find the idea of this paper interesting: using explanations that indicate the reasons of misclassification to improve the model performance is interesting and practically useful.\n\nHowever, there are two main issues: \n\n1. The sample selection from this paper is on web datasets, while authors only compared with two trivial methods, Random and Confidence. Though I am not familiar with the sample selection, I believe there exists some other more advanced baselines and I suggest authors compare with advanced baselines. Because, fundamentally, the proposed approach is for sample selection from an open dataset, instead of data augmentation, according to my understanding. Related key words may be learning from web images (found in the cited paper [29]), but others words may be also pertinent. I hope authors could address this issue with clear explanations and/or experiments.\n\n2. The presentation is not good. (1) The samples in Figure 1 are mis-labeled by the dataset or mis-classified by the model? I didn’t understand the presentation of Figure 1. (2) Better highlight the definition of under-represented sample/region, or mention it at beginning (Line 89). (3) The designed scores should be better explained, with intuitions and remarks. For example, which samples will obtain high utility scores, and in which way these samples will help the model to generalize better. Without some intuitive introductions, the formulas are difficult to understand. (4) The designed formulas are strange and there are no explanations. For example, why exponential is needed in eq.(2)? Why log is needed in eq. (6)? Will some other alternatives be also useful? e.g., self-entropy as utility? \n\nand several detailed questions:\n\n1. How many samples have been selected for each dataset? I have seen the reported numbers for two methods. I believe the number of samples used by this paper is much smaller.\n2. I briefly looked at the cited paper [29], which is a weakly supervised method. Authors used the collected samples from [29] for trainings on CUB. How the labels of these new samples are obtained? Or did I miss something?\n3. Are the compared methods using samples from the same image repositories?\n\n\n\n[29] C. Zhang, Y. Yao, H. Liu, G.-S. Xie, X. Shu, T. Zhou, Z. Zhang, F. Shen, and Z. Tang, “Web supervised network with softly update-drop training for fine-grained visual classification,” in AAAI, 2020.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": ".", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Zhsd9uoyXj7", "reviewer": "Reviewer_Wy9k", "summary": "The paper proposes to use interpretation to select effective instances for augmenting training data. A utility function is developed, where images with high scores mean that: 1) the images are helpful for better learning class-c data, 2) the images could significantly correct the drawbacks of the model M. The paper discusses two interpretation scenarios, interpretable models (GAP+linear model) and post-hoc interpretation. The experiment includes a comparison to two types of baseline methods, data augmentation methods and data selection methods. Ablation studies are also conducted to show the effectiveness of the two components in the utility function. Different parts in the current version are somehow fragmented. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The paper proposes an interesting application of interpretation to improve models. The current approach mainly combines existing interpretation methods, but the overall design is reasonable and well-motivated.\n\nQuality: In general, the proposed approach is technically sound. The core contribution is an interpretation-driven utility function with two terms. The first term encourages selecting instances for better learning class c which was not well learned in the model. The second term encourages selecting instances containing crucial concepts. In this way, interpretation is fully utilized. The effectiveness of the two terms is also supported in the ablation study. \nMeanwhile, there are several factors that could be further clarified or improved. First, why do we need NLP-based content for interpretation? Second, what is the running time of the approach? Third, the proposed method is more like an active learning method than data augmentation, but the baseline methods are not balanced to reflect this (i.e., typical methods for active learning or data selection are not included as baseline methods).\n\nClarity: The paper is, in general, well written and well organized. Figure 1 is also helpful for understanding the motivation. However, Section 4.2 could be extended with more details, especially for the last paragraph.\n\nSignificance: The proposed work should be interesting to XAI researchers. A potential limitation is that the interpretable model used in Section 4.1 is not a commonly used model in practice, which may limit the applicability of the proposed method. However, post-hoc methods, which are more flexible, are not fully discussed in this work.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please refer to the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "4b-xx5_xWew", "reviewer": "Reviewer_79xf", "summary": "The paper proposes a data augmentation framework called BRACE that utilizes concept-based explanations from existing interpretability methods (such as Comprehensible Convolutional Neural Network (CCNN) and GradCAM) to identify informative examples to be added to the training set. Specifically, the proposed approach defines a utility function that focuses on finding images that are 1) under-represented in a class (i.e., images containing visual features of the class but are predicted as other classes with high confidence by the classifier), and 2) contain visual concepts that have led to the misclassification. Experiments on CUB, CUB-Families and Tiny Imagenet show better performance for the proposed approach as compared to previous works and baselines.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper presents a very nice demonstration of how explainability can be used to improve image classification systems.\n- The paper is very well written and easy to follow.\n- The ablation studies clearly highlight the importance of both of the main components of the proposed framework.\n\n\nWeaknesses:\n- There seems to be some mismatch between the motivation and the proposed approach. In Line 87, the authors claim that the queried set of images is typically noisy but then their proposed solution focuses on under-representation and visual features that lead to misclassification. Shouldn’t the solution just be to throw out the noisy data? They seem to be orthogonal to each other. Please clarify.\n- If there are two highly similar images in the queried set of images such that they both result in high utility scores, the proposed method would choose to add both to the training set, correct? This doesn’t seem ideal. Is there a way to enforce some diversity in the selected set of images to maximize the information gain?\n- I appreciate the experiment showing better generalization with BRACE as compared to the baselines. But, how does BRACE compare against previous works in this experiment?\n- The derivation of the set of concepts that caused the classifier to misclassify images of class c to c_bar as well as their overlap with the visual features in an image is specific to an interpretability method. I think it is okay to consider this issue as out of scope for this work but it would be good for the paper to discuss some initial ideas about how they can be generalized to a family of interpretability methods.\n- Where are the feature vectors extracted from that are used in equation 2?\n- Lines 64-66: How does GradCAM explain model decisions in terms of human-friendly concepts?\n- The citation for the RCNN model in line 155 is incorrect.\n\n\n-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\nI thank the authors for their response. It answers all my concerns.\n\nAfter reading other reviews and the authors’ responses to them, I think all the major concerns from the other reviewers have also been answered. \n\nIn my opinion, the paper proposes a significant contribution that would be useful to the Explainable AI (XAI) community and, therefore, I would like to see the paper getting accepted (hence, keeping my acceptance rating). \n\nI would encourage the authors to incorporate the additional experiments and the qualitative results they provided in the rebuttal period into the revised version (camera-ready, if accepted) of their paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper discusses their work’s societal impact satisfactorily. However, I think the paper can expand on the limitations a bit more, e.g., it might be difficult to derive the set of concepts that lead to misclassification for some types of interpretability methods, etc.", "review_text": "", "extra_scores": {"ethical_concerns": "I do not think that there are any ethical concerns related to this work.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper has novelty as it proposes a form of data augmentation that uses explanations to select external images from poorly performing classes. All five reviewers gave a 6 or 7 rating. The authors provided detail responses to reviewers' comments. The reviewers and AC reached the agreement to accept the paper. 
Decision: Accept (Poster)