Image Score: how to select useful samples

There has long been debates on how we could interpret neural networks and understand the decisions our models make . Specifically , why deep neural networks tend to be error-prone when dealing with samples that output low softmax scores . We present an efficient approach to measure the confidence of decision-making steps by statistically investigating each unit ’ s contribution to that decision . Instead of focusing on how the models react on datasets , we study the datasets themselves given a pre-trained model . Our approach is capable of assigning a score to each sample within a dataset that measures the frequency of occurrence of that sample ’ s chain of activation . We demonstrate with experiments that our method could select useful samples to improve deep neural networks in a semi-supervised leaning setting . 1 INTRODUCTION . Since its development two decades ago , Convolutional Neural Networks ( LeCun et al. , 1998 ) gradually dominate in tasks such as image classification and object detection . Various network architectures ( Krizhevsky et al. , 2012 ; Simonyan & Zisserman , 2014 ; He et al. , 2016 ) all prove the power of CNNs . On the other hand , many people believe that the end-to-end training approach makes the network a black box . However , recent efforts in visualizing ( Zeiler & Fergus , 2014 ; Mahendran & Vedaldi , 2015 ) and understanding ( Bau et al. , 2017 ; Koh & Liang , 2017 ) CNNs reveal that despite the large number of hidden parameters , convolutional networks show intrinsic patterns during training . The idea of explainable artificial intelligence suggests that in order to further improve deep models , we should first be able to understand the intrinsic structures that the models learned from training , i.e . why the model classify a certain image to a specific class , and how did the model make that decision . Recently more researchers focus on the aforementioned problem and try to interpret CNNs . The network dissection method , proposed by Bau et al. , is an efficient way to score unit interpretability in terms of trained CNNs operating on Broden dataset . The method considers each image ’ s activation map and defines an intersection over union score to quantify each concept ’ s interpretability . In another paper ( Zhou et al. , 2014 ) , it is suggested that each individual unit behaves as detectors in a network trained to classify scenes . There are also papers that employ inverse method to study networks ( Nguyen et al. , 2016 ) , where the authors suggest that instead of trying to interpret CNNs , we should synthesize images base on the models , and interpretability of networks could be judged by the quality of synthesized images . Beyond simply analyzing convolutional neural networks , there are also models designed to learn a better representation of semantic meanings , which are encoded by specificity and generality of neurons in each layer ( Yosinski et al. , 2014 ) . Dynamic routing between capsules ( Sabour et al. , 2017 ) could parse images into capsules where each capsule encodes a specific semantic meaning . Zhang et al . proposed an interpretable convolutional neural network where high convolutional layers represent certain object parts ( Zhang et al. , 2018 ) . However , almost all of the existed works investigate only the models and ignore the relationship between models and samples . The network dissection method ( Bau et al. , 2017 ) focuses on the amount of units that could be interpreted , while the neural activation constellation method ( Simon & Rodner , 2015 ) could learn part models in an unsupervised manner . Different than the above-mentioned approached , we believe that sampling distributions , i.e . datasets , encode their own features ; and that no network would work on all distributions . As a consequence , relationships between models and samples should examined more carefully . Unlike traditional supervised learning that requires fully labeled data to train a classifier , semisupervised learning requires only a portion of data to be labeled , which demands less effort in data preparation and is more practical in real life . Consider a semi-supervised leaning task where both the labeled and the unlabeled data come from the same sampling distribution . A straightforward technique is that we first train a model on the labeled data , and then use this model to classify the unlabeled parts of the dataset to enlarge the training set . After this , all the data are labeled regardless of correctness . The model could then be re-trained by iteratively performing the above steps . However , experiments have shown that this approach does not work well on CNNs ( see section 3 for details ) . The reason is two-fold : • Parts of the data are labeled by our model correctly with extreme confidence , i.e . probability generated by softmax layer is close to 1 . This could happen only if our model already learned the features during training , therefore adding these data into the training set will not provide any improvement . • The model does not pick up some of the features that are contained in the unlabeled data during training . Therefore , those data are doomed to be labeled incorrectly and could potentially weaken our model upon added into the training set . To summarize these two problems : could we use a particular sample to improve our model ; and could we trust our model to label a particular sample correctly . We propose a method to automatically assign scores to data that represents both confidence ( could we trust it ) and interpretability ( could we use it ) based on a pre-trained network . Our approach mainly focuses on activations of individual neurons within a network . For a pre-trained CNN and some images within one category , we focus on images ’ activation maps generated by all of the convolutional and linear layers . The activation maps contain information about each image ’ s relationship with the activated neurons . Then our method could identify neurons that are commonly activated and thereby assigning scores to each individual image based on it ’ s chain of activation . Samples with high scores are assumed to be interpretable and experiments have shown that they are also trustworthy . This entire procedure is unsupervised , i.e . our approach does not need label information to assign image scores . Therefore , we could apply our method to semi-supervised learning tasks including image classification and object detection . Structure of this paper is the following : in section 2 we formally define image scores and then examine some key properties of our approach ; in section 3 we conduct experiments in a semisupervised learning setting to demonstrate how scoring the images could help with improving a pre-trained model , experiments are done in image classification and object detection tasks ; section 4 concludes the entire work . 2 APPROACH . 2.1 INTUITION . Given a pre-trained model and a dataset , we could say that the dataset consists of “ good ” and “ bad ” images in the sense of interpretability ( Figure 1 ) . While filters in good images could learn object themselves , filters in bad images would easily be confused with image background information , as shown in Figure 1 ( second row ; first , third , fourth , sixth , and seventh from the left ) . It is also possible that filters in bad images could not pick up feasible features at all , like these in Figure 1 ( second row ; second , and fifth from the left ) . In either of these two cases , we say the image could not be interpreted by the current model , and thus we would assign a relatively low score . By this definition of interpretability , if an image is assigned a high score , it ’ s chain of activation should include little to no background information ( see Figure 1 ; fourth row ) ; this makes the connection to confidence in that these images should be classified correctly . Based on the principles above , we propose the image score method to measure interpretability of individual images give a pre-trained model . The intuition is simple : all correctly classified images should have similar chain of activation , while incorrectly classified images should have very different activations both within themselves and with correctly classified images . 2.2 IMAGE SCORE . Consider image set I that contains all the images in one specific class . Denote I ∈ I an image . For each image I and layer l in a pre-trained CNN , we have a corresponding activation map XI , l . By saying layer , we mean a collection of functions that consist of a convolution function for convolutional layers or a linear function for classification layers , ( potentially ) a normalization function , an activation function , and ( potentially ) a pooling function . By saying activation map , we mean the tensor generated by a specific layer . LetX l = ∑ I∈I X I , l and threshold tl = max ( X l ) /2 . The idea here is that if a neuron is commonly activated when processing a class of images I , sum of activation values across I should be large in comparison with neurons that are not commonly activated . We choose the threshold to be related to L∞ norm , but this could also be done with other norms such as L1 and L2 . Rather than the activation values , we care more about whether a specific neuron is activated by some images at all . This is because after the softmax layer , we pick the index of the maximum element to be our prediction , and it doesn ’ t matter whether this maximum value is 1 or 0.101 as long as it is the largest . We create a binary mask of activation to reflect this . Denote M l the “ correct ” activation mask of layer l , and each element M li of M l satisfies M li = 1 { X li ≥ tl } , i.e . elements in binary mask M l is 1 if and only if the corresponding activation value in X l is greater than the threshold . Note that M l is a binary mask for all the images I ∈ I and it contains the information of “ important ” activations . Similarly , denote M I , l the activation mask for image I in layer l and M I , li = 1 { X I , l i ≥ tl } . As mentioned earlier , the intuition is that if some images are labeled correctly , they are classified accurately in a similar fashion . The rest of the job is simply to compare each image ’ s activation mask M I , l and the “ correct ” activation mask M l we constructed . For each image I , define score sI as sI = ∑ l log ( ∑ iM I , l i∑ iM l i ) The image score sI we defined represents similarity between an image ’ s chain of activation and the “ correct ” chain of activation . The “ ground truth ” that we established in turn reflects common activations among all images . 2.3 PROPERTIES OF IMAGE SCORES . Convolutional neural networks are susceptible to bias such that even if the model achieve good classification performance , it still encodes biased information . More often than not , a CNN might extract image background information to perfect itself , i.e . it may identify cloud as a feature when classifying airplanes . The encoded biased information is beneficial during training but problematic during testing since the biased information in training set and testing set might not be the same . As a matter of fact , this is why if we test on the training set , performance will supersede actual accuracy significantly , as shown in Figure 2 ( left ) . Yet regardless of dataset bias , both training accuracy and testing accuracy increase as image score increases . On the other hand , observe that the number of training and testing images fall into any interval of image scores do not show major difference . If we consider our model to be a classifier , as training proceeds more and more bias information will be introduced , making classification more fine-grained . As a result , most of the images will have minor differences in activation maps since we expect bias of each image to be different . Therefore , we expect to see less images to have very high scores , as illustrated in Figure 2-4 . In the meantime we would expect images in all intervals to gain accuracy until the loss function convergences , which is also revealed in the above figures .

Summary: This paper proposes a new metric called the image score that compares the similarity of activation between a given image with a pool of groundtruth images. The paper finds it useful for semi-supervised learning with self-teaching, where the network picks the most confident sample and use the network prediction as the label. It finds that the proposed method is better than 1) not using the unlabeled data and 2) using softmax as an indicator for model prediction certainty.