Title: SEMANTIC PRUNING FOR SINGLE CLASS INTER-PRETABILITY

Abstract: Convolutional Neural Networks (CNN) have achieved state-of-the-art performance in different computer vision tasks, but at a price of being computationally and power intensive. At the same time, only a few attempts were made toward a deeper understanding of CNNs. In this work, we propose to use semantic pruning technique toward not only CNN optimization but also as a way toward getting some insight information on convolutional filters correlation and interference. We start with a pre-trained network and prune it until it behaves as a single class classifier for a selected class. Unlike the more traditional approaches which apply retraining to the pruned CNN, the proposed semantic pruning does not use retraining. Conducted experiments showed that a) for each class there is a pruning ration which allows removing filters with either an increase or no loss of classification accuracy, b) pruning can improve the interference between filters used for classification of different classes c) effect between classification accuracy and correlation between pruned filters groups specific for different classes.

1. INTRODUCTION:

Convolutional Neural Networks (CNN) provides high quality features that are used in various tasks such as object recognition (Ouyang et al., 2015; Wang et al., 2015; Lin et al., 2017a; Hu et al., 2017) , scene classification (Zhou et al., 2014; Ren & Li, 2015) , semantic segmentation Girshick et al. (2014) ; Husain et al. (2017) , image captioning (Anderson et al., 2017; Rennie et al., 2017; Yao et al., 2017; Lu et al., 2017) , raw audio generation (van den Oord et al., 2016) , data analytics (Najafabadi et al., 2015) . Unlike engineered features, where one or multiple features have been designed with specific domain knowledge (SIFT (Lowe, 1999) ,SURF (Bay et al., 2008) , LBP (Ojala et al., 1994) , Canny (Canny, 1986) , etc.), features in CNN are obtained by directly learning representations (filters) from the data without explicit domain expertise.

While using a large number of learned filters in CNNs creates high-quality features, it also requires significant computational resources. The filter learning and the inference processes need resources such as GPU or TPUs to tackle the computational overhead. Recently, multiple optimization methods were proposed, with the target to reduce the computational overhead of either the learning or the inference processes. For instance, in the learning optimization approach, the following works have been done (Zhang et al., 2018; Lin et al., 2017b) . On the other hand, the number of the current works targeting the optimization of the inference are: reduced precision computation (Rastegari et al., 2016; Courbariaux et al., 2016; Zhou et al., 2016) , look-up and precomputing (Abdiyeva et al., 2018; Brendel & Bethge, 2019) , pruning (Anwar et al., 2015; Li et al., 2016; Zhu & Gupta, 2017; Raghu et al., 2017; Morcos et al., 2018; Ma et al., 2019) and knowledge distillation (Hinton et al., 2015) .

The work described in this paper continues in the direction of inference optimization by pruning. However, the pruning is only used as a tool for studying the interpretability of CNNs filter. Pruning is the process of removing redundant filters from the network (Anwar et al., 2015; Li et al., 2016; Zhang et al., 2017; Zylberberg, 2017; Zhu & Gupta, 2017; Raghu et al., 2017; Morcos et al., 2018; Ma et al., 2019) . In current approaches, the pruning process consists of the following steps: train CNN, remove redundant filters and retrains the network to preserve the original accuracy. The proposed pruning method, however, does not involve retraining of the pruned network. The motivation of this work is the interpretation of CNNs: pruned and unpruned filters are analyzed to evaluate class-wise relations and dependencies. In particular, the pruning applied to AlexNet is used to explore and study whether one can obtain a new single class classifier by dropping the selected filters in the original multi-class classifier instead of an expensive training process of a new classifier.

The interpretability of the network resulting from pruning can be seen as a filter-wise and signalwise network dissection: The removal of filters in a controlled manner allows us to determine how different classified objects affects other classes, how the density of the classes in the learned feature space affects the accuracy of a classification and how the structure of the learned multi-class feature space allows to improve the accuracy of a single-class classification. Therefore, the pruning used in this work can also be seen as an approach to CNN understanding (Bau et al., 2017; Zylberberg, 2017; Kindermans et al., 2017; Morcos et al., 2018) .

The proposed method, referred to as semantic pruning, is performed as follows: we start from a CNN trained to distinguish q classes. Then we reduce the multi-class CNN to the single class classification problem, by removing unrelated to the selected class labels. To remove filters, we measure each filter's contribution (activation magnitude) toward recognition of a target object class. Filters with the highest contributions were preserved, and the remaining filters were dropped from the network. The pruned and unpruned filters are analyzed for correlation, overlap and other measures allowing us to determine class-wise interactions.

The contribution of this paper can be summarized as the following:

• investigated the sensitivity of object classification to semantic pruning,

• study of the interference between class wise filters in CNN, This paper is organized as follows. Section 2 introduces semantic pruning concept. Section 3 describes the experiments and the results and Section 4 concludes the findings.

2. PROPOSED SEMANTIC PRUNING:

Let F i ∈ R M×M×D be a filter in layer j of a convolutional network. Let, B j ∈ R X×Y ×D be an input tensor to the layer j. X and Y are spatial dimensions. To calculate the output tensor B j+1 of the layer j input tensor B j is convolved with the set of filters L = {F 1 ..., F D ′ } of layer j. Lets denote the output of the convolution operation between input tensor B j and a single filter F i as R i : Definition 1 (Pruning). Let L = {F 1 , . . . , F D ′ } be a convolutional layer, F i ∈ R M×M×D be a single filter of the layer with an input tensor B j ∈ R X×Y ×D . B j+1 = L ⊙ B j (eq. 1) with B j+1 ∈ R X ′ ×Y ′ ×D ′ . Pruning layer L denoted as Ω(L) is the process of removing one or more

R i = F i ⊙ B j (1) Then B j+1 (:,:,i) = R i (a) (b)

F i from L resulting in a new set of filters Ω(L) = L ′ = {F 1 , . . . , F D ′′ }. Hence, a new output is B ′ j+1 = L ′ ⊙ B j with B ′ j+1 ∈ R X ′ ×Y ′ ×D ′′ such that D ′′ ≤ D ′ .

In this work, we propose a semantic pruning method that is applied with respect to each selected class separately. We determine the amount of pruned filters for each object class λ i ∈ Λ = {λ 1 , . . . , λ q } where q is the number of classes in the dataset. To determine which filters can be removed, the process of pruning requires a measure of contribution. In this work, we propose to use activation magnitude as filter removal criteria.

Definition 2 (Activation Magnitude r). Let R ∈ R X ′ ×Y ′ be a result of convolution between input tensor and a single filter in the layer, then activation magnitude for each filter F i is computed as follows:

ri = X ′ x=1 Y ′ y=1 R x,y,i (2) with r ∈ R 1×1×D ′ .

The schema for calculating r is shown in Figure 1b .

We propose to perform pruning for each class separately; hence, we split the dataset into subsets of images belonging to the same class. Let D = {δ 1 , . . . , δ n } be the dataset containing the n data samples. Let Λ = {λ 1 , . . . , λ q } be the number of object classes in the dataset. Then, D λc ⊂ D be a subset of D containing all data samples with class label λ c . The number of samples in D λc is denoted as p, such that p ≤ n. Let ri (δ o ) be the activation magnitude of a filter F i resulting from using input sample δ o ∈ D λc . We want to measure the response of each filter w.r.t. to the individual class. Therefore, we propose to accumulate the activation magnitudes of all samples belonging to the selected class. The resultant value is denoted as Accumulated Response. Definition 3 (Accumulated Response γ). ri λc is the Accumulated Response for filter F i and object class λ c and defined as shown in eq. 3

EQUATION

The process of semantic pruning starts by separating the dataset D into subsets D λc = {δ 1 , . . . , δ p } with p being the number of samples in the subset D λi , for object label λ c ∈ Λ = {λ 1 , . . . , λ q } with q being the number of object classes. Let m be the total number of filters in AlexNet. We calculate the accumulated response rλc = {r λc 1 , . . . , rλc m } for each filter in the network and for selected number of object classes from the dataset. The percentage of pruned filters is represented by the parameter θ (pruning ratio) such that θ = [0, . . . , 1]. For instance, for θ = 0.1, the 10% of filters with lowest accumulated response will be pruned. In this work, let F σ and F σ be the set of unpruned and pruned filters for a given γ respectively. The process of filter pruning can be performed using two different approaches: network wise and layer-wise. Network wise pruning (NWP) is performed on filters F from all layers of the network. Filters are sorted according to their corresponding accumulated responses, and the filters with the smallest values are removed in correspondence to the pruning ratio θ. On the other hand, Layer wise pruning (LWP) separates filters into groups, each containing filters from one layer. The pruning is then applied to each layer separately. However, as a result of removing certain filters, the activation response for the filters in the consecutive layers can get affected, and therefore, higher potential of pruning can be achieved. Therefore, we also provided a comparison between these two cases, referred to as Direct and Residual Pruning.

Two settings were used toward varying-parameter θ . These approaches will be referred to as discrete and continuous pruning. For discrete pruning, the values are changes with the fixed step size, while for continues pruning the wider range of values is explored.

3. EXPERIMENTS:

To illustrate the effectiveness of the semantic pruning approach, we applied the method on the model trained for an object classification task. In our experiments we used AlexNet (Krizhevsky et al., 2012) trained on ImageNet (Russakovsky et al., 2015) dataset. The AlexNet network consists of five convolutional and three fully connected layers. Fully connected layers are referred to as classifier. The convolutional layers are referred to as feature extractor.

The ImageNet data set contains 1000 object classes; however, due to computational constraints, we evaluated our method on randomly chosen 50 classes, to show the further potential of the method. The experimental section is organized as follows:

• Sensitivity to pruning • Study of the Filter Interference

3.1. SENSITIVITY TO PRUNING:

We started our study by firstly evaluating the sensitivity of the single class classifier for pruning.

In particular, we started with an evaluation of the discrete pruning approach. We applied it to both layer-wise (LWP) and network-wise pruning(NWP).

3.1.1. DISCRETE PRUNING:

The process of semantic pruning starts by separating the dataset into subsets of images belonging to the selected object label. Let m be the total number of filters in AlexNet. We calculated accumulated responses γ for all m filters in the network. Then we sorted the values in the descending order and applied four different settings to access the effect of discrete pruning:

• 0 score -removing all filters with γ ≤ 0 • 0.33 -removing 33% of Filters with lowest value of γ • 0.66 -removing 66% of Filters with lowest value of γ

• median (M) -Filters with γ value lower than the median of all γ were removed.

The procedure was repeated for all selected 50 classes separately.

Figure 2 shows the effect of discrete pruning on the number of parameters removed and preserved.

Experiments indicated that pruning filters with NWP-0 and NWP-33 affect the average network accuracy by around 10% (Figure 2 ), while layer-wise pruning looks to be more resistant toward higher ratios. However, both the LWP-M and NWP-66/LWP-66 result in a serious drop of network accuracy (green, red and purple lines in Figure 3 ), and the difference between layer-wise and network-wise pruning are not as large as expected (red and purple lines).

Figure 2 also shows that removing even the least active layer-wise filters reduces the general network accuracy. This can be seen when comparing NWP-33 and LWP-33 in Figure 2 and 3 respectively. However, it can also be observed that the classification of certain classes becomes more accurate while other classes suffer much steeper loss in classification accuracy for LWP-33 (Figure 3 yellow curve).

Finally, note that no normalization of activity measure was used. The reason for not using normalization is due to the fact that we are evaluating pruning with respect to one class at a time. Therefore, normalizing the activity with respect to the whole data set would result in sample dependent pruning and therefore remove the desired class wise pruning result. Additionally, normalizing with respect to one class at a time will introduce different minimal and maximal filter activities making a magnitude-based comparison between classes meaningless. Therefore, simply taking into account the NWP or LWP allows to account for both these effects and still provide reasonable information about the pruned and unpruned filters. In Figure 4 the Direct Pruning (DP) represents the number of filters removed by the semantic pruning method. The Residual Pruning (RP) represents the pruning resulting from the DP pruning. The RP pruning represents the filters that became inactive on average because their input becomes 0 on average after the DP pruning. Observe that similarly to the previous experiment, filters in 0 th layer do not get pruned at all. Similarly, filters in the 4 th layer do not get pruned as well at low θ. Filters from the 1 st and 2 nd layer is the one most pruned in DP while the filters of the 3 rd layer are pruned as a consequence in RP. Additionally, notice that at 50% of pruning ration almost all filters from the 2 nd layer are removed as a result of pruning. It is very interesting to see that even with higher pruning ratios, the 0 th layer is not pruned at all and the 4 th layer is pruned only at higher values of θ. This means that the encoding provided by the first and last layer seems to be the most crucial and the densest.

3.1.2. CONTINUOUS PRUNING:

The experiments with discrete pruning showed some important information about structural relationships of filters and the accuracy of classification of certain classes. However this information is only partial complete and therefore a continuous process of pruning was also evaluated with respect to individual class accuracies. In the continuous pruning, θ was set to 0 and was increased by 0.1 until reaching θ = 0.6 (representing 60% pruning ratio). Figure 5 show the plot of class-wise accuracy for twelve selected object labels. It can be noticed that:

• Several classes are not affected by the pruning unless a very large ratio (θ ≥ 0.15) of filters is pruned (Figure 5e-5h ). • Additionally, several classified classes has its accuracy increased after pruning. This is quite remarkable as this also entails that there can be interference between filters. (Figure 5a-5d ).

3.2. STUDY OF THE INTERFERENCE:

Semantic pruning showed that pruning least active filters from one class can result in increased accuracy of classification. The possible explanation for the phenomena is filters interference within an overcrowded region of the feature space.

To investigate these phenomena in more details, we decided to investigate how many different and shared filters are pruned, and how the pruning affected the class density within the feature space. In order to determine the overlap between object classes, inter-class feature overlap was computed by calculating the intersection of unpruned F σ and pruned F σ sets of filters. Let two object classes λ 1 and λ 2 be represented by F λ1 σ , F λ1 σ and F λ2 σ , F λ2 σ respectively. Then let ρ σ = F λ1 σ ∩ F λ2 σ and ρσ = F λ1 σ ∩ F λ2 σ be a double measure filter similarity. The reason for using both F σ and F σ is to utilize two sources of information. In other words, we want to determine similarities of features which are required, and what features are not required by the object class. Higher is the value of F σ higher the similarity of filters used for classification of λ 1 and λ 2 .

Figure 6 provides the correlation between pruned and un-pruned filters for the three classes. (Only three classes were selected due to the space limitations.) All three classes represent different behaviour toward pruning: performance improvement, pruning resistant and accuracy degradation.

The general trend for all three graphs for non-pruned filters is that the correlation between filters is constantly decreasing, as one can expect. More interesting trends, however, can be observed for the correlation between pruned filters.

The first object class we would look at is jigsaw puzzle. According to Figure 5c , there is a significant effect of the pruning on the classification accuracy of the jigsaw puzzle class. Possible explanation can be provided by the fact that the classes-neighbours (Figure 6a and Figure 6d ) for this class are semantically unrelated. Hence, the reduction in a correlation of pruned filters is reducing the interference with class-neighbours filters (θ ∈ {0.075, 0.2}), which match the region of the sudden jump in accuracy according to Figure 5c . It is also can be noticed that for θ = 0.3 semantically unrelated classes started to have increased in pruned-filter correlation to the jigsaw puzzle class, which reflected in the sudden accuracy drop in Figure 5c . A similar trend can be observed for another class jeep. At the region where θ ∈ {0.05, 0.15} , we can see that correlation with more semantically related classes: beach vagon, recreational vehicle, is increasing, while sax and jigsaw puzzle is decreasing. It is possible to assume that, the effect of interference from unrelated filters is reducing, and as a result, improves the model accuracy, as shown in Figure 5h . The last example is provided for the class from the group, which was negatively affected by the pruning (Figure 5k ).

Looking at Figure 6f we can see that lighter class has a high correlation for pruned filters with neighbouring classes; hence the effect of interference from the class-neighbours was not negated, which is reflected in the accuracy drop. However, one can observe that during the drop in pruned filter correlation for θ ∈ {0.15, 0.21} there is a sudden jump in classification accuracy (Figure 5k ).

More extended evaluation is provided in Figure 7 for the class apron. According to Figure 7b , at the pruning rations θ = [0.1, 0.2] the correlation between pruned filters is increasing with such classes as hoopskirt, Christmas stoking and miniskirt, while the correlation is reducing for volleyball and sax. Semantically, the first three classes are more related to the class apron than the last 2. In such a way, at pruning ratio, θ = [0.1, 0.2] filters would be more associated with the related classes, which at the same time should positively affect the performance. That is what can be observed in Figure 5e . There is a performance jump in that region for θ = [0.1, 0.2]. Further, the point can be supported by calculating the accumulated responses γ apron = {γ apron 1 , • • • , γ apron q } for all filters in AlexNet and mapping them to 2D space through PCA dimensionality reduction. 2D projections (Figure 7c ) shows that at the pruning ratio 0.15 (which is in range of 0.1-0.2) class apron lay closer to the correlated classes, and moving away from unrelated "sax" and "volleyball" classes. While at the pruning ratio 0.4, it got moved to the unrelated classes again, which also affected the model accuracy.

The general observation from this experiment leads to the conclusion that the objects are represented in the feature space in areas of different density. Interestingly, the accuracy of classes was improving for the pruning ratios at which the correlation of pruned filters with semantically related classes was increasing, while the correlation with semantically unrelated filters was decreasing. These observations are supporting the hypothesis about filter interference and giving a promising direction for further investigation and potential classification accuracy improvements. Conducted experiments showed an interesting observation about the correlation between accuracy and class-wise similarity between pruned filters F σ .

4. CONCLUSIONS:

We presented a study of semantic CNN pruning. The results indicate that classification classes are asymmetrically represented by filters resulting in the fact that some object classes have their classification accuracy increased when pruned for. We also observed that objects classes represented in densely populated parts of the feature space share very similar filters. Such object classes when pruned for the least active filters (small γ) have a tendency to affect close neighbouring classes more directly and therefore, the classification accuracy of the pruned object increases. Additionally, we also observed that the filters in the lowest layer are rarely pruned, similarly to the first layer. Finally, we observed that there is an interference between filters affecting classification accuracy, and there is a correlation between accuracy and class-wise similarity between pruned filters. As future work, we plan to extend our work to more classes and more models. In addition, we will study the effect of filter overlap on convolutional networks safety and adversarial examples phenomena.

:



Figures and Tables:

Figure 1: (a) Convolutional layer operation and (b) computation of the filter Activation Magnitude.

Figure 2: AlexNet average accuracy for different pruning rations θ Figure 3: Comparison of classification accuracy between unpruned AlexNet and pruned using NWP-M, LWP-33, NWP-66, LWP-66.

Figure 4 shows how the pruning affects the individual layers of filters in the feature extractor. Figure 4a shows the relative amount of pruned filters at ratio of 5%, Figure 4b at 25% and Figure 4c at 50%. The x axis indicates the layer and y axis the mount of filters.

Figure 5: Examples of (a)-(d) object classes that improved for pruning ratio θ > 0, (e)-(h) that for certain ratio θ > 0 have same accuracy of classification as when θ = 0 and (i)-(l) object classes for which classification accuracy was always worse than when classifying when θ = 0. x-axis -θ, y-axis -accuracy.

Figure 6: Correlation of filters (a)-(c) left after pruning, (d)-(f) removed after pruning

Peer review (iclr2020):
Summary --- (motivation) This paper proposes a new approach to pruning activations from neural networks, but uses it to understand neural nets better rather than trying to make them more efficient. It prunes filters per-class 1) to measure how sensitive image classes are to pruning and 2) to measure how similar classes are by comparing the filters they prune. (approach) Using only images from class c, CNN pre-activations are aggregated across spatial dimensions and examples. This gives an average feature vector for that class. T percent of neurons are pruned. Features with lower pre-activations (possibly negative with large magnitude) are pruned first with successively higher activations pruned later. No re-training is performed. (experiments) Experiments use AlexNet and 50 of the 1000 ImageNet challenge classes to show: 1. Pruning filters results in decreased accuracy with smaller decreases in accuracy as the first and last few filters are pruned. 2. More filters from middle layers are pruned than are those from early and late layers. 3. After some filters have been pruned, sometimes pruning can increase accuracy. 4. I could not understand section 3.2. Strengths --- The proposed pruning method is simple and efficient. I like the broad goal of understanding CNNs. We could use more papers that just focus on analysis like this one. I think the idea of class conditional pruning is novel. Weaknesses --- # Major Weaknesses I don*t see why these results are significant. I do not find these results very surprising (see next comment) and I do not see why the community will find them useful. * In particular, consider the conclusion. Sentences 2, 3, 5, and 6 seem to all be observations about what happened in the experiments. The conclusion should re-iterate the results, but it should also say why they are important. How does the work relate to the goals of the community at large? Which goals? Will this enable important new capabilities? What general concepts did we learn from this that we didn*t know before? Some of the positions in the paper could be more carefully considered. There are alternate explanations for many of these phenomena. * Pruning smallest activations doesn*t make sense to me. Typically redundant or un*important* activations are pruned because doing so has negligible impact on accuracy. The smallest activations are pruned here. Should the smallest activations be redundant or unimportant in some way? Does this rely on ReLU activations following the pre-activations? I think these values are not necessarily redundant and could be quite important (e.g., as measured by some saliency explanation like Integrated Gradients), but I could be wrong. It would be useful to provide a baseline which removes highest instead of lowest activations. * Much of the surprise about figure 5 seems to be because it is not monotonic but it was expected to be monotonic. I agree that these curves should generally go down as theta increases, but I don*t see why that relationship should be strict. I would be surprised if activations were not in some way dependent on one another. Furthermore, why does that dependence have to be interference? Couldn*t it also be that some activations are complementary (and thus ineffective when only one is present)? * Does Network Wise Pruning (NWP) favor more layers than others? It may be that some filters have higher Accumulated Responses per filter simply because there are more feature maps in the previous layer (thus more things summed up) and not because of what information they capture or their relationships with other filters. Does this happen? This could be an alternative to the following conclusion: *This means that the encoding provided by the first and last layer seems to be the most crucial and the densest.* # Other General Weaknesses: * AlexNet is a rather old architecture to use for this analysis, so I can*t be confident these methods or behaviors will generalize to other architectures. Does it hold for more modern architectures? # Missing details / Points of confusion: * The paper says the correlation from Figure 6 should be expected to decrease as theta increases. Why should this be expected? * What exactly does Figure 6 measure? I think it*s the ratio of the size of the intersection rho_sigma to the size of the union of the same two sets. However, the text calls the metric *correlation.* This should be made clearer. # Minor presentation weaknesses Some parts of the notation/explanation don*t make sense to me: * *Let Lambda=... be the number of object classes in the dataset.* But Lambda is defined as a set, not a numeral. * The number p needs more context/subscripts as it depends on class c and filter i. * *? = [0, . . . , 1]* This defines theta as a finite set, but I think it*s meant to be a number in the interval range (0, 1). * Why is lambda_c needed instead of just using the index c to specify a class? The variable lambda doesn*t seem to be any different than a class index. * *let F? and F? ? be the set of unpruned and pruned filters for a given ?...* I don*t see how this works. According to eq. 3 gamma depends on a particular filter i and class c, so the only filter F could contain is the ith one. (Later it became clear that F was only mean to class conditional, not filter conditional.) Figure 5: These plots shoul all have the same y axis range. This would make them comparable and allow readers to much more easily compare trends across classes. Similar steps should be taken so the same range is used any time theta is plotted on the x axis. Figure 3: I find it hard to get an overall ordering of the approaches in this figure because there is so much variance from class to class. It effectively conveys the variance, but I*d also like to know what the means across classes are for each method so I can compare the proposed approaches more effectively. *The results indicate that classification classes are asymmetrically represented by filters resulting in the fact that some object classes have their classification accuracy increased when pruned for.* * I*m not sure what it means to be asymmetrically represented by filters. Suggestions --- This analysis would have been more interesting with an existing pruning approach because we would already know that such an approach is good a removing unnecessary filters. Final Evaluation --- Quality: Experiments could have been cleaner, but they basically demonstrate the patterns the paper intended to show. Clarity: I could understand most experiments at a high level, but I found it hard to understand the motivation and the rest of the experiments. Significance: As explained above, I do not see why the paper is significant. Originality: The experiments and the proposed class conditional pruning approach are somewhat novel. The paper is somewhat novel, but do not find it very clear and I do not see why it is important, so I cannot reccomend it for acceptance.