Training Interpretable Convolutional Neural Networks towards Class-specific Filters

1 INTRODUCTION . Convolutional Neural Networks ( CNNs ) demonstrate extraordinary performance in various visual tasks ( Krizhevsky et al. , 2012 ; He et al. , 2016 ; Girshick , 2015 ; He et al. , 2017a ) . However , the strong expressive power of CNNs is still far from interpretable , which significantly limits its applications that require humans ’ trust or interaction , e.g . self-driving and medical image analysis ( Caruana et al. , 2015 ; Bojarski et al. , 2017 ) . In this paper , we argue that filter ambiguity is one of the most critical reasons that hampers the interpretability of CNNs . As a matter of fact , previous studies has shown that 1 ) filters in CNNs generally extract features of a mixture of various semantic concepts , including objects , parts , scenes , textures , materials and colors ( Zhang et al. , 2018b ; Bau et al. , 2017 ) ; and that 2 ) there is also redundant overlap between features extracted by different filters ( Prakash et al. , 2019 ) . The intricate many-to-many correspondence relationship between filters and features is so-called filter ambiguity as shown on the left of Figure 1 . Obviously , in high convolutional layers which might capture class-related feature , filter ambiguity contradicts our intention of an interpretable CNN , because it hinders humans from interpreting the concepts of a filter ( Zhang et al. , 2018b ) , which has been shown as an essential role in the visualization and analysis of networks ( Olah et al. , 2018 ) in human-machine collaborative systems ( Zhang et al. , 2017a ; c ) . Moreover , the unnecessary overlap between features extracted by different filters leads to under-utilization of a model ’ s expressiveness ( Prakash et al. , 2019 ) Therefore , reducing filter ambiguity is critical to obtain better feature with better interpretability and less redundancy . However , it is non-trivial to achieve such a goal barricaded by substantial challenges . First , most interpretability-related research simply focuses on post-hoc interpretation of filters ( Szegedy et al. , 2013 ; Bau et al. , 2017 ) , which manages to interpret the main semantic concepts captured by a filter but fails to alleviate the filter ambiguity prevalent in pretrained models . Second , many existing works such as VAEs ’ variants ( Higgins et al. , 2017 ; Burgess et al. , 2018 ; Kim & Mnih , 2018 ; Chen et al. , 2018 ; Kumar et al. , 2017 ) and InfoGAN ( Chen et al. , 2016 ) try to disentangle data representation and obtain better interpretability in an unsupervised way . However , it is proved that unsupervised learning on disentangled features without inductive bias is impossible ( Locatello et al. , 2018 ) , which challenges the works above . Considering the aforementioned challenges , we shed light on enforcing the one-class for one-filter relationship during training ( instead of post-hoc ) in a supervised manner only with classification labels . To this end , we propose a novel training strategy that coerces each filter into extracting features from only one or few classes for classification tasks , namely disentangling filters towards classspecific . Specifically , we design a Label Sensitive Gate ( LSG ) structure on the top of convolutional filters , which limits each filter ’ s activation only to its specific input label ( s ) . In our training process , we periodically insert LSG into the CNN and jointly minimize the classification cross-entropy and the sparsity of LSG , so as to keep the model ’ s performance on classification and meanwhile encourage class-specific filters . Experiments proved the LGS filter yield better representation for images hence leading to promising applications . Our training method makes data representation sparse and highly correlated with the labeled class , which not only illustrates the alleviation of filter ambiguity but also enhances the interpretability of the network . The advantages of our method are concretely substantiated by sparser correlation between filters , sparser filter-class correlation , better explanation for misclassification and more precise localization of labeled objects . Contributions The contributions of this work can be summarized as : ( 1 ) we propose a novel training strategy for CNNs which forces each filter to extract features mainly from only one or a few classes ; ( 2 ) moreover , we propose a metric to evaluate filter ambiguity , which can also be used as a regularization in our training to encourage class-specific filters ; ( 3 ) finally , our training enables filters to output sparse and class-related representation , which helps alleviate filter redundancy , improve interpretability of the model . 2 RELATED WORKS . Existing works related to our work include post-hoc filter interpretation , learning disentangled representation and model pruning . Post-hoc Interpretation for Filters is widely studied , which aims to interpret the patterns captured by filters in pretrained CNNs . Plenty of works visualize the pattern of a neuron as an image , which is the gradient ( Zeiler & Fergus , 2014 ; Mahendran & Vedaldi , 2015 ; Simonyan et al. , 2013 ) or accumulated gradient ( Mordvintsev et al. , 2015 ; Olah et al. , 2018 ) of a certain score about the activation of the neuron . Bau et al . ( 2017 ) determine the main visual patterns extracted by a convolutional filter by treating it as a pattern detector . Some other works transfer the representation in CNN into an explanatory graph ( Zhang et al. , 2017b ; 2018a ) or a decision tree ( Zhang et al. , 2019 ) , which aim to figure out the visual patterns of filters and the relationship between co-activated patterns . Posthoc filter interpretation helps to understand the main patterns of a filter but makes no change to the existing filter ambiguity of the pretrained models , while our work aims to train interpretable models . Learning Disentangled Representation refers to learning data representation that encodes different semantic information into different dimensions . As a principle , it proves impossible to learn disentangled representation without inductive bias ( Locatello et al. , 2018 ) . Unsupervised methods such as variants of VAEs ( Kingma & Welling , 2014 ) and InfoGAN ( Chen et al. , 2016 ) rely on regularization . VAEs ( Kingma & Welling , 2014 ) are modified into many variants ( Higgins et al. , 2017 ; Burgess et al. , 2018 ; Kim & Mnih , 2018 ; Chen et al. , 2018 ; Kumar et al. , 2017 ) , while their disentangling performance is sensitive to hyperparameters and random seeds . InfoGAN ( Chen et al. , 2016 ) learns disentangled hidden representations by maximizing the mutual information between input and generated images . Some other unsupervised methods rely on special network architectures including interpretable CNNs ( Zhang et al. , 2018b ) and CapsNet ( Sabour et al. , 2017 ) . As for supervised methods , Thomas et al . ( 2018 ) propose to disentangle with interaction with the environment ; Bouchacourt et al . ( 2018 ) apply weak supervision from grouping information , while our work applies weak supervision from classification labels . Model Pruning reduces structure redundancy and therefore is widely used for both model efficiency and better generalization , while our work uses it to alleviate filter ambiguity and improve interpretability . Many existing methods prune pretrained CNNs for better efficiency without significant deterioration in performance , including channel-wise pruning ( He et al. , 2017b ; Molchanov et al. , 2016 ; Liu et al. , 2017 ) and filter-wise pruning ( Luo et al. , 2017 ) . The methods mentioned imply that CNNs trained by standard training strategy tend to have redundant filters . Further research exploits redundant filters to improve generalization and suppress over-fitting by temporally pruning during training : Dense-Sparse-Dense ( Han et al. , 2016 ) encourages sparse weights by regularization and then recovers dense weights by removing the regularization ; RePr ( Prakash et al. , 2019 ) repeatedly prunes and reinitializes redundant filters to reduce filter overlap . 3 METHOD . Learning disentangled filters in CNNs alleviates filter ambiguity and meanwhile narrows the gap between human ’ s perception and CNN ’ s representations . In this section , we first present an ideal case of class-specific filters , which is a direction for our training , and then we elaborate on our method about how to train an interpretable network . 3.1 CLASS-SPECIFIC FILTERS . For CNNs in classification tasks , it is demonstrated that the filters in the last convolutional layer extract high-level features related to certain classes more or less ( Zeiler & Fergus , 2014 ) . We suppose the most ideal case , as shown in Figure 2 , is that each filter extracts features mainly from only one class , i.e . each filter is mainly relevant to one class . We call such filters ideally class-specific . Obviously , given the labeled class of an input , the classification performance changes little if we shut down the channels from the filters irrelevant to the class . This is the direction for our training method elaborated later . To have a rigorous definition of “ ideally class-specific ” , for a CNN , we use a matrixG ∈ { 0 , 1 } C×K to measure the relevance between filters and classes , where K is the number of filters , C is the number of classes , and each element Gkc represents the relevance between the k-th filter and c-th class , as shown in Figure 3 . The k-th filter extracts features mainly in the c-th class when and only when Gkc = 1 . Given the label y ∈ { 1 , 2 , ... , C } of an input , we can index a row Gy ∈ { 0 , 1 } K from the matrixG , which can be used as a gate multiplied to the feature maps to shut down those irrelevant channels . We call filters in a CNN as ideally class-specific filters , when the network ’ s prediction ỹG approaches that of original network ỹ after the feature maps from the last convolutional layer are multiplied by the gate Gy . 3.2 PROBLEM FORMULATION . In order to train a CNN towards disentangling filters to class-specific and meanwhile keep the original classification performance , as opposed to the standard training scheme that entangles filters , we introduce a Label-Sensitive Gate ( LSG ) path in addition to the standard path of forward propagation . In such a LSG path , some channels are shut down by multiplication with a learnable gate . This path ’ s classification performance is regarded as a regularization for disentanglement training . As shown in Figure 3 , let us denote θ as the network parameters . The network forward propagates in two paths : 1 ) the standard ( STD ) path predicting ỹθ , and 2 ) the LSG path with gate matrix G predicting ỹGθ where the learnable gate Gy for the input labeled with y is multiplied to the feature map before the linear layer . Initially , the elements in the gate matrix G for a CNN are unknown . Intuitively , we can search for them by exploring the binary space and find one solution that yields the best classification performance through LSG path , i.e. , to solve Φ0 ( θ ) = minG : Gkis one-hot CE ( y||ỹGθ ) , where the constraint on Gk applies to ∀k ∈ { 1 , 2 , ... , K } and Φ0 evaluates the performance of the CNN with filters disentangled . Therefore , it is natural to add Φ0 into training loss as a regularization that forces filters to be class-specific . Thus , we get the following formulation of the original problem to train a CNN towards ideally class-specific filters as min θ L0 ( θ ) = CE ( y||ỹθ ) + λΦ0 ( θ ) . ( 1 ) However , the problem is difficult to solve in practice . On the on hand , the assumption that each filter is ideally disentangled to extract only one class hardly holds , since it is usual for several classes to share one high-level feature in CNNs ; on the other hand , binary vectors in a non-continuous space are difficult to optimize with gradient descent . To address these two issues , we relax the constraints from two perspectives . First , we relax the one-hot vectorGk to sparse vector by L1 regularization ‖G‖1 where at least one element equals to 1 ( ∥∥Gk∥∥∞ = 1 ) . Since ||Gk||∞ = 1 ensures ‖G‖1 to be larger than K , we actually apply ∣∣ ‖G‖1− s∣∣ as the sparsity regularization term in order to make the norm of gate matrix close to a constant target norm s ≥ K. Second , we relax the binary vectorGk ∈ { 0 , 1 } C to a continuous vectorGk ∈ [ 0 , 1 ] C . Therefore Φ0 now becomes Φ ( θ ) = min G { CE ( y||ỹGθ ) + a ∣∣ ‖G‖1 − s∣∣ } s.t . ∥∥Gk∥∥∞ = 1 and Gkc ∈ [ 0 , 1 ] , ( 2 ) where a is a coefficient to balance classification and sparsity . Φ can be regarded as a metric of filter ambiguity , which means a CNN with higher class-specificity of filters will have a lower Φ ( contrary to common sense , here we expect Φ to be as small as possible ) . Replacing Φ0 in Equation 1 with Φ , we get an intermediate problem minθ ( CE ( y||ỹθ ) + λΦ ( θ ) ) . It is mathematically equivalent if we move minG within Φ to the left most ( see Appendix A for proof ) . Thus , we formulate a relaxed problem as min θ , G L ( θ , G ) = CE ( y||ỹθ ) + λ ( CE ( y||ỹGθ ) + a ∣∣ ‖G‖1 − s∣∣ ) , ( 3 ) s.t . ∥∥Gk∥∥∞ = 1 and Gkc ∈ [ 0 , 1 ] . The relaxed problem is easier to solve by jointly optimizing θ and G with gradient descent , compared to either the discrete optimization in the original problem , or the nested optimization in the intermediate problem . Solving the relaxed problem , we can obtain a CNN for classification with class-specific filters , where G precisely describes the correlation between filters and classes .

Summary: This paper proposed an interesting idea of using Label Sensitive Gate (LSG) structure to enforce models to learn disentangled filters for better interpretability of the DNN model. By periodically training with the sparse LSG structure, the model is forced to extract features from only a few classes. The model is trained efficiently in an alternate fashion (with respect to both the network parameters and the sparse gate matrix.) By disentangling the class-specific filters, the model becomes less redundant and more interpretable.