Toward Faithful Case-based Reasoning through Learning Prototypes in a Nearest Neighbor-friendly Space.

1 INTRODUCTION . With the ever-increasing usage of deep learning in real-world situations , interpretable and explainable machine learning has become more and more important . This is because the deeply complex nature and large number of parameters in deep learning models raise concerns about reliability , fairness , transparency , and trust . While some research has been focused on explaining the current ” black-box ” models by offering extra insight about their decisions ( Ribeiro et al. , 2016 ; Lundberg & Lee , 2017 ; Selvaraju et al. , 2017 ) , another approach is to create models that are inherently interpretable ( Rudin , 2019 ) . Some classes of machine learning models are deemed to be inherently more interpretable ( Doshi-Velez & Kim , 2017 ) . These include linear models like logistic regression , decision trees , rules-based models , naive bayes , and nearest neighbor models . These models are usually simple or abstract enough that a human can get an understanding of how they make specific decisions . Decisions made by nearest neighbor methods , in particular , can be easily understood by a human . But an important disadvantage of nearest neighbor models is that their inference computational cost scales with the amount of training data . This can make them impractical when the dataset is too large . One of the main ways suggested to combat the cost of nearest neighbor models is to use prototypes . Prototypes are points in the dataset that represent a group of points around them . If the number of prototypes is smaller than the number of data points , using them as a proxy of real data for nearest neighbor methods could significantly speed up the inference process . Depending on the number and the position of prototypes , performance differences compared to using the whole dataset can be minimal . Using prototypes for classification is not a new concept . Research was done on nearest prototype classifiers ( Bezdek & Castelaz , 1977 ; Kohonen , 1990 ; Seo et al. , 2003 ) and their validity ( Kuncheva & Bezdek , 1998 ; Graf et al. , 2009 ) . One important domain where these types of classifiers are used is in interpretable classification ( Bien & Tibshirani , 2011 ; Kim et al. , 2014 ; Chen et al. , 2018 ; Li et al. , 2018 ) . Recently Li et al . ( 2018 ) presented an explainable neural network structure that represents samples by their distance to some prototypes learned in an embedding space . A fully connected layer gets these distances as input and outputs the final class . It learns the prototypes along with the network weights . However , the drawback of that method is that it learns the prototypes in a poorly constructed embedding space . Because the structural similarities in the embedding space are not preserved , prototypes are less likely to be surrounded by a group of similar samples . Since these prototypes are the explanations for the model , naturally one may expect that the prototypes should represent unknown homogenous clusters that might exist within classes ( i.e . subclasses ) . For example different breeds of dogs within the class dog . We empirically show that Li et al . ( 2018 ) fails to preserve such subclasses and even destroys them . We argue that this is due to the lack of a mechanism for learning a proper embedding space in conjugation with learning the prototype positions . In this paper , we offer an interpretable prototype classification method called Deep Embedded Prototype Network ( DEPN ) . This method is a neural network architecture that finds a good embedding space and a number of suitable prototypes for classification . The embedding space and the prototypes in DEPN are constructed in such a way that high classification performance is achieved . In addition , the decisions of the classifier can be explained by showing the nearest prototypes as a comparative example . We show that the combination of our embedding space and prototype selection method leads to cohesive prototype explanations compared to the state-of-the-art Li et al . ( 2018 ) . The embedding space of DEPN is formed such that samples that are closer to each other in terms of Euclidean distance are more similar to each other than samples that are further away . It is because the embedding space is learned based on the nearest-neighbor rule . In particular , following ( Chauhan et al. , 2021 ) an upper bound of the classification error using nearest-neighbour classifier is minimized . While it is very hard to find annotated datasets containing similarity measures between each pair of samples , it is possible to create datasets that contain superclasses each consisting of multiple subclasses . Subclass similarity is , therefore , a good proxy for overall sample similarity . The advantage of ( Chauhan et al. , 2021 ) is that it has been shown to be able to create embedding spaces where even unlabeled subclasses of the data distribution are preserved in the embedding space . This happens despite the model not having any direct knowledge about subclass data . This means that for example , if one of the classes in the data consists of two subclasses , it is less likely to mix these two subclasses together in the resulting embedding space . For example , suppose that the problem is the classification between cats and dogs using only a picture of the animal . If the dataset consists of multiple dog and cat breeds , we can assume that it contains multiple subclasses of cats and dogs . If the nearest neighbors of a dog sample in the embedding space are other dog samples that belong to the same breed ( subclass ) , the closest samples to a prototype might also belong to the same subclass . We believe that if the subclasses of the closest prototype used to classify a new sample is the same as that of that sample , the prototype is a better explanation for the decision taken by the model compared to when the overall class label is correct , but the subclass shows a different breed of the animal . It is important to note that the method we provide does not have access to the subclass data . In fact , it doesn ’ t need the subclass labels to be present at all . The final result is a neural network that classifies the input using a number of prototypes . It can explain its decisions by offering the closest prototype of that class to the human user . Due to the way our embedding space preserves subclass information , the offered explanation is much more likely to be suitable compared to the previous state-of-the-art method Li et al . ( 2018 ) . In particular , our measurements show that our work preserves the subclass neighborhoods in the embedding space while Li et al . ( 2018 ) actively degrades them . This is while DEPN has the same overall performance when measured in superclass accuracy while showing , on average , 125 % improvement in subclass accuracy . There are previous works that might seem similar to our own . Deep k-Nearest-Neighbors ( DkNN ) by Papernot & McDaniel ( 2018 ) is an interpretability-based method for deep neural networks . This method uses nearest-neighbor labels in the intermediate layers of a neural network to assess the validity of a decision . The main idea behind DkNN is that when the network makes a decision , the nearest neighbors in the intermediate layers should also share the same label . If this is not the case for most of the layers , then maybe this decision is something that came up due to the quirks of the final layers of the network . While this focus on nearest neighbors might seem similar to our work , DkNN ultimately offers a confidence score for the decisions of the network . In contrast , our method offers examples ( prototypes ) as explanations for the decisions it takes . In fact , DkNN can be used alongside DEPN to offer confidence scores for its decisions . The work done by Li et al . ( 2018 ) is another method that is quite similar to our work . While the network architecture in both works is mostly the same ( especially if MsDNN-AE is used for the encoder ) , the way they utilize the architecture is quite different . This work is trained end-to-end , but DEPN uses a more gentle two-stage training procedure . The r1 loss used in this work does not consider the prototype labels while the one in DEPN does . This work uses the decoder outputs as the representations of its prototypes while the decoder in DEPN is an optional regularization system for MsDNN . If the decoder is not able to output good representations , this work will not be able to offer proper explanations for its decisions . The explanations in DEPN in contrast , are the nearest same class prototype , which will be represented by a training sample . Another difference is that the explanations offered by this method are much less likely to be of the same subclass as that of DEPN . The work done by Chen et al . ( 2018 ) offers prototype explanations-based on parts of an image . These explanations are inherently different from the sample based explanations DEPN offers . This method is also limited to data that can be split into parts and will not be feasible for other types of data ( like tabular data ) . 2 METHODOLOGY . Our method is a neural network with an uncommon structure suited for prototype training . It consists of a backbone encoder network that transforms the input to a metric space learned based on a nearest-neighbor rule . There is an optional decoder that will be used in an autoencoder which will get its inputs from the embedding space and reconstruct the original input . Then , we have n prototypes which will be modeled as a special layer on top of the encoder . The value of n is a hyperparameter . The larger the n , the more detailed and specific our explanations will become at the expense of more computational cost . Finally , we have a normal fully connected neural network layer that gets the output of the prototype layer and outputs the class likelihoods . This output layer can have a softmax activation function . The special prototype layer in our method gets the transformed samples from the embedding space as input and outputs a vector for each sample denoting the distance between that sample and each prototype in the embedding space . This arrangement was first proposed by Li et al . ( 2018 ) and lets us position the prototypes during the training process of the neural network itself . a schema of our network architecture is illustrated in Figure 2 . 2.1 TRAINING PROCEDURE . The training happens in two steps : The first step is to train the embedding space so that a good embedding space is formed at the output of the encoder . This is done by using the special MsDNN loss ( Chauhan et al. , 2021 ) on the embedding space . MsDNN loss is a variation of triplet loss ( Weinberger & Saul , 2009 ) that uses Gaussian estimations of the overall positive and negative samples as a substitute of the positive and negative samples themselves . It has been shown to be effective in preserving the unseen subclasses of the data and consequently is a good pick for our purpose . Optionally , a decoder that gets the transformed embeddings as inputs and reconstructs the original samples can also be used in this stage of the training , similar to MsDNNAE ( Chauhan et al. , 2021 ) . Training stops when accuracy using nearest-neighbor classification on a validation set is maximized . Our experiments show that unseen subclass accuracy , when measured by nearest-neighbor classification accuracy , generally follows the coarse-grained classification . This lets us optimize our embedding space using only the coarse-grained class data and still get within the general vicinity of the best settings for fine-grained subclass accuracy in the embedding space without having any subclass labels . The choice of hyperparameter σ in MsDNN is important to the viability of the created embedding space . This hyperparameters determines the size of the effective area for weighting the samples . Higher values would result in further samples being considered close neighbors for creating the positive and negative data point for MsDNN ’ s contrastive loss . We found that starting from small values and doubling the value of σ until the peak of coarse-grained validation accuracy to be a good procedure . Extremely large values of σ encourage subclasses to merge . We found out that fine-grained and coarse-grained accuracy follow the same pattern with respect to the value of σ . This means that coarse-grained validation accuracy is generally a good indicator of fine-grained accuracy as well . The second step of the training procedure trains the whole network in an end-to-end manner for classification but with the weights of the encoder network frozen . In essence , what this stage does is to train the prototype and the output layers of the network . The input samples are fed to the encoder and the output of the output layer is used for the final classification . The loss function used in this stage can be seen in equation 1 . L = λ1CEL ( ȳ , y ) + λ2r1 ( e , p ) + λ3r2 ( e , p ) ( 1 ) Where L is the loss value , CEL is the cross-entropy loss , ȳ represents the true coarse-grained labels of the input samples , y is the predicted output of the network , r1 and r2 are special losses defined on the prototype layer , e represents the transformed training samples in the embedding space , p represents the set of learned prototypes in the embedding space , and λ1 , λ2 , λ3 ∈ R+ are weights of each individual loss function . r1 and r2 are defined as : r1 = ∑ ei∈e min pj∈pci ( ‖pj − ei‖22 ) , pci = { pk ∈ p|∃n : yn = yi , ‖pk − en‖2 = minem∈e ( ‖pk − em‖2 ) } ( 2 ) r2 = ∑ pi∈p min ej∈e ( ‖pi − ej‖22 ) ( 3 ) Where ei is an element in e with index i , pci is the set of all prototypes that have the same class label as ei ( determined by the the label of the closest sample in the current batch to that prototype ) and pj is an element of pci . r1 is a loss function that sums up the distance between each input sample in the current batch and its nearest same class prototype . The class label of a prototype in this case is the class label of its nearest sample in the current batch . On the other hand , r2 sums up the distance between each prototype and its closest input sample in the current batch . Minimizing r2 will make sure that each prototype is close to at least one input sample . Minimizing r1 on the other hand makes sure that each sample is close to at least one prototype of the same class . Minimizing these two losses combined makes sure that the prototypes are well-positioned inside the distribution of the data . To increase the speed of the process , the initial position of the prototypes will be chosen as the position of a number of randomly selected training samples . Minimizing the overall loss function will position the prototypes in key positions of the data distribution and will train the output layer to use the distance of samples to each prototype to classify them . It is important to note that the freezing of the weights of the encoder ensures that these loss functions do not determine the embedding space itself and only use it to classify the samples using the prototypes . We found that not freezing the encoder will result in the destruction of the subclass neighborhoods in the embedding space . The stopping criteria in this stage of training are the plateauing of the loss function and the validation accuracy . In our experience , the latter happens sooner than the former . At this stage , the prototypes should be manually moved to the nearest sample in the training set . This sample is called the prototype sample and its class label is used as the class label of that prototype . At this point , the network is ready for inference . An optional procedure after this is to fine-tune the output layer using only cross-entropy loss due to the mentioned moving of the prototypes , but we saw no noticeable change in the output of the network even without this fine-tuning . A schema of the training procedures can be found in Figure 2.1 .

Summary: The paper introduces a novel an interpretable neural network for classification tasks that works by assigning input cases to prototypical cases based on their euclidean distance in an embedding space learned by the network. This is achieved by minimizing a loss function consisting of three components: cross entropy loss; distance between each case and the prototype with the same class, distance between each prototype and the closest case to it. The proposed method is specially useful when classification at fine-grained level is performed where in addition to the class to which an instance belongs, predicting the subclass is desirable as well. Experimental results are provided in the paper that validates the merits of the method compared to a set of variations of the proposed method and a state of the art approach as well.