Beyond Examples: Constructing Explanation Space for Explaining Prototypes

1 INTRODUCTION . `` Explanation '' is essentially based on understanding . When a fact is in doubt , an explanation is provided to resolve a question . Therefore , explanation must be human-interpretable and reliable . An explanation that is not human-interpretable does not resolve the issue , and an explanation that is not reliable may create confusion . In practice , it is difficult to find an explanation that perfectly satisfies even one condition . In this situation , when an insufficient explanation is given , a use may wonder why the explanation is given . We aim to address this question by providing a reliable explanation and providing additional explanations for a given explanation in this paper . With advances in deep learning , its performance improves rapidly . Advances have increased the use of deep neural networks ( DNNs ) , which is used in areas where model decisions significantly affect people - such as healthcare ( Caruana et al. , 2015 ) , radiation ( Reardon , 2019 ) , and autonomous driving . As a result , questions about the rationality of model predictions for industrial and social use have been raised , and the call for explanation is growing ( Goodman & Flaxman , 2017 ; Doshi-Velez & Kim , 2017 ) . Since most DNNs are black-box models , explanation of model prediction is often provided by a second model that explains a given black-box model ( Ribeiro et al. , 2016b ; Chen et al. , 2018a ; Lundberg & Lee , 2017 ; Guo et al. , 2018 ) . This second model is called a post-hoc interpretable model , which has been used for flexibility ( Ribeiro et al. , 2016a ) . However , this can lead people to misunderstanding the explanation because it is not offered by the model that provides the prediction . Therefore , as Rudin ( 2019 ) claimed , it is necessary to use an inherently-interpretable model in order to accurately explain DNNs without any mistakes , which results from an architecture to provide intrinsic explanation . We build an inherently interpretable model , by referring to methods that generate prototypes and conduct classification using Euclidean-distance in a latent space ( Li et al. , 2018 ; Chen et al. , 2019 ) . These prototype-based explanation methods provide explanations with examples and distances and show efficacy in diverse domains . However , prototype-based methods are powerful but not enough to understand a model prediction fully , as explanations must be both human-interpretable and reliable . As most of the existing prototype-based explanation models suffer from the problem that similar latent representations do not guarantee similarity in shape , and distances often provide explanations that are not human-interpretable . Users can raise doubts , but have no choice but to accept the given distances only as numerical values . Since there is no way to verify how distances are constructed , the reliability of the explanations is compromised . Input Prototypes Prototypes : Class 2 Prototypes Input ( b ) Relationships between input and prototypes ( c ) Relationships between prototypes ( d ) Prototype distribution visualization ( a ) Explanation given by prototypes and distances Prototype : Class 8 Prototype : Class 6 Figure 1 : Basic explanation : ( a ) Learned prototypes that represent the training dataset and distances between the input and prototypes . Additional explanations ( b-d ) : ( b ) Relationships between input and prototypes . Bottom of the figure explains that the explanation space enables us to see how input 9 gradually becomes prototype 4 . ( c ) Relationships between prototypes . Providing gradual changes from prototype to another prototype that explains how model discriminates prototypes . ( d ) Prototype distribution visualization with two dimensions based on standard deviation demonstrating that prototypes are not luckily chosen . To solve these problems , we construct an explanation space that provides gradual explanations and reliable distance . By using VAE , a latent space can be visualized as an image that users can recognize . We also use VAE training techniques to regulate a latent space to ensure latent representations share similarities in appearance . This means that a distance in the latent space can have more `` intrinsic meaning '' . We call the latent space obtained through the process the explanation space . Our model provides additional gradual explanation as in Figure 1 . Meanwhile , the prototypes and the distances between the input data and prototypes are given as basic explanation in Figure 1 ( a ) . Through these relationships , users are able to understand how distances are composed . Instead of providing the distances between the input and prototypes , it is possible to better understand a model by showing which features are changing in terms of relationship . Therefore , our proposed model allows users to interpret distances more reliably . In this paper , we developed the model that performs classification through distances between prototype distributions and the encoded input distribution in the latent space of the Variational Autoencoder ( VAE ) ( Kingma & Welling , 2014 ) . After VAE encodes input data in a Gaussian distribution form , we compare the encoded input distribution and generated prototype distributions through four measures with various number of prototype distributions to provide explanations with high classification accuracy . Furthermore , we use the method ( Sensoy et al. , 2018 ) of placing a Dirichlet distribution on the class probabilities to improve accuracy by reducing the uncertainty of the model . Contributions . Our key contributions are as follows : • By constructing the explanation space , we can visualize additional gradual explanations . Through gradual explanations , relationships are provided rather than just providing each prototype image . • By our dense explanation space approach using VAE , similar latent representations tend to resemble each other in appearance . Therefore , distance as explanation is more meaningful because similarity is guaranteed . • We present a method to find a baseline number of prototypes to describe the training dataset . In experiments , we show changes in accuracy and explanations of the model by varying numbers of prototypes . 2 RELATED WORK . Explanation Methods for DNN While there are diverse suites of methods to explain DNN , it can be categorized based on the access of the model ’ s internal information and how they provide explanations ( Jeyakumar et al. , 2020 ) . The model-agnostic method is to approximate the relationships between input data and output decision while treating the model as a black-box . There are methods to create a local explanation ( Ribeiro et al. , 2016b ; Chen et al. , 2018a ; Lundberg & Lee , 2017 ; Chen et al. , 2018b ) by selecting features that play an essential role in making decisions and methods providing an interpretable model through a global approximate ( Guo et al. , 2018 ) . On the other hand , in knowing the internal information of the model , the model-transparent method provides explanations by saliency map using gradient in classifier ( Simonyan et al. , 2014 ; Smilkov et al. , 2017 ; Sundararajan et al. , 2017 ) , class activation map using global average pooling ( Zhou et al. , 2016 ; Selvaraju et al. , 2017 ) , and feature map based on attention ( Liu et al. , 2020 ) . The third method provides explanations by examples capturing relationship between test input and the model ’ s decision . Considering that examples are based on training data ( Bien & Tibshirani , 2011 ; Koh & Liang , 2017 ; Arık & Pfister , 2020 ) or learned prototype data ( Li et al. , 2018 ; Chen et al. , 2019 ; Hase et al. , 2019 ) , our work is closely aligned with these prototype-based classification models . The example-based method shows its efficacy to explain complex concepts in psychology ( Aamodt & Plaza , 1994 ) and education ( Renkl , 2014 ) domains , and is also accepted by the general public as the most preferred explanation method to explain DNN ( Jeyakumar et al. , 2020 ) . The following three prototype-based classification methods are similar to our method in that they use the distance between the encoded input and prototypes for classification and explanation . Li et al . ( 2018 ) learn and visualize the prototypes for the entire data through autoencoder . Chen et al . ( 2019 ) go further from here , characterizing training image patches as class-specific prototypes for partial explanations , while Hase et al . ( 2019 ) feature a hierarchical demonstration of prototypes through related groups for non-flat explanations . As most of prototype-based explanation methods are based on Li et al . ( 2018 ) , we use it as a baseline model . Our framework goes further from these methods , expands prototypes from vectors to distribution in latent space and constructs explanation space for diverse explanations with reliability . In addition to prototype-based methods , there are several methods that use the distance in latent space to understand the internal behavior of a model or to provide explanation using VAE ( Antorán et al. , 2021 ; O ’ Shaughnessy et al. , 2020 ) or GAN ( Lang et al. , 2021 ; Yang et al. , 2021 ) . Uncertainty in deep learning The largest body of research on estimating uncertainty in deep learning is Bayesian neural networks ( Neal , 2012 ; Gal & Ghahramani , 2015 ; Kendall & Gal , 2017 ) . To obtain predictive uncertainty , Bayesian neural networks use prior distributions on model parameters and infer the posterior distribution . Although exact inference in Bayesian methods is intractable , a range of approximate techniques has been proposed , such as variational inference ( Kingma et al. , 2015 ; Gal & Ghahramani , 2016 ) and stochastic gradient hamiltonian monte carlo ( SG-HMC ) ( Chen et al. , 2014 ) . However , in practice , these Bayesian methods are outperformed by Deep Ensembles ( Lakshminarayanan et al. , 2017 ) . This non-bayesian technique uses multiple neural networks trained from different initializations and averages their predictions as the model output . It is simple and effective but requires high computational costs at training and test time . Another non-bayesian technique is based on evidential deep learning ( Sensoy et al. , 2018 ; Amini et al. , 2019 ) , which we use to estimate uncertainty . These methods use a single deterministic neural network and apply prior distributions on model predictions , not model parameters . 3 METHODS . Our model consists of three parts : ( i ) VAE layer , ( ii ) Distance measuring layer ( iii ) Uncertainty layer , as shown in Figure 2 . The VAE layer is used to encode input into a form of distribution and decode the values obtained at prototype distributions and relationships with recognizable image data . The distance between prototype distributions and encoded input distribution is used as input for one fully-connected layer to create an inherently interpretable classification model . Moreover , since classification is performed with distances in the latent space of VAE , we use an uncertainty layer to increase accuracy . 3.1 USING VARIATIONAL AUTO ENCODER . Let D = { X , Y } be the training dataset consists of images x ∈ RN and labels y ∈ { 1 , . . . , K } . The encoder f : q ( z|x ) of the VAE maps an input image to a Gaussian distribution dz ∼ N ( µz , σz2 ) ( µz , σz ∈ Rd ) which is reparameterized to generate latent vector z ∈ Rd . After that , the generated z is reconstructed in the form of image x̂ = g ( z ) through decoder g : p ( x|z ) . Unlike autoencoder , VAE regulates the latent space created by the encoder as isotropic unit Gaussian ( p ( z ) ∼ N ( 0 , I ) ) , which is why we used VAE as a feature extractor . By regularizing latent space and sampling latent vectors from distribution , VAE constructs appropriate dense latent space that creates similar latent representations from similar-shaped images , which plays an essential role in the interpretability of our method described in Section 4 . Furthermore , our model learns prototype distributions as m independent multivariate Gaussian distributions D = { d1 , d2 , . . . , dm } , di ∼ N ( µi , σ2i ) ∈ Rd and visualizes µi as prototype . For that , the structures of disentangled VAE are applied in order to disentangle the explanation components at prototype distributions through dimensions . A method of encouraging the independence of representation distribution q ( z ) by lowering the KL divergence between q ( z ) and q̄ ( z ) : = ∏d j=1 q ( zj ) , and methods of lowering the KL divergence of encoder f and isotropic unit Gaussian p ( z ) to constrain the latent space stronger are additionally used with vanilla VAE . Applying these methods would result in the following objective : 1 n n∑ i=1 [ Eq ( z|xi ) [ logp ( xi|z ) ] − βKL ( q ( z|xi ) ‖p ( z ) ) ] − γKL ( q ( z ) ‖q̄ ( z ) ) . In this objective function , β and γ are hyperparameters that control the power of the constraint terms .

Summary: This paper extends existing series of prototype-based DNN for image classification. Their model uses VAE to ensure similar latent representations share similarities in appearance, a problem that many existing prototype-based CNN models suffer; They also include an uncertainty layer which improves the predictive performance. The authors also conducted human evaluations via Amazon Mechanical Turks to validate the interpretability of their model