Original: B EYOND E XAMPLES : C ONSTRUCTING E XPLANATION S PACE FOR E XPLAINING P ROTOTYPES
Revised: B EYOND E XAMPLES : C ONSTRUCTING E XPLANATION S PACE FOR E XPLAINING P ROTOTYPES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: As deep learning has been successfully deployed in diverse applications, there is ever increasing need for explaining its decision.
Revised: As deep learning has been successfully deployed in diverse applications, there is ever increasing need for explaining its decision.

Original: Most of the existing methods produced explanations with a second model that explains the ﬁrst black-box model, but we propose an inherently interpretable model formore faithful explanations.
Revised: Most of the existing methods produced explanations with a second model that explains the ﬁrst black-box model, but we propose an inherently interpretable model for faithful explanations.

Original: Our method constructs an explanation space in which similarities in terms of human-interpretable features at images share similar latent representations by using a variational autoencoder.
Revised: Our method constructs an explanation space in which similarities in terms of humaninterpretable features at images share similar latent representations by using a variational autoencoder.

Original: This explanation space provides additional explanations of the relationships, going beyond previous classiﬁcation networks that provide explanations by distances and learned prototypes.
Revised: This explanation space provides additional explanations of the relationships, going beyond previous classiﬁcation networks that provide explanations by distances and learned prototypes.

Original: In addition, our distance has more intrinsic meaning by VAE training techniques that regulate the latent space.
Revised: In addition, our distance has more intrinsic meaning by VAE training techniques that regulate the latent space.

Original: With user study, we validate the quality of explanation space and additional explanations.
Revised: With user study, we validate the quality of explanation space and additional explanations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: With the recent deepening interest in deep learning, the performance of deep learning has evolved at a tremendous rate.
Revised: "Explanation" is essentially based on understanding. When a fact is in doubt, an explanation is provided to resolve a question. Therefore explanation must be human-interpretable and reliable. An explanation that is not human-interpretable does not solve the question, and an explanation that is not reliable creates another question.

Revised: Unfortunately in the real world, it is difﬁcult to ﬁnd an explanation that perfectly satisﬁes even one condition.

Revised: In this situation, people receive an insufﬁcient explanation and they may ask - "Why is this explanation given?".

Revised: We aim to address this question by providing a reliable explanation and providing additional explanations for the given explanation in this paper.

Original: Furthermore, advances in deep  learning  have increased the use of deep neural networks (DNNs) in various areas of society, resulting inincreased utilization in areas where model decisions signiﬁcantly affect people, such as healthcare (Caruana et al., 2015), radiation (Reardon, 2019), and autonomous driving.
Revised: With recent interest in deep learning, performance of deep learning has evolved at a tremendous rate. Advances have increased the use of deep neural networks (DNNs), resulting in utilization in areas where model decisions signiﬁcantly affect people - such as healthcare (Caruana et al., 2015), radiation (Reardon, 2019), and autonomous driving.

Original: As a result, questionshave been raised about the rationality of model predictions for industrial and social use, and calls for "explanation" are growing (Goodman & Flaxman, 2017; Doshi-Velez & Kim, 2017).
Revised: As a result, questions about the rationality of model predictions for industrial and social use have been raised, and calls for "explanation" are growing (Goodman & Flaxman, 2017; Doshi-Velez & Kim, 2017).

Original: Although it is difﬁcult to naturally provide humanunderstandable explanations due to the opacity, hidden internal structure, and highly nonlinear complexity of DNNs, the scientiﬁc community strives to analyze the behavior of models considered as black-box and to reasonably explain their prediction (Ribeiro et al., 2016b; Lundberg & Lee, 2017;

Original: Simonyan et al., 2014).

Original: Since most DNNs are black-box models, the explanations of model predictions are often provided by a second model that explains the ﬁrst black-box model (Ribeiro et al., 2016b;Chen et al., 2018a;
Revised: Since most DNNs are black-box models, explanation of model prediction is often provided by a second model that explains the ﬁrst black-box model (Ribeiro et al., 2016b;

Original:  Lundberg & Lee, 2017; Guo et al., 2018).
Revised: Chen et al., 2018a; Lundberg & Lee, 2017; Guo et al., 2018).

Original: The second modelthat provides this form of explanation is called a posthoc - interpretable model, which has been used for ﬂexibility (Ribeiro et al., 2016a) and maintain the accuracy by ensuring complexity of black-box models based on the myth of accuracyinterpretability trade-off (Gunning & Aha, 2019).
Revised: This second model is called a post-hoc interpretable model, which has been used for ﬂexibility (Ribeiro et al., 2016a).

Original: However, this can lead people to misunderstand the explanation because it is not offered by the model that provides the predictions.
Revised: However, this can lead people to misunderstanding the explanation because it is not offered by the model that provides the prediction.

Original: Therefore, inherently - interpretable model which provides its own explanations is required to accurately explain DNN without any mistakes, even if it is a small probability (Rudin, 2019).
Revised: Therefore, as Rudin (2019) claimed, it is necessary to use an inherently-interpretable model in order to accurately explain DNNs without any mistakes, which are due to its nature of providing its own explanation.

Original: To build  inherently interpretable model,we developed our model by referring to methods that generate prototypes and conduct classiﬁcation using Euclidean-distance in the latent space (Li et al., 2018; Chen et al., 2019).
Revised: We build an inherently interpretable model, by referring to methods that generate prototypes and conduct classiﬁcation using Euclidean-distance in the latent space (Li et al., 2018; Chen et al., 2019).

Original: Our method belongs to example-based explanation method that provide explanations based on understandable examples, such as images from training data (Koh & Liang, 2017; Kim et al., 2016; Arık & Pﬁster, 2020) or representative prototypes (Li et al., 2018).
Revised: These prototype-based explanation methods provide explanations with examples and distances and prove their ability in diverse domains. However, prototype-based methods are powerful but not enough for fully understanding the model’s prediction, as explanations must be both humaninterpretable and reliable. As most of the existing prototype-based explanation models suffer from the problem that similar latent representations do not guarantee similarity in shape, distances often provide explanations that are not human-interpretable.

Original: This method proves their ability to explain complex concepts in psychology (Aamodt & Plaza, 1994) and education (Renkl, 2014) domains and is also surveyed to general public as the most preferred explanation method in deep learning paper (Jeyakumar et al., 2020).
Revised: Users can raise doubts but have no choice but to accept the given distances only as numerical values.

Original: Through that, example-based explanation method has been studied as one of the crucial axes while there are various ways to provide explanations for model.
Revised: Since there is no way to verify how distances are composed, the reliability of the explanations is compromised.

Original: [Figure]
Revised: [Figure]

Original: However, examples are powerful but not enough.
Revised: To solve these problems, we construct the explanation space that provides gradual explanations and reliable distance.

Original: Understanding the model’s predictions by similar examples can lead to misunderstanding and overgeneralization (Kim et al., 2016). Thus, attempts have been made to expand the representative power of the explanations by showing concepts (Kim et al., 2018; Ghorbani et al., 2019), counterfactual data (Kim et al., 2016;
Revised: By using VAE, the latent space can be visualized as an image that users can recognize. Also, we leverage VAE training techniques to regulate the latent space to ensure latent representations share similarities in appearance. This means that the distance in the latent space can have more "intrinsic meanings".

Original: Goyal et al., 2019), and causal factors (O’Shaughnessy et al., 2020). We go further from the existing methods by constructing explanation space that provide gradual explanations and giving reliable distance.
Revised: We call the latent space obtained through these processes as the explanation space. Our model provides additional gradual explanations as illustrated in Figure 1.

Original: Among our model, (i) relationships between  input data and prototypes, (ii) relationships between each prototypes, and (iii) description of the prototype’s surroundings are given as illustrated in Figure 1. Meanwhile, prototypes and distance between inputdata and prototypes are given for basic explanation.
Revised: Meanwhile, prototypes and distances between the input data and prototypes are given as a basic explanation in Figure 1(a). Through these relationships, users are able to understand how distances are composed. Rather than only providing distances with the input and prototypes, it is possible to increase understanding of the model by showing which features are changing through relationships.

Original:  In this paper, we developed the model that progresses classiﬁcation through the distance between prototype distributions and  encoded input distribution in the latent space of the Variational Autoencoder (VAE)(Kingma & Welling, 2014).By VAE, latent space can be visualized as an image that people can recognize so we call that explanation space.
Revised: Therefore, our proposed model allows users to conﬁrm the reliability of the distance. In this paper, we developed the model that performs classiﬁcation through distances between prototype distributions and the encoded input distribution in the latent space of the Variational Autoencoder (VAE)(Kingma & Welling, 2014).

Original: We leverage VAE training techniques to regulate the latent space so that distances in the latent space have more "intrinsic meanings".

Original: After VAE encodes input data in  Gaussian distribtuion form, we compare the encoded input distribution and generated prototype distributions through four measures with various number of prototype distributions to provide explanations with high classiﬁcation accuracy.
Revised: After VAE encodes input data in a Gaussian distribution form, we compare the encoded input distribution and generated prototype distributions through four measures with various number of prototype distributions to provide explanations with high classiﬁcation accuracy.

Original: Furthermore, we use the method (Sensoy et al., 2018)  placing a Dirichlet distribution on the class probabilities to improvethe accuracy by reducing the uncertainty of the model.
Revised: Furthermore, we use the method (Sensoy et al., 2018) of placing a Dirichlet distribution on the class probabilities to improve accuracy by reducing the uncertainty of the model.

Original: Contributions.
Revised: Contributions.

Original: Our key contributions are as follows:
Revised: Our key contributions are as follows:

Original: • By constructing  explanation space, we can visualizethe additional gradual explanations.
Revised: • By constructing the explanation space, we can visualize additional gradual explanations.

Original: Through gradual explanations, relationships are provided rather than just providing each prototype image.
Revised: Through gradual explanations, relationships are provided rather than just providing each prototype image.

Original: • We use VAE to construct dense explanation space that ensures similar latent representations share similarities in appearance so that distance provided as an explanation became possible to function as a numerical value that guarantees similarity.
Revised: • Similar latent representations sharing similarities in appearance are ensured by our constructed dense explanation space using VAE. Therefore the distance provided as an explanation becomes possible to function as a numerical value that guarantees similarity.

Original: • We present a method to ﬁnd a baseline number of prototypes to describe the training dataset.
Revised: • We present a method to ﬁnd a baseline number of prototypes to describe the training dataset.

Original: With experiments, we show changes in accuracy and explanations of the model through the varying numbers of prototypes.
Revised: In experiments we show changes in accuracy and explanations of the model by varying numbers of prototypes.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Explanation Methods for DNN While there are diverse suites of methods to explain DNN, it can be categorized based on the access of the model’s internal information and how they provide explanations (Jeyakumar et al., 2020).
Revised: Explanation Methods for DNN While there are diverse suites of methods to explain DNN, it can be categorized based on the access of the model’s internal information and how they provide explanations (Jeyakumar et al., 2020).

Original: The model-agnostic method is to approximate the relationships between input data and output decision while treating the model as a black-box.
Revised: The model-agnostic method is to approximate the relationships between input data and output decision while treating the model as a black-box.

Original: There are methods to create a local explanation (Ribeiro et al., 2016b; Chen et al., 2018a; Lundberg & Lee, 2017; Chen et al., 2018b) by selecting features that play an essential role in making decisions and methods providing an interpretable model through a global approximate (Guo et al., 2018).
Revised: There are methods to create a local explanation (Ribeiro et al., 2016b; Chen et al., 2018a; Lundberg & Lee, 2017; Chen et al., 2018b) by selecting features that play an essential role in making decisions and methods providing an interpretable model through a global approximate (Guo et al., 2018).

Original: On the other hand, in knowing the internal information of the model, the model-transparent method provides explanations by saliency map using gradient in classiﬁer (Simonyan et al., 2014; Smilkov et al., 2017; Sundararajan et al., 2017), class activation map using global average pooling (Zhou et al., 2016; Selvaraju et al., 2017), and feature map based on attention (Liu et al., 2020).
Revised: On the other hand, in knowing the internal information of the model, the model-transparent method provides explanations by saliency map using gradient in classiﬁer (Simonyan et al., 2014; Smilkov et al., 2017; Sundararajan et al., 2017), class activation map using global average pooling (Zhou et al., 2016; Selvaraju et al., 2017), and feature map based on attention (Liu et al., 2020).

Original: The third method provides explanations by instances capturing relationship between test input and the model’s decision.
Revised: The third method provides explanations by examples capturing relationship between test input and the model’s decision.

Original: Considering that instances are based on training data (Bien & Tibshirani, 2011;{+
 ;+} Koh & Liang, 2017;
Revised: Considering that examples are based on training data (Bien & Tibshirani, {+
 ;+} Koh & Liang, 2017;

Original: Arık & Pﬁster, 2020), counterfactual images (Antorán et al., 2021; Lang et al., 2021), or learned prototype data (Li et al., 2018; Chen et al., 2019; Hase et al., 2019), our work is closely aligned with these prototype-based classiﬁcation models.
Revised: Arık & Pﬁster, 2020) or learned prototype data (Li et al., 2018; Chen et al., 2019; Hase et al., 2019), our work is closely aligned with these prototype-based classiﬁcation models.

Revised: The example-based method proves their ability to explain complex concepts in psychology (Aamodt & Plaza, 1994) and education (Renkl, 2014) domains and is also accepted by the general public as the most preferred explanation method to explain DNN (Jeyakumar et al., 2020).

Original: The following three prototype-based classiﬁcation methods are similar with our method in that they use the distance between  encoded input and prototypes for classiﬁcation and explanation.
Revised: The following three prototype-based classiﬁcation methods are similar to our method in that they use the distance between the encoded input and prototypes for classiﬁcation and explanation.

Original: Li et al.
Revised: Li et al.

Original: (2018) learns and visualizes the prototypes for the entire data through autoencoder.
Revised: (2018) learn and visualize the prototypes for the entire data through autoencoder.

Original: Chen et al.
Revised: Chen et al.

Original: (2019) goes further from here, characterizing training image patches as class-speciﬁc prototypes for partial explanations, while Hase et al.
Revised: (2019) go further from here, characterizing training image patches as class-speciﬁc prototypes for partial explanations, while Hase et al.

Original: (2019) features a hierarchical demonstration of prototypes through related groups for non-ﬂat explanations.
Revised: (2019) feature a hierarchical demonstration of prototypes through related groups for non-ﬂat explanations.

Original: As these methods are based on Li et al.
Revised: As most of prototype-based explanation methods are based on Li et al.

Original: (2018), we use it as a baseline model.
Revised: (2018), we use it as a baseline model.

Original: Our framework goes further from these methods, expands prototypes from vectors to distribution in latent space and constructs explanation space for diverse explanations with reliability.
Revised: Our framework goes further from these methods, expands prototypes from vectors to distribution in latent space and constructs explanation space for diverse explanations with reliability.

Revised: In addition to the prototype-based methods, there are several methods that use the distance in latent space to understand the internal behavior of a model or to provide an explanation using VAE (Antorán et al., 2021; O’Shaughnessy et al., 2020) or GAN (Lang et al., 2021;

Original:  Uncertainty in deep learning The largest body of research on estimating uncertainty in deep learning is Bayesian neural networks (Neal, 2012; Gal & Ghahramani, 2015; Kendall & Gal, 2017).
Revised: Yang et al., 2021). Uncertainty in deep learning The largest body of research on estimating uncertainty in deep learning is Bayesian neural networks (Neal, 2012; Gal & Ghahramani, 2015; Kendall & Gal, 2017).

Original: To obtain predictive uncertainty, Bayesian neural networks apply prior distributions on model parameters and infer the posterior distribution.
Revised: To obtain predictive uncertainty, Bayesian neural networks use prior distributions on model parameters and infer the posterior distribution.

Original: Although exact inference in Bayesian methods is intractable, a range of approximate techniques has been proposed, such as variational inference (Kingma et al., 2015; Gal & Ghahramani, 2016) and Stochastic Gradient Hamiltonian Monte Carlo (SG-HMC) (Chen et al., 2014).
Revised: Although exact inference in Bayesian methods is intractable, a range of approximate techniques has been proposed, such as variational inference (Kingma et al., 2015; Gal & Ghahramani, 2016) and Stochastic Gradient Hamiltonian Monte Carlo (SG-HMC) (Chen et al., 2014).

Original: However, in practice, these Bayesian methods are outperformed by Deep Ensembles (Lakshminarayanan et al., 2017).
Revised: However, in practice, these Bayesian methods are outperformed by Deep Ensembles (Lakshminarayanan et al., 2017).

Original: This non-bayesian technique uses multiple neural networks trained from different initializations and averages their predictions as the model output.
Revised: This non-bayesian technique uses multiple neural networks trained from different initializations and averages their predictions as the model output.

Original: It is simple and effective but requires high computational costs at training and test time.
Revised: It is simple and effective but requires high computational costs at training and test time.

Original: Another non-bayesian technique is based on evidential deep learning (Sensoy et al., 2018;
Revised: Another non-bayesian technique is based on evidential deep learning (Sensoy et al., 2018;

Original: Amini et al., 2019), which we use to estimate uncertainty.
Revised: Amini et al., 2019), which we use to estimate uncertainty.

Original: These methods use a single deterministic neural network and apply prior distributions on model predictions, not model parameters.
Revised: These methods use a single deterministic neural network and apply prior distributions on model predictions, not model parameters.

Original:  M ETHODS
Revised:  M ETHODS

Original: Our model consists of three parts: (i) VAE layer, (ii) Distance measuring layer (iii) Uncertainty layer, as shown in Figure 2.
Revised: Our model consists of three parts: (i) VAE layer, (ii) Distance measuring layer (iii) Uncertainty layer, as shown in Figure 2.

Original: The VAE layer is used to encode input into a form of distribution and decode the values obtained at prototype distributions and relationships with recognizable image data.
Revised: The VAE layer is used to encode input into a form of distribution and decode the values obtained at prototype distributions and relationships with recognizable image data.

Original: The distance between prototype distributions and encoded input distribution is used as input for one fully-connected layer to create an inherently interpretable classiﬁcation model.
Revised: The distance between prototype distributions and encoded input distribution is used as input for one fully-connected layer to create an inherently interpretable classiﬁcation model.

Original: Moreover, since classiﬁcation is conducted through distance with embedded data, we use the uncertainty layer to increase accuracy.3. U SING V ARIATIONAL A UTO E NCODER Let D =
Revised: Moreover, since classiﬁcation is conducted through distance in the latent space of VAE, we use the uncertainty layer to increase accuracy.

Original: { X, Y } be the training dataset consists of images x ∈ R N and labels y ∈ { 1 , . .

Original: . , K } .

Original: The encoder f : q ( z | x ) of the VAE changes an input image to the form of Gaussian distribution d z ∼ N ( µ z , σ z 2 ) ( µ z , σ z ∈ R d ) which reparameterized to latent vector z ∈ R d .

Original: After that, the generated z is reconstructed in the form of image ˆ x = g ( z ) through decoder g : p ( x | z ) .

Original: Unlike

Original: Encoder 𝑓
Revised: Encoder 𝑓

Original: Decoder 𝑔
Revised: Decoder 𝑔

Original: Input image x
Revised: Input image x

Original: Reconstructed image ො x 𝜇 𝑧 𝜎 𝑧 z d 𝑧 … 𝜇𝜇𝜇𝜇 𝑚 dddd 𝑚
Revised: Reconstructed image ො x 𝜇 𝑧 𝜎 𝑧 z d 𝑧 … 𝜇𝜇𝜇𝜇 𝑚 dddd 𝑚

Original: VAE layers … 𝜇 𝑧
Revised: VAE layers … 𝜇 𝑧

Original: Distance measuring layer …
Revised: Distance measuring layer …

Original: Fully-connected layer ℎ
Revised: Fully-connected layer ℎ

Original: Uncertainty layer
Revised: Uncertainty layer

Revised: Uncertainty calculation 𝛼𝛼𝛼 𝑘 𝑢 l ogits 𝑙

Revised: 3. U SING V ARIATIONAL A UTO E NCODER

Revised: Let D =

Revised: { X, Y } be the training dataset consists of images x ∈ R N and labels y ∈ { 1 , . .

Revised: . , K } .

Original: Uncertainty calculation 𝛼𝛼𝛼 𝑘 𝑢 l ogits 𝑙 autoencoder, VAE regulates the latent space created by the encoder as isotropic unit Gaussian ( p ( z ) ∼ N (0 , I ) ), which is why we used VAE as a feature extractor.
Revised: The encoder f : q ( z | x ) of the VAE maps an input image to a Gaussian distribution d z ∼ N ( µ z , σ z 2 ) ( µ z , σ z ∈ R d ) which is reparameterized to generate latent vector z ∈ R d . After that, the generated z is reconstructed in the form of image ˆ x = g ( z ) through decoder g : p ( x | z ) . Unlike autoencoder, VAE regulates the latent space created by the encoder as isotropic unit Gaussian ( p ( z ) ∼ N (0 , I ) ), which is why we used VAE as a feature extractor.

Original: By regularizing latent space and sampling latent vectors from distribution, VAE constructs appropriate dense latent space that creates similar latent representations from similar-shaped images, which play an essential role in the interpretability of our method described in Section 4.
Revised: By regularizing latent space and sampling latent vectors from distribution, VAE constructs appropriate dense latent space that creates similar latent representations from similar-shaped images, which plays an essential role in the interpretability of our method described in Section 4.

Original: Furthermore, our model learns prototype distributions as m independent multivariate Gaussian distributions D = { d 1 , d 2 , . . .
Revised: Furthermore, our model learns prototype distributions as m independent multivariate Gaussian distributions D = { d 1 , d 2 , . . .

Original: , d m } , d i ∼ N ( µ i , σ 2 i )
Revised: , d m } , d i ∼ N ( µ i , σ 2 i )

Original: ∈ R d and visualizes µ i as prototype.
Revised: ∈ R d and visualizes µ i as prototype.

Original: For that, the structures of disentangled VAE are applied in order to disentangle the explanation components at prototype distributions through dimensions.
Revised: For that, the structures of disentangled VAE are applied in order to disentangle the explanation components at prototype distributions through dimensions.

Original: A method of encouraging the independence of representation distribution q ( z ) by lowering the KL divergence between q ( z ) and ¯ q ( z ) :
Revised: A method of encouraging the independence of representation distribution q ( z ) by lowering the KL divergence between q ( z ) and ¯ q ( z ) :

Original: = (cid:81) dj
Revised: = (cid:81) dj

Original: =1 q ( z j ) , and method of lowering the KL divergence of encoder f and isotropic unit Gaussian p ( z ) to constrain the latent space stronger are additionally used with vanilla VAE.
Revised: =1 q ( z j ) , and methods of lowering the KL divergence of encoder f and isotropic unit Gaussian p ( z ) to constrain the latent space stronger are additionally used with vanilla VAE.

Original: Applying these methods would result in the following objective:
Revised: Applying these methods would result in the following objective:

Original: [Equation]

Original: n (cid:88)

Original: [Equation]
Revised: [Equation]

Original: At this objective function, β and γ are hyperparameters that control the power of the constraint terms.
Revised: In this objective function, β and γ are hyperparameters that control the power of the constraint terms.

Original: 3. C LASSIFICATION NETWORK USING DISTANCE
Revised: 3. C LASSIFICATION NETWORK USING DISTANCE

Original: Once the input is encoded by latent distribution d z , logits based on the distance from the prototype distributions are produced for the classiﬁcation.
Revised: Once the input is encoded by latent distribution d z , logits based on the distance from the prototype distributions are produced for the classiﬁcation.

Original: This process is divided into (i) measuring the distance between learned D = { d 1 , d 2 , . . .
Revised: This process is divided into (i) measuring the distance between learned D = { d 1 , d 2 , . . .

Original: , d m } and d z in the latent space and (ii) producing the logits by measured distance using the fully-connected layer h for the classiﬁcation.
Revised: , d m } and d z in the latent space and (ii) producing the logits by measured distance using the fully-connected layer h for the classiﬁcation.

Original: Since prototype distributions D and encoded input d z both are in the form of Gaussian distribution, we use 2-Wasserstein distance (Givens et al., 1984) as measure for the distance.
Revised: Since prototype distributions D and encoded input d z both are in the form of Gaussian distribution, we use 2-Wasserstein distance (Givens et al., 1984) as a distance measure.

Original: The 2-Wasserstein distance W for two multivariate Gaussian distribution N ( µ 1 , Σ 1 ) , N ( µ 2 , Σ 2 )( µ i ∈ R d , Σ i ∈ 
Revised: The 2-Wasserstein distance W for two multivariate Gaussian distribution N ( µ 1 , Σ 1 ) , N ( µ 2 , Σ 2 )( µ i ∈ R d , Σ i ∈ R

Original: R d × R d ) is as follows:
Revised: d × R d ) can be computed as:

Original: [Equation]
Revised: [Equation]

Original: Since we assume the prototype distributions D to be independent and encoder f predicts diagonal covariance matrices, Σ
Revised: Since we assume the prototype distributions D to be independent and encoder f predicts diagonal covariance matrices, Σ

Original: i = diag( σ 1 i , . . .
Revised: i = diag( σ 1 i , . . .

Original: , σ di ) and  above equation simpliﬁed as follows:
Revised: , σ di ) and the above equation simpliﬁed as:

Original: [Equation]
Revised: [Equation]

Original: Three other measurements KL-divergence (KLD), Jensen-Shannon divergence (JSD), Jensen-Tsallis distance (JTD) (Choi et al., 2019) are also experimented in Section 5.1.
Revised: Three other measurements KL-divergence (KLD), Jensen-Shannon divergence (JSD), Jensen-Tsallis distance (JTD) (Choi et al., 2019) are also experimented in Section 5.1.

Original: The distances between D and d z are used as input of fully-connected layer h to generate logits.
Revised: The distances between D and d z are used as input of fully-connected layer h to generate logits.

Original: The reason for classiﬁcation by passing only one fully-connected layer is to construct a weight matrix.
Revised: The reason for classiﬁcation by passing only one fully-connected layer is to construct a weight matrix.

Original: By constructing a weight matrix using h , the model can numerically conﬁrm the impact of each prototype distribution on class selection.
Revised: By constructing a weight matrix using h , the model can numerically conﬁrm the impact of each prototype distribution on class selection.

Original: This weight matrix gives interpretability to the model’s prediction even when the number of given prototype distributions m and the class K are different.
Revised: This weight matrix gives interpretability to the model’s prediction even when the number of given prototype distributions m and the class K are different.

Original: 3. U SING UNCERTAINTY 
Revised: 3. U SING UNCERTAINTY LAYER

Original: Our method uses the distancebetween distributions d z , D at latent space for classiﬁcation.
Revised: Repeating the sampling process could increase the uncertainty of the classiﬁcation process through the distance at latent space.

Original: Also, with considering the uncertainty, we improve the classiﬁcation performance  about 1 % with a method (Sensoy et al., 2018) that uses Dirichlet distribution to reduce the uncertainty of the model.
Revised: Therefore, with considering the uncertainty, we improve the classiﬁcation performance by about 1 % with a method (Sensoy et al., 2018) that uses Dirichlet distribution to reduce the uncertainty of the model.

Original: In the training process, the uncertainty loss L uncertain is calculated using the Dirichlet parameters obtained through logits l .
Revised: In the training process, the uncertainty loss L uncertain is calculated using the Dirichlet parameters obtained through logits l .

Original: Detailed method for uncertainty layer is in Appendix A.5.
Revised: The detailed method for uncertainty layer is in Appendix A.5.

Original: After going through all of these processes, the ﬁnal objective function is composed as follows:
Revised: After going through all of these processes, the ﬁnal objective function is composed as follows:

Original: [Equation]
Revised: [Equation]

Original: where L uncertain is the uncertainty classiﬁcation term based on cross-entropy loss and D and d z is the VAE term that described at Section 3 . 1 .
Revised: where L uncertain is the uncertainty classiﬁcation term based on cross-entropy loss and D and d z is the VAE term that described at Section 3 . 1 .

Original: Moreover, λ is a hyperparameter for adjusting the ratio between them.
Revised: Moreover, λ is a hyperparameter for adjusting the ratio between them.

Original:  C ONSTRUCTING INTERPRETABLE EXPLANATIONS 4. R ELIABLE EXPLANATION SPACE
Revised:  C ONSTRUCTING INTERPRETABLE EXPLANATIONS 4. R ELIABLE EXPLANATION SPACE

Original: Our method increases the reliability of the explanation through distance and the prototypes.
Revised: Our method increases the reliability of the explanation through distance and the prototypes.

Original: As noted in Brendel & Bethge (2019),  interpretation of previous prototype-based classiﬁcation methods is difﬁcult because only the Euclidean distance between prototypes and latent representations is given, while having a similar latent representation does not guarantee that images share similarities in human-interpretable features.
Revised: As noted in Brendel & Bethge (2019), the interpretation of previous prototype-based classiﬁcation methods is difﬁcult because only the Euclidean distance between prototypes and latent representations is given, while having a similar latent representation does not guarantee that images share similarities in human-interpretable features.

Original: Therefore, by sampling with distributions and regularizing latent space through VAE, our model constructs dense explanation space that ensures similar latent representations share similarities in appearance.
Revised: Therefore, by sampling with distributions and regularizing latent space through VAE, our model constructs dense explanation space that ensures similar latent representations share similarities in appearance.

Original: Figure 3 makes this more evident.
Revised: Figure 3 makes this more evident.

Original: Looking at the visualized latent space by t-SNE (Van der Maaten & Hinton, 2008), embedded data by the previous method tends to separate clusters, often make the case that the distance between two points in the same class farther than the distance between two different class points, as shown in Figure 3(a).
Revised: Looking at the visualized latent space by t-SNE (Van der Maaten & Hinton, 2008), embedded data by the previous method tends to separate clusters, often make the case that the distance between two points in the same class farther than the distance between two different class points, as shown in Figure 3(a).

Original: In contrast, the proposed model increases the reliability of explanation through distance by showing that each class is clustered together and exist close to each other that forms dense latent space.
Revised: In contrast, the proposed model increases the reliability of explanation through distance by showing that each class is clustered together and exist close to each other that forms dense latent space.

Original: Furthermore, by showing latent representations contained within the prototype distribution d i as images, the prototype’s reliability can also be increased by their similarities.
Revised: Furthermore, by showing latent representations contained within the prototype distribution d i as images, the prototype’s reliability can also be increased by their similarities.

Original: As illustrated in Figure 1(d), features within the distribution are similar to the prototype µ i at basic explanation, demonstrating that prototypes are not luckily chosen.
Revised: As illustrated in Figure 1(d), features within the distribution are similar to the prototype µ i at basic explanation, demonstrating that prototypes are not coincidentally chosen.

Original: We made quantitative comparisons through user study as well as qualitative comparisons to ensure the reliability of the explanation space.
Revised: We made quantitative comparisons through user study as well as qualitative comparisons to ensure the reliability of the explanation space.

Original: We measured people’s understanding through an interactive reconstruction experiment (Ross et al., 2021) to show the reliability of the explanation space of our method.
Revised: We measured the user’s understanding through an interactive reconstruction experiment (Ross et al., 2021) to show the reliability of the explanation space of our [Figure]

Original:  This experiment is to generate an image similar to a randomly sampled image by changing each dimension value of the latent space through a slide using the decoder of the models.
Revised: method. This experiment is to generate an image similar to a randomly sampled image by changing each dimension value of the latent space through a slide using the decoder of the models.

Original: As the user explores the latent space and generates an image similar to the sampled image, we can see how the conﬁguration of the latent space is easy to understand for the user.
Revised: As the user explores the latent space and generates an image similar to the sampled image, we can see how the conﬁguration of the latent space is easy to understand for the user.

Original: The evaluation was conducted through Amazon Mechanical Turk.
Revised: The evaluation was conducted through Amazon Mechanical Turk.

Original: In order to improve the quality of the evaluation, all experiments[Figure] were conducted through 20 Mechanical Turk Masters.
Revised: In order to improve the quality of the evaluation, all experiments were conducted through 20 Mechanical Turk Masters.

Original: The success rate by model is shown in Table 1.
Revised: The success rate by model is shown in Table 1.

Original: Detailed settings for experiments are in Appendix A.2.
Revised: Detailed settings for experiments are in Appendix A.2.

Original: [Table]
Revised: [Table]

Original: As Ross et al.
Revised: As Ross et al.

Original: (2021) concluded that a higher success rate on interactive reconstruction tasks shows improved interpretability, we concluded that the explanation space by our method improved interpretability because the success rate obtained through the interactive reconstruction experiment showed a higher value than the baseline model.
Revised: (2021) concluded that a higher success rate on interactive reconstruction tasks shows improved interpretability, we concluded that the explanation space by our method improved interpretability because the success rate obtained through the interactive reconstruction experiment showed a higher value than the baseline model.

Original: The distance value in the explanation space having more intrinsic meaning because the explanation space created by our method showed improved interpretability compared to the baseline model.
Revised: The distance value in the explanation space having more intrinsic meaning because the explanation space created by our method showed improved interpretability compared to the baseline model.

Original: Therefore, the experimental results could be used as quantitative evaluation to support that our method created a reliable explanation space.
Revised: Therefore, the experimental results could be used as quantitative evaluation to support that our method created a reliable explanation space.

Original: 4. S ELECTING  NUMBER OF PROTOTYPE DISTRIBUTIONS
Revised: 4. S ELECTING THE NUMBER OF PROTOTYPE DISTRIBUTIONS

Original: In the prototype-based explanation model, the number of prototypes is critical.
Revised: In the prototype-based explanation model, the number of prototypes is critical.

Original: As can be seen in the model structure, the number of prototypes affects the input of the fully-connected layer and inevitably affects accuracy.
Revised: As can be seen in the model structure, the number of prototypes affects the input of the fully-connected layer and inevitably affects accuracy.

Original: Moreover, since explanations are also provided through prototypes, it is important to set the number of prototypes.
Revised: Moreover, since explanations are also provided through prototypes, it is important to set the number of prototypes.

Original: However, since there was no information to refer other than the number of classes to determine the number of prototypes that can represent the dataset, the previous methods (Li et al., 2018; Chen et al., 2019; Hase et al., 2019) speciﬁed the number of prototypes to an arbitrary and heuristic value determined by a human.
Revised: However, since there was no information to refer other than the number of classes to determine the number of prototypes that can represent the dataset, the previous methods (Li et al., 2018; Chen et al., 2019; Hase et al., 2019) speciﬁed the number of prototypes to an arbitrary and heuristic value determined by a human.

Original: So we present a method to provide a baseline number of prototype distributions in our framework.
Revised: We propose a method to produce the baseline number of prototype distributions in our framework.

Original: Since we have set the prototype distributions as independent Gaussian distributions rather than vectors, prototype distributions cover the latent space.
Revised: Since we have set the prototype distributions as independent Gaussian distributions rather than vectors, prototype distributions cover the latent space.

Original: Furthermore, as prototype distributions are representative of the whole dataset, it shares similarities with the Gaussian Mixture Model (GMM)  that the whole dataset is represented in the form of multiple Gaussian distributions.
Revised: Furthermore, as prototype distributions are representative of the whole dataset, it shares similarities with the Gaussian Mixture Model (GMM) in that the whole dataset is represented in the form of multiple Gaussian distributions.

Original: So we choose an optimal number of components of the GMM at latent space as a baseline number of prototype ditributions.
Revised: We choose an optimal number of components of the GMM in latent space as a baseline number of prototype distributions.

Original: We use the Bayesian Information Criterion (BIC), which is known to be the most appropriate criterion for GMM (Steele & Raftery, 2010).
Revised: We use the Bayesian Information Criterion (BIC), which is known to be the most appropriate criterion for GMM (Steele & Raftery, 2010).

Original: The experimental results for this can be seen in the Section 5.3.
Revised: The experimental results for this can be seen in the Section 5.3.

Original: 4. E XPLAINING  RELATIONSHIP OF EXPLANATIONS
Revised: 4. E XPLAINING THE RELATIONSHIP OF EXPLANATIONS

Original: The explanations of our model are divided into basic explanations and explanations of the basic explanations.
Revised: The explanations of our model are divided into basic explanation and explanations of the basic explanation.

Original: For fundamental explanations, visualized images of the learned prototypes and the distance between encoded input and prototype distributions are given as previous prototype-based explanation methods.
Revised: For fundamental explanations, visualized images of the learned prototypes and the distance between encoded input and prototype distributions are given as previous prototype-based explanation methods.

Original: To explain these basic explanations, we additionally provide two gradual explanations.
Revised: To explain this basic explanation, we additionally provide two gradual explanations.

Original: These explanations can be obtained because our method constructs dense explanation space, unlike previous methods.
Revised: These explanations can be obtained because our method constructs dense explanation space, unlike previous methods.

Original: As a result, it is possible to generate interconnection between latent representations as images.
Revised: As a result, it is possible to generate interconnection between latent representations as images.

Original: We represent changing elements by interpolating these latent representations and deﬁne them as relationships.
Revised: We represent changing elements by interpolating these latent representations and deﬁne them as relationships.

Original: The relationships between inputs and prototypes can be visualized by our model.
Revised: The relationships between inputs and prototypes can be visualized by our model.

Original: In situations where distance is given for interpretable information, the factors that can reduce the distance are critical for understanding the model’s prediction.
Revised: In situations where distance is given for interpretable information, the factors that can reduce the distance are critical for understanding the model’s prediction.

Original: For example, Figure 1(b) shows that the input must be thinner to get closer to prototype 9 and shows that the upper part is most important to go near prototype 4.

Original: If the interpretation of these explanations is difﬁcult, deeper explanations can be computed by accepting more images from the relationships.

Original: Additionally, relationships between prototypes are given.
Revised: Additionally, relationships between prototypes are given.

Original: Since the prototypes are representative of the whole dataset, relationships between prototypes show which features are needed to change from one class to another class.
Revised: Since the prototypes are representative of the whole dataset, relationships between prototypes show which features are needed to change from one class to another class.

Original: This explanation can be used as basic information about how the model discriminates classes.
Revised: This explanation can be used as basic information about how the model discriminates classes.

Original: After the model is trained, basic explanations and explanations for basic explanations are visualized as follows.
Revised: After the model is trained, basic explanation and explanations for basic explanation are visualized as follows.

Original: First, the prototype, provided as the fundamental explanation, is visualized by sending µ i of the prototype distribution through the decoder g as a prototype.
Revised: First, the prototype, provided as the fundamental explanation, is visualized by sending µ i of the prototype distribution through the decoder g as a prototype.

Original: Next, the relationships between prototype µ i and prototype µ j are visualized as sending µ i + ( µ j − µ i ) r/k for r = 0 , 1 , . . .
Revised: Next, the relationships between prototype µ i and prototype µ j are visualized as sending µ i + ( µ j − µ i ) r/k for r = 0 , 1 , . . .

Original: , k through the decoder g , which can be controlled by increasing the value of k for detailed explanation and decreasing the value of k for a brief explanation.
Revised: , k through the decoder g , which can be controlled by increasing the value of k for detailed explanation and decreasing the value of k for a brief explanation.

Original: Finally, the relationships between inputs and prototypes are provided in the same way, and visualizations of prototype distributions are based on µ values and σ values.
Revised: Finally, the relationships between inputs and prototypes are provided in the same way, and visualizations of prototype distributions are based on µ values and σ values.

Original: The disadvantage of our model comes from this visualization.
Revised: The disadvantage of our model comes from this visualization.

Revised: Many training samples are required to generate explanation images.

Original: Visualized images of our explanations are fundamentally generated through VAE, making it challenging to generate human-recognizable images for complex datasets that are difﬁcult for VAE to generate the observable image.
Revised: In addition, visualized images of our explanations are fundamentally generated through VAE, making it challenging to generate human-recognizable images for complex datasets that are difﬁcult for VAE to generate the observable image.

Original: However, this is a problem related to VAE’s expressive power, and there are many high-resolution expressive methods (Lee et al., 2020; Vahdat & Kautz, 2020) based on VAE, so applying these methods would solve these problems in future work.
Revised: However, this is a problem related to VAE’s expressive power, and there are many high-resolution expressive methods (Lee et al., 2020; Vahdat & Kautz, 2020) based on VAE, so applying these methods would solve these problems in future work.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we demonstrate our proposed method to generate visual explanations with classifying image datasets.

Original: We conduct an ablation study for the uncertainty layer using four measures, and the change in the number of prototype distributions is analyzed.

Original: Experiment details MNIST and Fashion-MNIST (Xiao et al., 2017) datasets are used for our experiments and  splited by 0.9, 0.1 for training dataset and test dataset.
Revised: Experiment details MNIST and Fashion-MNIST (Xiao et al., 2017) datasets are used for our experiments and are splited by 0.9, 0.1 for training dataset and test dataset.

Original: For training our model, we use Adam (Kingma & Ba, 2015) optimizer with learning rate to 0.0001 and weight decay to 0.00005, and set hyperparameter λ to 50.
Revised: For training our model, we use Adam (Kingma & Ba, 2015) optimizer with learning rate of 0.0001 and weight decay of 0.00005, and set hyperparameter λ to 50.

Original: To comparethe various VAE structures, we use vanilla 
Revised: To compare various VAE structures, we use vanilla VAE ( β = 1 , γ = 0 ), β -VAE ( β = 4 , γ = 0 )

Original: VAE ( β = 1 , γ = 0 ), β -VAE ( β = 4 , γ = 0 ) (Higgins et al., 2017) and FactorVAE ( β = 1 , γ = 6 . 4 ) (Kim & Mnih, 2018).
Revised: (Higgins et al., 2017) and FactorVAE ( β = 1 , γ = 6 . 4 ) (Kim & Mnih, 2018).

Original: We trained 500 epochs with VAE and β -VAE  about 2 hours, and about 3 hours are used with FactorVAE using a single 1080 Ti GPU. 
Revised: We train 500 epochs with VAE and β -VAE for about 2 hours, and about 3 hours were used with FactorVAE using a single 1080 Ti GPU. [Table]

Original: [Table]
Revised: [Table]

Original: 5. C LASSIFICATION ACCURACY
Revised: C LASSIFICATION ACCURACY

Original: The accuracy of our model on MNIST is 99.33 ± 0.04 % with providing visualized explanations as
Revised: The accuracy of our model on MNIST is 99.33 ± 0.04 % with providing visualized explanations in

Original: Figure 1.
Revised: Figure 1.

Original: Furthermore, as shown inthe Table 2, our model achieved testing accuracy 93.00 ± 0.13 % on Fashion-MNIST.
Revised: Furthermore, as shown in Table 2, our model achieved testing accuracy 93.00 ± 0.13 % on Fashion-MNIST.

Original: This result is comparable to non-interpretable classiﬁcation networks such as
Revised: This result is comparable to non-interpretable classiﬁcation networks such as

Original: VGG16 (93.5 % ) and GoogleNet (93.7 % ) reported on dataset website.
Revised: VGG16 (93.5 % ) and GoogleNet (93.7 % ) reported in the dataset website.

Original: Moreover, we achieve higher accuracy than  baseline model’s
Revised: Moreover, we achieve higher accuracy than the baseline model’s

Original: (Li et al., 2018) accuracy 91.03 ± 0.32 % that only provides basic explanation.
Revised: (Li et al., 2018) accuracy 91.03 ± 0.32 % that only provides basic explanation.

Original: We construct baseline model with same layers and same hyperparameters.
Revised: We construct baseline model with same layers and same hyperparameters.

Original: [Figure]
Revised: [Figure]

Original: As shown in Table 2, we conduct experiments with four measurements and three VAE structures.
Revised: As shown in Table 2, we conduct experiments with four measurements and three VAE structures.

Original: First, according to measures,the results ofthe experiments using Wasserstein-distance show the highest accuracy through three VAE structures.
Revised: According to measures, results of experiments using Wasserstein-distance show the highest accuracy through three VAE structures.

Original: In addition, it achieved the highest diversity and stability of the learned prototypes as it provides a usable gradient when distributions are supported on non-overlapping domains (Kolouri et al., 2019), indicating that it is the most suitable measurement in creating prototypes using the distance between distributions.
Revised: In addition, it achieves the highest diversity and stability of the learned prototypes as it provides a usable gradient when distributions are supported on non-overlapping domains (Kolouri et al., 2019). This indicates that Wasserstein-distance is the most suitable measurement in creating prototypes using distances between distributions.

Original: The role of the uncertainty layer to increase accuracy was also clearly revealed.
Revised: The role of the uncertainty layer to increase accuracy is also clearly revealed.

Original: In most of the 12 cases, accuracy increased when the uncertainty layer was applied.
Revised: In most of the 12 cases, accuracy increased when the uncertainty layer is applied.

Original: Among three VAE structures, the vanilla VAE showed the highest accuracy, but FactorVAE also do not showed signiﬁcant difference in accuracy.
Revised: Among three VAE structures, the vanilla VAE shows the highest accuracy, but FactorVAE also shows no signiﬁcant difference in accuracy.

Original: However, FactorVAE forms more reliable explanation space as Section 4.1, it was used to create Figures 1, 4, 5.
Revised: However, FactorVAE forms more reliable explanation space as Section 4.1, it is used to create Figures 1, 4, 5.

Original: 5. P ROVIDED EXPLANATIONS : B EYOND EXAMPLES
Revised: 5. P ROVIDED EXPLANATIONS : B EYOND EXAMPLES

Original: Our model generates explanations for prediction as Figure 4.
Revised: Our model generates explanations for prediction as Figure 4.

Original: In Figure 4(a), for the black T-shirt input image, our model shows prototypes with the lowest distance having shapes of tops.
Revised: In Figure 4(a), for the black T-shirt input image, our model shows prototypes with the lowest distance having shapes of tops.

Original: On the other hand, the distances from the prototypes that looked like bags, trousers, and shoes is higher than 6.5, indicating that the prototypes and distance information provide accurate explanations.
Revised: On the other hand, the distances from the prototypes that looked like bags, trousers, and shoes are higher than 6.5, indicating that the prototypes and distance information provide accurate explanations.

Original: Furthermore, a fully-connected layer is applied as a weight matrix corresponding to a similarity connection between these distances and logits, showing relevance between prototype distributions and classes.
Revised: Furthermore, a fully-connected layer is applied as a weight matrix corresponding to a similarity connection between these distances and logits, showing relevance between prototype distributions and classes.

Original: However, this explanation alone makes it difﬁcult to understand why the distance from the prototypes in the same T-shirt class is different and whether the different distances came out even if the prototypes were in the form of dresses.
Revised: However, this explanation alone makes it difﬁcult to understand why the distance from the prototypes in the same T-shirt class is different and whether the different distances came out even if the prototypes were in the form of dresses.

Original: Therefore, our method provides additional explanations for this basic explanation.
Revised: Therefore, our method provides additional explanations for this basic explanation.

Original: Additionally, our model provides the relationships between prototypes and inputs as an explanation of the factors that inﬂuenced distance, such as Figure 4(b).
Revised: Additionally, our model provides the relationships between prototypes and inputs as an explanation of the factors that inﬂuenced distance, such as Figure 4(b).

Original: For example, from relationships between  input and prototypes for class dress, the nearest prototype can be seen with only the thickness of the torso changing, followed by the next prototype with longer sleeves and the farthest prototype with no sleeves.
Revised: For example, from relationships between the input and prototypes for class dress, the nearest prototype can be seen with only the thickness of the torso changing, followed by the next prototype with longer sleeves and the farthest prototype with no sleeves.

Original: Since prototypes with changes in the sleeves having similar distances, we can assume that the model is sensitive to sleeves in the relationship between input and dress prototypes.
Revised: Since prototypes with changes in the sleeves have similar distances, we can assume that the model is sensitive to sleeves in the relationship between input and dress prototypes.

Original: The bigger the change in the sleeves of the clothes, the farther the distance becomes.
Revised: The bigger the change in the sleeves of the clothes, the farther the distance becomes.

Original: Also, relationships between prototypes are visualized as additional information for how the model discriminates prototypes, such as Figure 4(c).
Revised: In addition, relationships between prototypes are visualized as additional information for how the model discriminates prototypes, such as Figure 4(c).

Original: The user studywas also conducted to show that the relationship helpsthe understandingof model predictions.
Revised: We conduct a user study to show that the relationship helps understanding model predictions.

Original: Among the explanations obtained through our model, we tried to compare human understanding whengiven the (i) prototypes and distance as Figure 4(a), (ii) prototypes and distance, and the relationship between input and prototype as Figure 4(a), (b).
Revised: With the explanations obtained through our model, we tried to compare human understandings when (i) basic explanation (Figure 4(a)), (ii) basic explanation and the relationship between input and prototype (Figure 4(a), (b)) are given.

Original: For each set of explanations, we survey the case of correct classiﬁcation and misclassiﬁcation.
Revised: For each set of explanations, we survey the case of correct/wrong classiﬁcation.

Revised: We prepare the following questions to check whether users were able to respond to factors that inﬂuenced model predictions.

Original: In the case of correct classiﬁcation, we askabout the factors that affect the difference in the distance between prototypes within the same class.
Revised: In the case of correct classiﬁcation, we ask factors that affect differences in the distance between prototypes within the same class.

Original: In the case of misclassiﬁcation, we askedabout the factors that made the distance from the prototype of the wrong class close.
Revised: In the case of misclassiﬁcation, we asked factors that made the distance from the prototype of the wrong class close.

Original: The relationships between input and  prototypes provided in the survey are as follows: (i) Correct classiﬁcation: two prototypes of the same class as input.
Revised: The relationships with the input and following prototypes were given during the survey: (i) Correct classiﬁcation: two prototypes of the same class as input.

Original: (ii) 
Revised: (ii) Wrong classiﬁcation: prototype

Original: [Figure]
Revised: [Figure]

Original: Misclassiﬁcation: prototype closest to  input among misclassiﬁed classes, and prototype closest to  input among true classes.
Revised: closest to the input among misclassiﬁed classes, and prototype closest to the input among true classes.

Original: Additional information  is included in Appendix A.3.
Revised: Additional information about the survey is included in Appendix A.3.

Original: [Table]
Revised: [Table]

Original: We conducted two surveys through Amazon Mechanical Turk with 50 Mechanical Turk masters.
Revised: We conducted two surveys through Amazon Mechanical Turk with 50 Mechanical Turk masters.

Original: Each survey consists ofquestions, 6 per correct classiﬁcation and misclassiﬁcation.
Revised: Each survey consists of 12 questions, 6 per correct classiﬁcation and misclassiﬁcation.

Original: The average response rate for answering the factor is in Table 3.
Revised: The average response rate for answering the factor is in Table 3.

Original: This result shows that participants were better able to answer factors for model prediction when given the additional explanation of the relationship between inputs and prototypes.
Revised: This result shows that participants were better able to answer factors for model predictions when given the additional explanations of the relationship between inputs and prototypes.

Original: It shows that relationships increase human understanding of the model.
Revised: It shows that relationships increase human understandings of factors that compute distances.

Original: 5. C HANGING THE NUMBER OF PROTOTYPE DISTRIBUTIONS
Revised: 5. C HANGING THE NUMBER OF PROTOTYPE DISTRIBUTIONS

Original: As mentioned above, we choose optimal number of Gaussian Mixture Model’s components as a proper number of prototype distributions.
Revised: As mentioned above, we chose the optimal number of Gaussian Mixture Model’s components as a proper number of prototype distributions.

Original: We use the EM algorithm to ﬁt Gaussian Mixture Model to the dataset on latent space.
Revised: We use the EM algorithm to ﬁt Gaussian Mixture Model to the dataset on latent space.

Original: First, trained VAE encodes µ values of the entire dataset to place them on the latent space.
Revised: First, trained VAE encodes µ values of the entire dataset to place them on the latent space.

Original: Since our prototype distributions consist of independent multivariate Gaussian distributions, we used the Gaussian Mixture Model with each component has its own diagonal covariance matrix.
Revised: Since our prototype distributions consist of independent multivariate Gaussian distributions, we use the Gaussian Mixture Model with each component having its own diagonal covariance matrix.

Original: As the ﬁrst local minimum BIC score indicates strong evidence for the model (Fraley & Raftery, 1998), we determined that the number of components at the ﬁrst local minimum is the optimal number of components.
Revised: As the ﬁrst local minimum BIC score indicates strong evidence for the model (Fraley & Raftery, 1998), we determine that the number of components at the ﬁrst local minimum is the optimal number of components.

Original: The number of components obtained was 20.3, 19.8, 23.8 on average in VAE, β -VAE, and FactorVAE.
Revised: The number of components obtained is 20.3, 19.8, 23.8 on average in VAE, β -VAE, and FactorVAE.

Original: Through these results, the baseline number of prototype distributions is set as 20.
Revised: Through these results, the baseline number of prototype distributions is set as 20.

Original: To verify this, we experiment with numbers of prototype distributions m =
Revised: To verify this, we experiment with numbers of prototype distributions m =

Original: and the results are in Figure 5.
Revised: and the results are in Figure 5.

Original: In Figure 5(a), the accuracy begins to converge around 20.
Revised: In Figure 5(a), the accuracy begins to converge around 20.

Original: Also, when prototype numbers are small,  model converges extensively slowly.
Revised: In addition, when the number of prototypes are small, the model converges extensively slowly.

Original: In Figure 5(b), when the number of prototypes increases from less than the dataset class, various appearances appear.
Revised: In Figure 5(b), when the number of prototypes increases from less than the dataset class, various appearances appear.

Original: However, when the number of prototypes increases excessively, similar appearance prototypes are inevitably raised.
Revised: However, when the number of prototypes increases excessively, similar appearance prototypes are inevitably raised.

Original: These results demonstrate that it is proper to use the optimal number of components in Gaussian Mixture Modelas a reference for  prototype distributions.
Revised: These results demonstrate that it is proper to reference on the optimal number of components of the Gaussian Mixture Model for the number of prototype distributions.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: The main contribution of our paper is to further enhance human understanding of model predictions by providing additional explanations of relationships.
Revised: The main contribution of our paper is to further enhance human understanding of model predictions by providing additional explanations of relationships.

Original: Further to existing prototype-based explanation methods that generate explanations by prototype examples and distances, our model visualizes relationships between input and prototypes that can explain the factors relevant to distance.
Revised: Further to existing prototype-based explanation methods that generate explanations by prototype examples and distances, our model visualizes relationships between input and prototypes that can explain the factors relevant to distance.

Original: Moreover, due to creating a dense explanation space, the distance became possible to function as a numerical value that guarantees similarity, which led to increase reliability.
Revised: Moreover, due to creating a dense explanation space, the distance became possible to function as a numerical value that guarantees similarity, which led to increase reliability.

Original: ODEL A RCHITECTURE D ETAILS
Revised: ODEL A RCHITECTURE D ETAILS

Original: In our proposed model, we use vanilla VAE, β -VAE, and FactorVAE for the VAE layer.
Revised: In our proposed model, we use vanilla VAE, β -VAE, and FactorVAE for the VAE layer.

Original: For VAE and β -VAE, we use a Convolutional Neural network for the encoder and a Deconvolutional Neural
Revised: For VAE and β -VAE, we use a Convolutional Neural network for the encoder and a Deconvolutional Neural

Original: Network for the decoder.
Revised: Network for the decoder.

Original: Detailed architecture explanation is provided in Table 4.
Revised: Detailed architecture explanation is provided in Table 4.

Original: For FactorVAE, discriminator is additionally used for distinguishing whether the input was drawn from q ( z ) or ¯ q ( z ) :
Revised: For FactorVAE, discriminator is additionally used for distinguishing whether the input was drawn from q ( z ) or ¯ q ( z ) :

Original: = (cid:81) dj
Revised: = (cid:81) dj

Original: =1 q ( z j ) to calculate KL divergence between q ( z ) and ¯ q ( z ) .
Revised: =1 q ( z j ) to calculate KL divergence between q ( z ) and ¯ q ( z ) .

Original: The discriminator is made of 6 MLP layers with 1000 hidden units each and leaky ReLU is used for non-linearity, that generates 2 logits as outputs.
Revised: The discriminator is made of 6 MLP layers with 1000 hidden units each and leaky ReLU is used for non-linearity, that generates 2 logits as outputs.

Original: We use Adam optimizer for discriminator with learning rate 0.0001 and weight decay 0.00005.
Revised: We use Adam optimizer for discriminator with learning rate 0.0001 and weight decay 0.00005.

Original: A. D ETAILS FOR INTERACTIVE RECONSTRUCTION EXPERIMENT
Revised: A. D ETAILS FOR INTERACTIVE RECONSTRUCTION EXPERIMENT

Original: We constructed the experiments in the same way as in the paper Ross et al.
Revised: We constructed the experiments in the same way as in the paper Ross et al.

Original: Through the interactive reconstruction method shown in the paper, we evaluate the models: Li et al.
Revised: Through the interactive reconstruction method shown in the paper, we evaluate the models: Li et al.

Original: (2018), proposed model with VAE, and proposed model with FactorVAE.
Revised: (2018), proposed model with VAE, and proposed model with FactorVAE.

Original: The evaluation was conducted through Amazon Mechanical Turk, and for a fair evaluation, everyone paid $3 .
Revised: The evaluation was conducted through Amazon Mechanical Turk, and for a fair evaluation, everyone paid $3 .

Original: 75 per evaluation.
Revised: 75 per evaluation.

Original: In order to improve the quality of the evaluation, all experiments were conducted through Mechanical
Revised: In order to improve the quality of the evaluation, all experiments were conducted through Mechanical

Original: Turk Masters.
Revised: Turk Masters.

Original: The experiments were conducted with models trained through the MNIST dataset.
Revised: The experiments were conducted with models trained through the MNIST dataset.

Original: Success was considered if the user-generated image and the sampled image had a similarity of at least 75 % .
Revised: Success was considered if the user-generated image and the sampled image had a similarity of at least 75 % .

Original: In addition, it is designed so that users can skip the question only if they generate images for at leastseconds.
Revised: In addition, it is designed so that users can skip the question only if they generate images for at leastseconds.

Original: Experiments were conducted with 20 participants per model and 7 questions with random images were provided.
Revised: Experiments were conducted with 20 participants per model and 7 questions with random images were provided.

Original: An example of interactive reconstruction experiment is shown in Figure 6.
Revised: An example of interactive reconstruction experiment is shown in Figure 6.

Original: A. U SER STUDY ABOUT RELATIONSHIPS
Revised: A. U SER STUDY ABOUT RELATIONSHIPS

Original: The survey was conducted after showing precautions to the participants ﬁrst.
Revised: The survey was conducted after showing precautions to the participants ﬁrst.

Original: Participants were asked to answer the factors that inﬂuenced the model’s decision by looking at the given information.
Revised: Participants were asked to answer the factors that inﬂuenced the model’s decision by looking at the given information.

Original: At this time, since the given information may be insufﬁcient to judge the factor, two options were provided: the option of not knowing and the option of guessing.
Revised: At this time, since the given information may be insufﬁcient to judge the factor, two options were provided: the option of not knowing and the option of guessing.

Original: In the case of the participants who judged that they could guess factors, they were also asked to answer which factors they guessed.
Revised: In the case of the participants who judged that they could guess factors, they were also asked to answer which factors they guessed.

Original: Consent form and example of survey without relationships is shown in Figure 7.
Revised: Consent form and example of survey without relationships is shown in Figure 7.

Original: Consent form and example of survey with relationships is shown in Figure 8.
Revised: Consent form and example of survey with relationships is shown in Figure 8.

Original: A. D ETAIL EXPLANATIONS FOR MEASUREMENTS
Revised: A. D ETAIL EXPLANATIONS FOR MEASUREMENTS

Original: We use KL-divergence, Jensen-Shannon divergence (JSD), Jensen-Tsallis distance (JTD) and Wasserstein distance for measuring distance between prototype distributions D
Revised: We use KL-divergence, Jensen-Shannon divergence (JSD), Jensen-Tsallis distance (JTD) and Wasserstein distance for measuring distance between prototype distributions D

Original: = { d 1 , d 2 , . . .
Revised: = { d 1 , d 2 , . . .

Original: , d m } and encoded input distribution d z .
Revised: , d m } and encoded input distribution d z .

Original: Since it was not described in detail in this paper, we provide additional explanations in this section.
Revised: Since it was not described in detail in this paper, we provide additional explanations in this section.

Original: For two multivariate Gaussian distributions p = N ( µ 1 , Σ 1 ) , q = N ( µ 2 , Σ 2 )( µ i ∈ R d , Σ i ∈
Revised: For two multivariate Gaussian distributions p = N ( µ 1 , Σ 1 ) , q = N ( µ 2 , Σ 2 )( µ i ∈ R d , Σ i ∈

Original: R
Revised: R

Original: d × R d ) ,
Revised: d × R d ) ,

Original: KL-divergence is as follows:
Revised: KL-divergence is as follows:

Original: [Equation]
Revised: [Equation]

Original: From independent distributions D and encoded input distribution that has diagonal covariance matrix Σ i = diag( σ 1 i , . . .
Revised: From independent distributions D and encoded input distribution that has diagonal covariance matrix Σ i = diag( σ 1 i , . . .

Original: , σ di ) , the above equation simpliﬁes to:
Revised: , σ di ) , the above equation simpliﬁes to:

Original: [Equation]
Revised: [Equation]

Original: Through this equation, with m = 12 ( p + q ) , Jensen-Shannon divergence also simpliﬁes in the same manner because Jensen-Shannon divergence deﬁned as:
Revised: Through this equation, with m = 12 ( p + q ) , Jensen-Shannon divergence also simpliﬁes in the same manner because Jensen-Shannon divergence deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: Unlike JSD, Jensen-Tsallis divergence uses the Tsallis entropy instead of the Shannon entropy.
Revised: Unlike JSD, Jensen-Tsallis divergence uses the Tsallis entropy instead of the Shannon entropy.

Original: The
Revised: The

Original: Tsallis entropy S q,k ( p ) is generalization of the Shannon entropy which becomes Shannon entropy when q converges to 1 and k equals 1.
Revised: Tsallis entropy S q,k ( p ) is generalization of the Shannon entropy which becomes Shannon entropy when q converges to 1 and k equals 1.

Original: In previous works, they use speciﬁc Jensen-Tsallis divergence with q = 2 and k = 4 and refer to JTD since it satisﬁes the conditions of a metric.
Revised: In previous works, they use speciﬁc Jensen-Tsallis divergence with q = 2 and k = 4 and refer to JTD since it satisﬁes the conditions of a metric.

Original: In our condition,
Revised: In our condition,

Original: JTD is as follows:
Revised: JTD is as follows:

Original: [Equation]
Revised: [Equation]

Original: By using KL-divergence, JSD, and JTD as a distance measure, the proposed model learns prototypes as illustrated in Figure 9.
Revised: By using KL-divergence, JSD, and JTD as a distance measure, the proposed model learns prototypes as illustrated in Figure 9.

Original: Using these metric for distance, the model accuracy is high enough, but the prototypes do not form observable images, which is contrary to our purpose of explaining the model predictions.
Revised: Using these metric for distance, the model accuracy is high enough, but the prototypes do not form observable images, which is contrary to our purpose of explaining the model predictions.

Original: This is because these three measures fail to provide usable gradients when distributions are supported on non-overlapping domains.
Revised: This is because these three measures fail to provide usable gradients when distributions are supported on non-overlapping domains.

Original: For this reason, the model fails to provide usable gradients when the prototype distributions and the embedded input distribution do not overlap, thus not forming a proper latent space as shown in the Figure 9, which makes prototypes unstable.
Revised: For this reason, the model fails to provide usable gradients when the prototype distributions and the embedded input distribution do not overlap, thus not forming a proper latent space as shown in the Figure 9, which makes prototypes unstable.

Original: A. D ETAILED ANALYSIS DUE TO APPLICATION OF UNCERTAINTY LAYER
Revised: A. D ETAILED ANALYSIS DUE TO APPLICATION OF UNCERTAINTY LAYER

Original: In the process of forming the latent space in our proposed model, VAE formulates the reconstruction loss through the latent vector sampled from the embedded input distribution.
Revised: In the process of forming the latent space in our proposed model, VAE formulates the reconstruction loss through the latent vector sampled from the embedded input distribution.

Original: Since classiﬁcation was performed through the distance in the latent space, we tried to improve the classiﬁcation accuracy by lowering the uncertainty of the model.
Revised: Since classiﬁcation was performed through the distance in the latent space, we tried to improve the classiﬁcation accuracy by lowering the uncertainty of the model.

Original: By adding the calculation process to logits without changing the structure of the model, it was possible to predict uncertainty and increase the accuracy by making the model robust against adversarial perturbations.
Revised: By adding the calculation process to logits without changing the structure of the model, it was possible to predict uncertainty and increase the accuracy by making the model robust against adversarial perturbations.

Original: Furthermore, since the model is not trained to handle uncertain information, it not only increases accuracy but also makes similar features exist in similar positions in the latent space.
Revised: Furthermore, since the model is not trained to handle uncertain information, it not only increases accuracy but also makes similar features exist in similar positions in the latent space.

Original: The embedded dataset visualized with t-SNE is illustrated in
Revised: The embedded dataset visualized with t-SNE is illustrated in

Original: Figure 10 showing that data of the same class forms more aggregated clusters in latent space when the uncertainty layer is applied.
Revised: Figure 10 showing that data of the same class forms more aggregated clusters in latent space when the uncertainty layer is applied.

Original: Through this latent space property, prototypes form more obvious observations that can express the whole dataset better, as Figure 11, because prototype distributions are learned through distances at latent space.
Revised: Through this latent space property, prototypes form more obvious observations that can express the whole dataset better, as Figure 11, because prototype distributions are learned through distances at latent space.

Original: For detailed explanations of the uncertainty layer setting in our model is as follows.
Revised: For detailed explanations of the uncertainty layer setting in our model is as follows.

Original: Given a sample x i , let there be one-hot vector
Revised: Given a sample x i , let there be one-hot vector

Original: y i with ground truth class j , then y ij = 1 and y ik = 0 for all k (cid:54) =
Revised: y i with ground truth class j , then y ij = 1 and y ik = 0 for all k (cid:54) =

Original: j , and logit l i that comes through the distance measuring layer of the model.
Revised: j , and logit l i that comes through the distance measuring layer of the model.

Original: We compute the evidence e i as exponential of logit l i .
Revised: We compute the evidence e i as exponential of logit l i .

Original: Then use the evidence to derive the uncertainty probability u i
Revised: Then use the evidence to derive the uncertainty probability u i

Original: = K/S i , where total S i
Revised: = K/S i , where total S i

Original: = (cid:80) Kj
Revised: = (cid:80) Kj

Original: =1 ( e ij + 1) , and parameters of Dirichlet distribution
Revised: =1 ( e ij + 1) , and parameters of Dirichlet distribution

Original: α ik = e ik
Revised: α ik = e ik

Original: + 1 for class
Revised: + 1 for class

Original: [Table]
Revised: [Table]

Original: Input
Revised: Input

Original: [Table]
Revised: [Table]

Original: image
Revised: image

Original: Input z
Revised: Input z

Original: [Table]
Revised: [Table]

Original: R
Revised: R

Original: Conv2d (32 channels, 4 × 4 kernels, stride 2, pad 1)
Revised: Conv2d (32 channels, 4 × 4 kernels, stride 2, pad 1)

Original: Conv2d (256 channels, 1 × 1 kernels, stride 1, pad 0)
Revised: Conv2d (256 channels, 1 × 1 kernels, stride 1, pad 0)

Original: BatchNorm, LeakyReLU(0.2) BatchNorm, LeakyReLU(0.2)
Revised: BatchNorm, LeakyReLU(0.2) BatchNorm, LeakyReLU(0.2)

Original: Conv2d (32 channels, 4 × 4 kernels, stride 2, pad 1)
Revised: Conv2d (32 channels, 4 × 4 kernels, stride 2, pad 1)

Original: ConvTranspose2d (64 channels, 4 × 4 kernels, stride 1, pad 0)
Revised: ConvTranspose2d (64 channels, 4 × 4 kernels, stride 1, pad 0)

Original: BatchNorm, LeakyReLU(0.2) BatchNorm, LeakyReLU(0.2)
Revised: BatchNorm, LeakyReLU(0.2) BatchNorm, LeakyReLU(0.2)

Original: Conv2d (64 channels, 4 × 4 kernels, stride 2, pad 1)
Revised: Conv2d (64 channels, 4 × 4 kernels, stride 2, pad 1)

Original: ConvTranspose2d (32 channels, 4 × 4 kernels, stride 2, pad 1)
Revised: ConvTranspose2d (32 channels, 4 × 4 kernels, stride 2, pad 1)

Original: BatchNorm, LeakyReLU(0.2) BatchNorm, LeakyReLU(0.2)
Revised: BatchNorm, LeakyReLU(0.2) BatchNorm, LeakyReLU(0.2)

Original: Conv2d (256 channels, 4 × 4 kernels, stride 1, pad 0)
Revised: Conv2d (256 channels, 4 × 4 kernels, stride 1, pad 0)

Original: ConvTranspose2d (32 channels, 4 × 4 kernels, stride 2, pad 1)
Revised: ConvTranspose2d (32 channels, 4 × 4 kernels, stride 2, pad 1)

Original: BatchNorm, LeakyReLU(0.2)
Revised: BatchNorm, LeakyReLU(0.2)

Original: BatchNorm, LeakyReLU(0.2)
Revised: BatchNorm, LeakyReLU(0.2)

Original: Conv2d (2*10 channels, 1 × 1 kernels, stride 1, pad 0)
Revised: Conv2d (2*10 channels, 1 × 1 kernels, stride 1, pad 0)

Original: ConvTranspose2d (1 channels, 4 × 4 kernels, stride 2, pad 1)
Revised: ConvTranspose2d (1 channels, 4 × 4 kernels, stride 2, pad 1)

Original: Sigmoid
Revised: Sigmoid

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: k .
Revised: k .

Original: Through the obtained parameter α ik and S i , which is the sum of total evidence and class K , the cross-entropy loss changes as follows:
Revised: Through the obtained parameter α ik and S i , which is the sum of total evidence and class K , the cross-entropy loss changes as follows:

Original: [Equation]
Revised: [Equation]

Original: Where ψ ( · ) is a digamma function, and after this process it has the same effect as specifying the class of uncertainty.
Revised: Where ψ ( · ) is a digamma function, and after this process it has the same effect as specifying the class of uncertainty.

Original: Also, we add a regularizing term for misleading evidence.
Revised: Also, we add a regularizing term for misleading evidence.

Original: For predicted parameters α i with true class j, misleading evidence is represented as α ik for all k (cid:54) =
Revised: For predicted parameters α i with true class j, misleading evidence is represented as α ik for all k (cid:54) =

Original: j .
Revised: j .

Original: With equation 5,
Revised: With equation 5,

Original: we
Revised: we

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: regularize them using KL divergence with the uniform Dirichlet distribution as follows:
Revised: regularize them using KL divergence with the uniform Dirichlet distribution as follows:

Original: [Equation]
Revised: [Equation]

Original: where λ t is the annealing coefﬁcient that starts from 0 and becomes 1 over training epoch t = 50 ,
Revised: where λ t is the annealing coefﬁcient that starts from 0 and becomes 1 over training epoch t = 50 ,

Original: D ( p i | (cid:104) 1 , . .
Revised: D ( p i | (cid:104) 1 , . .

Original: , 1 (cid:105) ) is the uniform Dirichlet distribution, and ˜ α
Revised: , 1 (cid:105) ) is the uniform Dirichlet distribution, and ˜ α

Original: i =
Revised: i =

Original: y i
Revised: y i

Original: + (1 − y i ) (cid:12) α
Revised: + (1 − y i ) (cid:12) α

Original: i for removing non-misleading evidence.
Revised: i for removing non-misleading evidence.

Original: A. A DDITIONAL RESULTS FOR MNIST EXPERIMENTS
Revised: A. A DDITIONAL RESULTS FOR MNIST EXPERIMENTS

Original: As shown in Figure 12(a), prototypes with the shape of 0 are the closest to the input.
Revised: As shown in Figure 12(a), prototypes with the shape of 0 are the closest to the input.

Original: Distances to numbers such as 2, 3, 8, 9, which have curved parts similar to 0, are closer than the distance to numbers such as 1, 4, 7, which have sharp shapes.
Revised: Distances to numbers such as 2, 3, 8, 9, which have curved parts similar to 0, are closer than the distance to numbers such as 1, 4, 7, which have sharp shapes.

Original: Moreover, in the same class, the distance from the input was similar.
Revised: Moreover, in the same class, the distance from the input was similar.

Original: Further explanations are given by the relationships between input and prototypes and the relationships between each prototype, as shown in Figure 12(b), (c).
Revised: Further explanations are given by the relationships between input and prototypes and the relationships between each prototype, as shown in Figure 12(b), (c).

Original: We can infer
Revised: We can infer

Original: [Figure]
Revised: [Figure]

Original: Prototypes
Revised: Prototypes

Original: [Figure]
Revised: [Figure]

Original: ↑
Revised: ↑

Original: [Figure]
Revised: [Figure]

Original: Width
Revised: Width

Original: [Figure]
Revised: [Figure]

Original: Rotation
Revised: Rotation

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table] [Figure] [Table]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table] [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure][Figure] [Table] [Figure]
Revised: [Figure]