Question: How does/should the explanation respond to similar inputs?
Evidence: Interpretable explanation models have to provide consistent explanations for similar inputs for stability. This is because the reliability of explanations is not guaranteed if different explanations are provided for similar inputs. To demonstrate the stability of our method, we measure the changes in distances provided as an explanation when the input is distorted. For fairness of measurements, the distance change is normalized by distance before the input is distorted. We measure these distance changes for each prototype of all inputs. In Table 5, our method shows much more stability for input distortions. In particular, there is a large difference in terms of average and median in the case of the baseline model, which means explanations that provide significantly different distances exist when the inputs are distorted. Based on our experiment, we claim our method provides a more reliable explanation to users than the baseline model.
Answer: As mentioned in [1], which you referenced, similar explanations should be provided for similar inputs. This leads to the stability of the explanation noted in [1]. We conduct additional experiments to demonstrate the stability of our explanation. In a prototype-based method, the explanation changes through a change in distance as the input changes. After distorting the input, we measure the changes in distances.

For fairness of measurements, the (normalized) distance change is calculated by |"distance from the prototype obtained from the input" – "distance from the prototype obtained from the distorted input"| / "distance from the prototype obtained from the input". We measure these distance changes for each prototype of all inputs. We also used torchvision's RandomPerspective function to distort the input while maintaining a similar shape. The distance change values obtained in this way are as follows.

Distortion FactorVAE - median FactorVAE - avg VAE - median VAE - avg Baseline - median Baseline - avg
0.1 0.0222 0.0294 0.0293 0.0390 0.0701 0.1580
0.2 0.0421 0.0535 0.0595 0.0735 0.1535 0.4666
0.3 0.0502 0.0634 0.0730 0.0923 0.1908 0.6697
0.4 0.0630 0.0784 0.0907 0.1095 0.2575 1.1177
0.5 0.0732 0.0921 0.1044 0.1283 0.3224 1.6922
From the above results, it is observed that our model shows much more stable results for input distortions. In particular, in the case of the baseline model, there is a large difference in terms of average and median, which means that there are explanations that provide significantly different distances when the inputs are distorted. This may be because, as we claimed, similar latent representations did not guarantee the similarity of shapes in existing models. Based on our experimental results, it is claimed that the explanation provided by our model is more “stable”. Our explanation shows better stability by providing similar distances for similar inputs, and it will be more reliable.

The above table will be produced as a plot with the X-axis as distortion, and the Y-axis as mean and median. We will add the above description with a plot to section 4.1.

We greatly appreciate this particular reviewer’s comment because it was helpful for us to create a new quantitative metric, and to evaluate the quality of explanations.