Title: VISUAL EXPLANATION FOR DEEP METRIC LEARNING

Abstract: This work explores the visual explanation for deep metric learning and its applications. As an important problem for learning representation, metric learning has attracted much attention recently, while the interpretation of such model is not as well studied as classification. To this end, we propose an intuitive idea to show where contributes the most to the overall similarity of two input images by decomposing the final activation. Instead of only providing the overall activation map of each image, we propose to generate point-to-point activation intensity between two images so that the relationship between different regions is uncovered. We show that the proposed framework can be directly deployed to a large range of metric learning applications and provides valuable information for understanding the model. Furthermore, our experiments show its effectiveness on two potential applications, i.e. cross-view pattern discovery and interactive retrieval.

1. INTRODUCTION:

Learning the similarity metrics between arbitrary images is a fundamental problem for a variety of tasks, such as image retrieval (Oh Song et al. (2016) ), verification (Schroff et al. (2015) ; Luo et al. ( 2019)), localization (Hu et al. (2018) ), video tracking (Bertinetto et al. (2016) ), etc. Recently the deep Siamese network (Chopra et al. (2005) ) based framework has become a standard architecture for metric learning and achieves exciting results on a wide range of applications (Wang et al. (2019) ). However, there are surprisingly few papers conducting visual analyses to explain why the learned similarity of a given image pair is high or low. Specifically, which part contributes the most to the similarity is a straightforward question and the answer can reveal important hidden information about the model as well as the data.

Previous visual explanation works mainly focus on the interpretation of deep neural network for classification (Springenberg et al. (2014) ; Zhou et al. (2016; 2018) ; Fong & Vedaldi (2017) ). Guided back propagation (Guided BP) (Springenberg et al. (2014) ) has been used for explanation by generating the gradient from prediction to input, which shows how much the output will change with a little change in each dimension of the input. Another representative visual explanation, class activation map (CAM) Zhou et al. (2016) , generates the heatmap of discriminative regions corresponding to a specific class based on the linearity of global average pooling (GAP) and fully connected (FC) layer. However, the original method only works on this specific architecture configuration and needs retraining for visualizing other applications. Based on the gradient of the last convolutional layer instead of the input, Grad-CAM (Selvaraju et al. (2017) ) is proposed to generate activation maps for all convolutional neural network (CNN) architectures. Besides, other existing methods explore network ablation (Zhou et al. (2018) ), the winner-take-all strategy (Zhang et al. (2018) ), inversion (Mahendran & Vedaldi (2015) ), and perturbation (Fong & Vedaldi (2017) ) for visual explanation.

Since the verification applications like person re-identification (re-ID) (Luo et al. (2019) ) usually train metric learning models along with classification, recent work (Yang et al. (2019) ) starts to leverage the classification activation map to help improve the overall performance, but the activation map of metric learning is still not well explored. For two given images, a variant of Grad-CAM has been used for visualization of image retrieval (Gordo & Larlus (2017) ) by computing the gradient from the cosine similarity of the embedding features to the last convolutional layers of both images. However, Grad-CAM only provides the overall highlighted regions of two input images, the relationship between each activated region of two images is yet to be uncovered. Since the similarity is calculated from two images and possibly based on several similar patterns between them, the relationship between these patterns is critical for understanding the model. In this paper, we propose an activation decomposition framework for visual explanation of deep metric learning and explore the relationship between each activated region by point-topoint activation response between two images. As shown in Fig. 1 , the overall activation map of the proposed method is generated by decomposing the similarity along each image. In this example, the query image (A) has a high activation on both the eyes and mouth areas, but the overall map of the retrieved image (B) only highlights the eyes. It is actually hard to understand how the model works only based on the overall maps. For image B, the mouth region (green point) has a low activation which means the activation between the mouth and the whole image A is low compared to the overall activation (similarity). However, by further decomposing this activation (green point) along image A, that is to give the activation between the mouth region of image B and each position in image A, the resulting partial (or point-specific) activation map (green box) reveals that the mouth region of image B still has a high response on the mouth region of image A. This partial activation map can be generated for each pixel, which renders the point-to-point activation intensity representing the relationship between regions in both images, e.g. eye-to-nose or mouth-to-mouth. The partial activation map provides much more refined information about the model which is crucial for explanation. The contributions are summarized as follows.

• We propose a novel explanation framework for deep metric learning architectures and it may serve as an analysis tool for a host of applications, e.g. face recognition, person re-ID.

• The proposed method uncovers the point-to-point activation intensity which is not explored by existing methods. Our experiments further show the importance of the partial activation map on several new applications, i.e. cross-view pattern discovery and interactive retrieval.

• Our analysis suggests that two widely believed arguments (Section 2.1, 4) about CAM and Grad-CAM are inaccurate. 2017)) are designed in classification context where the output score is generated from one input image. Among the approaches that do not require architecture change, one intuitive idea (Zhou et al. (2016) ; Zhang et al. (2018) ) is to look into the model and see where contributes the most to the final prediction. CAM and its variant Grad-CAM are among the most widely used approaches, but recent works simply consider CAM as a heuristic linear combination of convolutional feature maps which is limited to the original architecture. It is also believed (Zhou et al. (2016; 2018); Selvaraju et al. (2017) ) that CAM only applies to the specific architecture configuration of GAP and one FC layer (Argument 1). On the contrary, we argue that CAM is only a special case of activation decomposition on the specific architecture, and the idea applies to much more architectures, even beyond the classification problem. It is also believed that Grad-CAM is a generalization of CAM for arbitrary CNN architecture (Argument 2), but the original paper only provides the proof on CAM's architecture. We show that Grad-CAM is not equivalent to activation decomposition for some architectures (Section 4.3). Another direction of explanation is to check the black box model by modifying the input and observing the response in the prediction.

2. RELATED WORK:

A representative method Fong & Vedaldi (2017) aims to optimize the blurred region and see which region has the strongest response on the output signal when it is blurred. Perturbation optimization is applicable for any black box model, but the optimization can be computational expensive.

2.2. INTERPRETATION FOR METRIC LEARNING:

Guided BP (Springenberg et al. ( 2014)) can be easily adopted to metric learning, but recent works (Adebayo et al. (2018) ; Fong & Vedaldi (2017)) claim that gradient can be irrelevant to the model and guided-BP fails on the sanity check (Adebayo et al. (2018) ). Among methods that pass the sanity check, Grad-CAM has been used for visualization of image retrieval (Gordo & Larlus (2017) ), but no quantitative result is provided. Instead of adding more gradient heuristics like Grad-CAM++ (Chattopadhay et al. (2018) ), we propose to explain the result of Grad-CAM under the decomposition framework. As shown in Section 4.3, by removing the heuristic step in Grad-CAM, the meaning of the generated activation map can be explained as the first two terms of the proposed framework.

The perturbation optimization needs reformulation for metric learning and can be more computational expensive if the point-to-point activation map is desired. To better introduce our method, we first review the formulation of CAM and illustrate our idea on a simple architecture (CNN+GAP as in Fig. 2 ) for metric learning. As shown in Eq. 1, CAM is actually a spatial decomposition of the prediction score of each class, and the original method only applies for CNN with GAP and one FC layer without bias. The decomposition clearly shows how much each part of the input image contributes to the overall prediction of one class and provides valuable information about how the decision is made inside the classification model. The idea is based on the linearity of GAP:

3. ACTIVATION DECOMPOSITION ON A SIMPLE ARCHITECTURE:

EQUATION

Here S c denotes the overall score (before softmax) of class c and ω k,c is the FC layer parameter for the k-th channel of class c. A i,j,k denotes the feature map of the last convolutional layer at position (i, j), and Z is the normalization term of GAP. In fact, the result k ω k,c A i,j,k can be considered as a decomposition of S c along (i, j). For a two-stream architecture (see Fig. 2 ) with GAP and the cosine similarity metric (S) for metric learning application (e.g. image retrieval), we propose to do decomposition along (i, j, x, y) so that the relationship between different parts of two images are uncovered:

S = k GAP (A q k )GAP (A r k ) = 1 Z k ( i,j A q i,j,k x,y A r x,y,k ) = 1 Z i,j,x,y ( k A q i,j,k A r x,y,k ).

(2) Here the normalization terms of the cosine similarity (L2 norm) and GAP are included in Z. We use (i, j), (x, y) for different streams because cross-view applications (Hu et al. (2018) ) may have different image sizes for two streams. A denotes the feature map of the last convolutional layer. The superscripts q and r respectively denote the query and the retrieved image in this paper. For each query point (i, j) in the query image, the corresponding activation map in the retrieved image is given by k A q i,j,k A r x,y,k which is the contribution of features at (i, j, x, y) to the overall cosine similarity. Like CAM and Grad-CAM, bilinear interpolation is implemented to generate the contribution of each pixel pair and the full resolution map. The overall activation maps of two images are generated by a simple summation along (i, j) or (x, y). Although we only show the positive activation in the map, the negative value is still available for counterfactual explanation (Selvaraju et al. (2017) ).

Since recent works (Liu et al. (2017) ; Wang et al. (2018) ) have highlighted the superiority of L2 normalization, we use the cosine similarity S as the default metric. With L2 normalization, the squared Euclidean distance D equals to 2 -2S so that S and D are equivalent as metrics. Although there are still a number of existing works (Luo et al. (2019) ; Ristani & Tomasi (2018) ) training with the Euclidean distance without L2 normalization, Luo et al. (2019) shows that the cosine similarity still performs well as the evaluation metric. We further empirically show that the cosine similarity works well for this case (Section 5.4).

4. EXTENSION FOR COMPLEX ARCHITECTURES:

Recent metric learning approaches usually leverage more complex architectures to improve performance, e.g. adding several FC layers after the flattened feature or global pooling. Although different metric learning applications have different head architectures (e.g. GAP+FC) on CNN, the basic components are highly similar. We introduce a unified extension to make our method applicable to most existing state-of-the-art architectures for various applications, including image retrieval (Wang et al. (2019) ), face recognition (Deng et al. (2019) ), re-ID (Luo et al. ( 2019)) and geo-localization (Hu et al. (2018) ). Note that the extension also applies to classification architectures. In Section 4.1, we address linear components by considering them together as one linear transformation. Then Section 4.2 focuses on transforming nonlinear components to linear in the validation phase.

4.1. LINEAR COMPONENT:

The GAP can be considered as a special case of flattened layer (directly reshape the last convolutional layer A ∈ R m×n×p as one vector Â ∈ R mnp without pooling) multiplied by a transformation matrix T GAP ∈ R l×mnp where l denotes the length of the feature embedding vector (see Appendix A.3 for details). Without loss of generality, we consider a two-stream framework with flattened layer followed by one FC layer with weights Ŵ ∈ R l×mnp and biases B ∈ R l . Since the visual explanation is generated at the test phase when typical components such as FC layer and batch normalization (BN) are linear, all the linear components together are formulated as one linear transformation g( Â) = Ŵ Â+B = i,j W i,j A i,j +B in the FC layer. Here W i,j ∈ R l×p and A i,j ∈ R p denote the weights matrix and feature vector corresponding to position (i, j). Although B is ignored in CAM, we keep it as a residual term in the decomposition. Then Eq. 2 is re-formulated as:

SZ = g q (A q ) • g r (A r ) = ( i,j W q i,j A q i,j + B q ) • ( x,y W r x,y A r x,y + B r ) = i,j,x,y (W q i,j A q i,j ) • (W r x,y A r x,y ) + i,j (W q i,j A q i,j • B r ) + x,y (W r x,y A r x,y • B q ) + B q • B r .

(3)

Here Z denotes the normalization term for cosine similarity (L2 norm), and • is the inner product. The decomposition of S has four terms, and the last three terms contain the bias B. The first term clearly shows the activation response for location pair (i, j, x, y) and is considered as the point-topoint activation in this paper. The second and third terms correspond to the activation between one image and the bias of the other image. Although they can be considered as negligible bias term, they actually contribute to the overall activation map. For the overall activation map of the query image, the second term can be included, because W q i,j A q i,j • B r varies at different (i, j) positions while the third and last terms stay unchanged. Similarly, the first and third terms are considered when calculating the overall map for the retrieved image. We investigate both settings about the bias term (i.e. w or w/o) on the overall activation map and they are referred to as "Decomposition": (W q i,j A q i,j ) • ( x,y W r x,y A r x,y ), and "Decomposition+Bias" : (W q i,j A q i,j ) • ( x,y W r x,y A r x,y + B r ) in Section 5. The last term is pure bias which is the same for every input image. Therefore, we ignore this term if not mentioned. Non-linear transformation increases the difficulty of activation decomposition, fortunately, the most widely used non-linear components -global maximum pooling (GMP) and rectified linear unit (ReLU) -can be transformed as linear operations in the validation phase by multiplying a mask1 . We only focus on GMP and ReLU in this paper, and all the mentioned components together have covered all the popular architectures for deep metric learning applications. In the validation phase, the GMP can be considered as a combination of a maximum mask M and GAP or flattened layer as shown in Fig. 3 . The result of Hadamard product (M A) is considered as the new feature map which can be directly applied to Eq. 3. Similarly, by adding a mask for ReLU, the FC layer with ReLU can be included in W and B in the validation phase.

4.2. NON-LINEAR COMPONENT:



4.3. RELATIONSHIP WITH GRAD-CAM:

Grad-CAM is another way for generating overall activation map on more complex architecture. When we calculate the activation map based on Grad-CAM, what do we get? Selvaraju et al. ( 2017) has shown that Grad-CAM is equivalent to CAM on GAP based architecture. However, for a classification architecture with flattened feature and one FC layer, the prediction score for class c may be formulated as S c = i,j,k W i,j,k,c A i,j,k where W is the reshaped weights of FC layer following the shape of the last convolutional feature (A). From our activation decomposition perspective, the contribution of each position (i, j) is clearly given by k W i,j,k,c A i,j,k , while the Grad-CAM map for class c at (i, j) position is given by:

EQUATION

In this case, the result of Grad-CAM is different from activation decomposition because of the GAP in Eq. 4. For architectures using GMP, they are also different, because only the maximal value of each channel contributes to the overall prediction score, while Grad-CAM puts the same weight for features at all positions (see Appendix A.4 for details). Although Selvaraju et al. ( 2017) empirically shows that the heuristic GAP step can help improve the performance of localization, this step makes the meaning of the generated map unclear. By removing this step, which means combining the gradient and feature directly as A i,j,k ∂Sc ∂A i,j,k , the modified version would generate the same result as activation decomposition for flattened and GMP based architectures. As for the metric learning architecture discussed in Section 4.1, the Grad-CAM map of query image is written as (the derivation is included in Appendix A.5):

EQUATION

E q denotes the embedding vector of query image before L2 normalization and Z is the normalization term. The gradient term

∂(E q /|E q |) ∂E q

, which comes from the L2 normalization, would put less weights on dominant channels so that the generated activation map becomes more scattering (Appendix A.5). It can be removed by calculating gradient from E q • E r without L2 normalization. If we remove the gradient term as well as the GAP term, the result is actually equivalent to the overall map of "Decomposition+Bias" given by the first two terms of Eq. 3 as (W q i,j A q i,j ) • ( x,y W r x,y A r x,y + B r ).

5. EXPERIMENT:



5.1. WEAKLY SUPERVISED LOCALIZATION:

Weakly supervised localization has been adopted as evaluation metric in most existing works, we thus conduct localization experiment in the context of metric learning to evaluate the overall ac-tivation map. We follow the state-of-the-art image retrieval method Wang et al. (2019) on CUB (Welinder et al. (2010) ), which contains over 10,000 images from 200 bird spices. CUB is a challenging dataset for metric learning and the bounding box is available for localization evaluation.

We first train a model on CUB with the loss function and architecture of Wang et al. (2019) , and then conduct the weakly supervised localization to show the differences between the variants of proposed methods. As CAM and Grad-CAM generate the mask with different threshold settings (0.2 and 0.15), we use multiple thresholds and find the result is sensitive to threshold (see Appendix A.1). We report the best result of each method in Table1. The "Grad-CAM (no norm)" means the Grad-CAM with gradient computed from the product of two embedding vectors without normalization and the "Decomposition+Bias" denotes the first two terms of Eq. 3. The proposed framework outperforms Grad-CAM for metric learning as well as CAM based on classification. As justified in Section 4.3, computing the gradients before normalization does improve the performance of Grad-CAM by a large margin. Since the architecture of Wang et al. ( 2019) is based on GMP, "Grad-CAM (no norm)" is not equivalent to "Decomposition+Bias" as shown in Section 4.3. 3 also support the fact that "Triplet" model is more likely to focus on the background region rather than the object as we witness the localization accuracy drop for Triplet loss. Although "Decomposition+Bias" can be more sensitive to different loss functions, implying that the bias term does provide valuable information on overall map, both settings of the proposed method have the same trend on accuracy. Therefore, our activation decomposition framework can help shed light on the generalization ability of loss functions for metric learning. 2017)), the model is capable of learning similar patterns in different views which may provide geometric information of the two views, e.g. the camera pose or orientation information. We conduct orientation estimation experiment to show the advantage of partial activation map compared with the overall activation map on providing geometric information based on cross-view patterns. In our experiment, we take street-to-aerial image geo-localization as an example and train a Siamese network following the loss function of Hu et al. (2018) . The objective of geo-localization is to find the best matched aerial-view image in a reference dataset for a query street-view image. We conduct the experiment on CVUSA (Zhai et al. ( 2017)), which is the most popular benchmark for this problem containing 35,532 training pairs and 8,884 test pairs. Each pair consists of a query street image and the corresponding orientation-aligned aerial image at the same GPS location. The orientation example is illustrated as angle degree in Fig. 6 (the yellow lines on both images correspond to 0 • ). We train the Siamese image matching model with randomly rotated aerial images so that the activation map is approximately rotation invariant for aerial image. As shown in Fig. 6 , the overall activation map may contain multiple highlighted regions contributing to the overall matching/similarity score. Our method can further provide detailed point-to-point relationship which is critical for orientation estimation. Here we utilize the maximum activated region for orientation estimation to show the superiority of the partial map compared with the overall map. We first generate the overall activation map as well as the partial map corresponding to the maximum activation position of the query street-view image. For both activation maps, the activation peaks from different views are selected for orientation estimation as shown in Fig. 6 . In this example, the overall activation maps contain multiple highlighted regions and the relationship between them are unclear. The activation peaks in the two views (the cyan boxes) actually correspond to different objects which leads to a failure estimation. However, the partial activation map clearly shows the relationship between these regions and provides more accurate estimation (the red line). We also present the quantitative results in Fig. 7 . As can be seen from this figure, the partial activation map significantly outperforms the overall map for cross-view orientation estimation. Partial activation based orientation estimation has over 16% samples with angle error less than ±3.5 • (the red bar at 0 • ), while overall map based method only has less than 12%.

5.4. APPLICATION II: INTERACTIVE RETRIEVAL:

Verification applications like face recognition and re-ID usually retrieve images with a complete query image. However, the complete query image may be not available in some scenarios. For example, only part of a person is captured in surveillance image/video due to occlusion or viewpoint. Interactive retrieval, which retrieves images with interactively selected region of the query image, can provide critical information for such circumstance. Since our framework provides pointto-point activation map, it can be served as a reasonable tool for measuring partial similarity. With the partial similarity, we are able to interactively retrieve images having similar parts with the query image instead of the overall most similar image, which may contain irrelevant or noisy information. As shown in Fig. 8 , the partial activation generated by the proposed method works well on retrieving people with similar clothes and faces with similar beard. We follow the pipeline of recent approaches on face recognition (Deng et al. (2019) ) and person re-identification (Luo et al. ( 2019)). For re-ID, the model is trained and evaluated on Market-1501 (Zheng et al. (2015) ) where some validation images only contain a small part of a person. Although the model is trained with Euclidean distance without L2 normalization, cosine similarity still works well as the evaluation metric.

Overall Partial:

Figure 9 : Explanation for the failure case by decomposition.

For face recognition, we take the trained model on CASIA-WebFace (Yi et al. (2014) ) and evaluate the model on FIW (Robinson et al. ( 2018)) where the face identification and kinship relationship are available. The interactive search is adopted by simply matching the equivalent partial feature in Eq. 3 (W q i,j A q i,j ) with the reference embedding features. Fig. 8 shows that different images are retrieved when searching with different partial features on the same query image. However, we do find some failure cases as shown in the last row of Fig. 8 with the red box. There is no shoes in the failure image, but why is this image retrieved with a top rank? In the activation map of Fig. 9 , three regions of the query image have a high activation on the retrieved image. And there is a high activation on the purse in the retrieved image corresponding to the shoes in the query image. This might be because of their similar red color and the arm of the retrieved person may appears like a leg from a specific viewpoint. This example also validates the importance of partial activation map generated by our framework for explanation.

6. CONCLUSION:

We propose a simple yet effective framework for visual explanation of deep metric learning based on the idea of activation decomposition. The framework is applicable to a host of applications, e.g. image retrieval, face recognition, person re-ID, geo-localization, etc. Experiments show the importance of visual explanation for metric learning as well as the superiority of both the overall and partial activation map generated by the proposed method. Furthermore, we introduce two potential applications, i.e. cross-view pattern discovery and interactive retrieval, which highlight the importance of partial activation map.

A APPENDIX:

A.1 WEAKLY SUPERVISED LOCALIZATION DETAILS As discussed in Appendix A.4, the activation map of Grad-CAM on GMP architecture is more scattering, as larger threshold is needed for Grad-CAM to generate comparable result in our experiment. For the feature A ∈ R m×n×p , the GAP (global average pooling) is equivalent to the flattened feature Â ∈ R mnp followed by transformation matrix T GAP ∈ R p×mnp :

EQUATION

Here (i, j) denotes the spatial coordinates. By reshaping the T GAP to p × m × n × p as T * GAP , the matrix is given by:

EQUATION

The matrix T GAP is simply calculated by reshaping T * GAP to p × mnp. As shown in Section 4.2, GMP is equivalent to GAP with a maximum mask in validation phase. The transformation matrix of GMP (global maximum pooling) is given by:

EQUATION

Here M ∈ R m×n×p is the maximum matrix of A where only the maximum position of each channel has nonzero value as 1. T GM P is computed by reshaping T * GM P to p × mnp.

A.4 GRAD-CAM ON GMP:

Considering a classification architecture with GMP and FC (W ∈ R p×l and no bias), the prediction score for class c is given by S c = k max i,j (A :,:,k )W k,c = i,j,k A i,j,k M i,j,k W k,c , where M is the maximum matrix in Appendix A.3. The decomposition of position (i, j) is k A i,j,k M i,j,k W k,c and only the maximum position of each channel has a nonzero value. However, for Grad-CAM, the activation map is given by:

EQUATION

where all (i,j) positions have a nonzero weight resulting in a more scattering activation map.

A.5 GRAD-CAM FOR METRIC LEARNING For metric learning architecture like Section 4.1, the similarity is formulated as S = E q •E r |E q ||E r | , where E q ∈ R l and E r ∈ R l are the embedding vector of query and retrieved image. |x| denotes the L2 norm and a • b is the inner product of a and b. The Grad-CAM map of query image is given by: GradCAM i,j = A q i,j • GAP ( ∂S ∂A q ) = (GAP (

EQUATION

With the gradient chain rule, the gradient is written as:

EQUATION

By expanding E r as x,y W r x,y A r x,y + B r (Section 4.1) and merging Eq. 10 with Eq. 11, the Grad-CAM map is reformulated as:

EQUATION

Here the Z is the normalization term for simplicity. ∂E/|E| ∂E is the l × l Jacobian matrix given by:

EQUATION

)

1 |E| is the normalization term. For dominant channel i, the weight (1 -

E 2 i |E| 2

) is small resulting in a more scattering activation map.

A.6 QUALITATIVE RESULTS:

We provide more qualitative results of using our activation decomposition framework for the two new applications, i.e., cross-view pattern discovery and interactive retrieval. 

:

The mask is computed once for each input image.

Figures and Tables:

Figure 1: An overview of activation decomposition. The overall activation map on each image highlights the regions contributing the most to the similarity. The partial activation map highlights the regions in one image that have large activation responses on a specific position in the other image, e.g. mouth or eye.

Figure 2: Activation decomposition on a simple architecture for deep metric learning.

Figure 3: Maximum mask for GMP. Z is the constant normalization term for GAP.

Figure 4: Qualitative results on CUB.

Figure 5: Qualitative results for model diagnosis.

Example of cross-view pattern discovery, e.g. image orientation estimation. (Best viewed in color) When the metric learning is applied to cross-view applications, e.g. image retrieval (Zhai et al. (2017); Hu et al. (2018); Tian et al. (

Figure 7: Estimation error distribution of overall and partial map. (Better viewed with zoom in)

Figure 8: Top retrieved images by interactive retrieval on face and re-ID datasets. The red box on the left column indicates the query part.

Figure 11: Top retrieved images by interactive retrieval on face recognition.

Localization accuracy for 0.5 IOU on CUB validation set.

Table content: <table><tr><td/><td>Method</td><td>Accuracy</td><td/><td/><td/><td/><td/></tr><tr><td>Classification</td><td>CAM (Zhou et al. (2016))</td><td>41.0%</td><td/><td/><td/><td/><td/></tr><tr><td/><td>Grad-CAM</td><td>16.7%</td><td/><td/><td/><td/><td/></tr><tr><td>Metric Learning</td><td>Grad-CAM (no norm) Decomposition+Bias (ours)</td><td>48.3% 44.9%</td><td>Input</td><td>GradCAM</td><td>(no norm) GradCAM</td><td>+Bias Decomposition</td><td>Decomposition</td></tr><tr><td/><td>Decomposition (ours)</td><td>50.6%</td><td/><td/><td/><td/><td/></tr></table>

Top-1 recall accuracy on CUB.

Table content: <table><tr><td>Loss</td><td>Train</td><td>Validation</td></tr><tr><td>MS</td><td>82.69%</td><td>65.45%</td></tr><tr><td colspan="2">Triplet 82.98%</td><td>60.55%</td></tr></table>

Localization accuracy (0.5 IOU) on CUB training set.

Table content: <table><tr><td/><td/><td/><td colspan="2">Decomposition+Bias Decomposition</td></tr><tr><td/><td/><td>MS</td><td>52.10%</td><td>58.99%</td></tr><tr><td/><td/><td>Triplet</td><td>39.93%</td><td>55.60%</td></tr><tr><td/><td/><td>Accuracy Drop</td><td>12.17%</td><td>3.39%</td></tr><tr><td/><td>Multiple Similarity Loss</td><td>Triplet Loss</td><td>Multiple Similarity Loss</td><td>Triplet Loss</td></tr><tr><td>Input</td><td colspan="2">Decomposition</td><td colspan="2">Decomposition+Bias</td></tr></table>

Localization accuracy (IOU 0.5) with different thresholds on CUB validation set.

Table content: <table><tr><td colspan="5">Threshold Grad-CAM Grad-CAM(no norm) Decomposition+Bias Decomposition</td></tr><tr><td>0.15</td><td>16.71%</td><td>16.83%</td><td>23.49%</td><td>17.38%</td></tr><tr><td>0.2</td><td>16.26%</td><td>17.06%</td><td>32.01%</td><td>19.77%</td></tr><tr><td>0.3</td><td>14.48%</td><td>21.59%</td><td>44.94%</td><td>34.92%</td></tr><tr><td>0.4</td><td>9.43%</td><td>35.01%</td><td>37.85%</td><td>50.64%</td></tr><tr><td>0.5</td><td>4.43%</td><td>47.00%</td><td>21.99%</td><td>45.78%</td></tr><tr><td>0.6</td><td>1.54%</td><td>48.27%</td><td>9.35%</td><td>27.98%</td></tr><tr><td>0.7</td><td>0.27%</td><td>20.90%</td><td>2.25%</td><td>9.11%</td></tr><tr><td colspan="2">A.2 MODEL DIAGNOSIS DETAILS</td><td/><td/><td/></tr></table>

Localization accuracy (IOU 0.5) on CUB training set with MS (multiple similarity) loss.

Table content: <table><tr><td colspan="3">Threshold Decomposition+Bias Decomposition</td></tr><tr><td>0.2</td><td>41.05%</td><td>26.16%</td></tr><tr><td>0.3</td><td>52.10%</td><td>44.41%</td></tr><tr><td>0.4</td><td>42.77%</td><td>58.99%</td></tr><tr><td>0.5</td><td>24.97%</td><td>51.17%</td></tr><tr><td>0.6</td><td>9.35%</td><td>27.98%</td></tr></table>

Localization accuracy (IOU 0.5) on CUB training set with Triplet loss.

Table content: <table><tr><td colspan="3">Threshold Decomposition+Bias Decomposition</td></tr><tr><td>0.2</td><td>36.60%</td><td>24.00%</td></tr><tr><td>0.3</td><td>39.93%</td><td>38.57%</td></tr><tr><td>0.4</td><td>29.03%</td><td>55.60%</td></tr><tr><td>0.5</td><td>14.22%</td><td>53.49%</td></tr><tr><td>0.6</td><td>4.93%</td><td>32.49%</td></tr><tr><td colspan="2">A.3 TRANSFORMATION MATRIX FOR GAP AND GMP</td><td/></tr></table>

Peer review (iclr2020):
This work proposes a novel approach for visualizing the predictions of neural network models on pairwise tasks, e.g. predicting whether two images are similar. The authors show that to see similarity between two images down on the pixel/region level, it is not sufficient to apply methods like Grad-CAM which do not aim for decomposition. Instead, the authors* approach namely targets decomposition, and shows the benefit of decomposition through intuitive examples and qualitative results. The authors also quantitatively show the benefit of their method. First, they measure the performance of their method vs CAM and Grad-CAM, on the weakly supervised localization (WSL) task. They also show how their method reveals the disadvantages of standard triplet loss compared to a recent metric learning loss. My concerns: 1) While showing performance on WSL is appealing as it allows for a way to quantitatively evaluate the method, I wonder if there are other ways to evaluate, that explicitly measure the quality of the proposed technique for allowing interpretability and understanding of the base model*s performance. 2) The Triplet vs MS experiment is interesting, but I*m not sure this is the most convincing way to show that this proposed visualization technique is better than something else. Just because something shows one method is worse than another, doesn*t mean that this better/worse assessment is accurate. Further, how would Grad-CAM do on the same task? 3) The retrieval experiment only shows qualitative results, and again, no baseline is compared. 4) Similarity is a relative judgement; it*s hard to say if two items are similar, but easier to say if A and B are more similar than A and C. It seems the proposed method doesn*t consider negatives, which is perhaps a limitation.