Analogies and Feature Attributions for Model Agnostic Explanation of Similarity Learners
Authors: Karthikeyan Natesan Ramamurthy
Amit Dhurandhar
Dennis Wei
Zaid Bin Tariq
Venue: ICLR 2022 Submitted (2022)

Abstract: Post-hoc explanations for black box models have been studied extensively in classification and regression settings. However, explanations for models that output similarity between two inputs have received comparatively lesser attention. In this paper, we provide model agnostic local explanations for similarity learners applicable to tabular and text data. We first propose a method that provides feature attributions to explain the similarity between a pair of inputs as determined by a black box similarity learner. We then propose analogies as a new form of explanation in machine learning. Here the goal is to identify diverse analogous pairs of examples that share the same level of similarity as the input pair and provide insight into (latent) factors underlying the model's prediction. The selection of analogies can optionally leverage feature attributions, thus connecting the two forms of explanation while still maintaining complementarity. We prove that our analogy objective function is submodular, making the search for good-quality analogies efficient. We apply the proposed approaches to explain similarities between sentences as predicted by a state-of-the-art sentence encoder, and between patients in a healthcare utilization application. Efficacy is measured through quantitative evaluations, a careful user study, and examples of explanations.

Under review as a conference paper at ICLR 2022
ANALOGIES AND FEATURE ATTRIBUTIONS FOR MODEL AGNOSTIC EXPLANATION OF SIMILARITY LEARNERS
Anonymous authors Paper under double-blind review
ABSTRACT
Post-hoc explanations for black box models have been studied extensively in classification and regression settings. However, explanations for models that output similarity between two inputs have received comparatively lesser attention. In this paper, we provide model agnostic local explanations for similarity learners applicable to tabular and text data. We first propose a method that provides feature attributions to explain the similarity between a pair of inputs as determined by black box similarity learner. We then propose analogies explanation machine lcarning. Here the goal to identify diverse analogous pairs of examples that share the same level of similarity the input pair and provide insight into (latent) factors underlying the model's prediction. The selection analogies can optionally leverage feature attributions, thus connecting the two forms of explanation while still maintaining complementarity. We prove that our analogy objective function is submodular, making the search for analogies efficient. We apply the proposed approaches to explain similarities between sentences as predicted by a state-of-the-att sentence encoder, and between patients in a healthcare utilization application. Efficacy is measured through quantitative evaluations, a careful user study, and examples of explanations.
new form of
INTRODUCTION
The goal of a similarity function to quantify the similarity between two objects. The learning of similarity functions from labeled examples, or equivalently distance functions, has traditionally been studied within the area of similarity or metric learning (ul3s With the advent of deep learning, learning complex similarity functions has found its way into additional important applications such as health care informatics, face recognition, handwriting analysis/signature verification, and search engine query matching. For example, learning pairwise similarity between patients in Electronic Health Records (EHR) helps doctors in diagnosing and treating future patients (Zhu et al. 2016).
Although deep similarity models may better quantify similarity, the complexity of these models could make them harder to trust. For decision-critee systems like patient diagnosis and treatment, it would be helpful for users understand why black box model assigns certain level of similarity two objects. Providing explanations for similarity models is therefore an important problem.
Explanations for ML models have been studied extensively in classification and regression settings. Local explanations in particular have received a lot of attention (Ribeiro et al. 2016 Lundberg Lee 2017 Selvaraju al. 2020) given that entities (viz. individuals) are primarily interested in understanding why a certain decision was made for them, and building globally interpretable surrogates for black box model typically much more challenging. Local explanations can uncover potential issues such reliance hence aiding in debugging a black box model. Appropriately aggregating local explanations can also provide reasonable global understanding (van der Linden et al. 2019 Ramamurthy al. 2020).
unimportant or unfair factors in region of the input space,
In this paper, we develop model-agnostic local explanation methods for similarity learners, which is a relatively under-explored area. Given black box similarity learner and a pair of in- puts, our first method produces feature attributions for the output of the black box. We dis- cuss why the direct application of LIME (Ribeiro al., 2016) and other first-order methods

---

Under review as a conference paper at ICLR 2022
is less satisfactory for similarity models. We then propose a quadratic approximation using Mahalanobis distance. A simplified example of the output is shown as shading in Figure Our second contribution is to propose p novel
type of explanation in the form of analogies for given input pair. The importance of analogy- based explanations was recently advocated for by (Hullermeier) 2020). We formalize this with an objective function that captures the intuitive desiderata of (1) closeness in degree of simi-
Input: (block box distance: 0.19)
a) As well the dolphin scheme, the chaos has allowed foreign companies to angage in damaging logging and fishing operations without proper monitoring export controls.
b) chaernalchhos has a allowed foreign companies set damaging commercaal logging and fishing operations without proper moni- toring or export controls.
Analogy:
(black box distance: 0.21)
larity to the input pair, (2) diversity among the
a) Singapore is already the United States' 12th-largest trading part- ner, with two way trade totaling more than $34 billion.
analogous pairs, and (3) a notion of analogy, i.e.,
b) Although small city-state, Singapore is the 12th-largest trading partner of the United States, with trade volume of $33.4 billion last year.
members of each analogous pair have a similar relationship to each other as members of the in- put pair. We prove that this objective is submod- ular, making it efficient to find good analogies within large dataset. An example analogy is shown in Figure [ The analogy is understand- able since, in the input one of the sentences provides more context (i.e. presence of a fraud- ulent scheme called "dolphin scheme"), similar to the analogy where Singapore being a "small city-state" " is the additional context. This thus suggests that analogies can uncover appropri- ate (latent) factors to explain predictions, which may not be apparent from explicit features such as words/phrases. The proposed feature- and amalogy-based methods are applied to text and tabular data, to explain similarities between i) sentences from the Semantic Textual Similarity (STS) dataset (Cer et al.) 2017), ii) patients in terms of their healthcare utilization using Medical Expenditure Panel Survey (MEPS) data, and iii) iris species (IRIS). The proposed methods outperform feature- and exemplar-bass based baselines in both the quantitative evaluation and human user study we conduct, showing high fidelity to the black box similarity learner and providing reasons that users find sensible. We also present examples of feature- and analogy- -based explanations and illustrate specific insights.
Figure 1: Our bastd and analogy- -baeed explanations of similarity between two input sen- tences. The former is represented by shading (darker is more important) derived from row sums of the matrix in Figure 2b The latter suggests that the presence of more context in one of the sen- tences dolphin scheme in the input, Singapore being a small city-state in the analogy is impor- tant in explaining the black box's similarity score, whereas details such as the particular scheme or words such as "foreign companies" " or "fishing op- erations" may not be required.
PROBLEM DESCRIPTION
Given a pair of examples x = (x1122) T & R where m is the dimension of the space, and black box model (.) 1TM é R + R. our goal is to "explain" the prediction δBB (x) of the black box model. One type of explanation takes the form of sparse set of features (i.e. < m if m is large) that are most important in determining the output, together possibly with weights to quantify their importance. An alternative form of explanation consists of other example pairs that have the same (or similar) output from the black box model as the input pair. The latter constitutes a new form of (local) explanation which we term as amalogy-based explanation. Although these might seem to be similar to based explanations (Gurumoorthy et al., 2019). which are commonly used to locally explain classification models, there is an important difference: Exemplars are typically close to the inputs they explain, whereas analogies do not have to be. What is desired is for the relationship between members of each analogous pair to be close to the relationship of the input pair (I1,
3 RELATED WORK
A brief survey of local and global explanation methods are available in Appendix [A] We are aware of only few works that explain similarity models (Zheng et al.) 2020 Plummer et al., 2020 Zhu et al. 202T). All of these however are primarily applicable to image data. Moreover, they either require white-box access or are based on differences between saliency maps. Our methods on the other hand are model agnostic and apply to tabular and text data, as showcased in the experiments.
The Joint Search LIME (JSLIME) method, proposed in (Hamilton al. 2021) for model-agnostie explanations of image similarity, has parallels to our feature-based explanations. However, JSLIME is geared toward finding corresponding regions between query and retrieved image, whereas our feature-based method explains the distance predicted by a similarity model by another, simpler distance function and is more natural for tabular data (See Appendix F. for more details).
2

---

Under review as a conference paper at ICLR 2022
There is rich literature similarity/metrie learning methods, see e.g. (Kulis 2013) for survey. However, the goal in these works is to learn a global metric from labeled examples. The labels may take the form of real-valued similarities or distances (regression similarity learning) (Weinberger & Saul 2009); binary similar/disimillar labels (Tariq et al.) 2020), which may come from set membership consist of pairwise "must-link" or "cannot-link" constraints (Zhang et al. (019);; or triplets (I, 2) where is more similar to x than (contrastive learning) (Schultz & Joachims) 2004 Hofter & Ailon 2013). Importantly, the metric does not have be interpretable, and recent deep learning models are certainly not. In our setting, we are given similarity function as a black box and local level. Hence, the two problems are distinct. Mathematically, the feature-based method of Section 4.1 belongs to the regression similarity learning category (Kar & Jain 2012), but the supervision comes from the given black box. Note that our notion of analogies is different from the area of analogy mining (Hope et al.] (017). where representations are learnt from datasets in order to retrieve information with a certain intent.
wish explain its outputs
EXPLANATION METHODS
We propose two methods to explain similarity learners. The first is a feature-based explanation, while the second is new type of explanation termed analogy- -baeed explanation. The two explanations complement each other, while at the same time are also related as the analogy-based explanation can optionally use the output of the feature-based method input pointing synergies between the two.
4.1 CEATURE-BASED SIMILARITY EXPLANATIONS
We assume that the black box model εBB (2, y) is distance function between two points x and y, i.e., smaller (1, y) implies greater similarity. We do not assume that óRB satisfies all four axioms of a metric, although the the proposed local approximation is a metric and may be more suitable if δnn satisfies some of the axioms.
Given the literature on post-hoc explanation of classifiers and regressors, a natural way to obtain a feature-basee explanation of døb (æ, y) is to regard it as a function of a single input - the concatenation of (xr 2). Then LIME (Ribeiro et al. 2016) or other first-order gradient-based methods (Simonyan et al. 2013) can produce local linear approximation of бвB (z. y) at (æ. y) of the form 9 Art gy Ay. This approach cannot create interactions and thus cannot provide explanations in terms of distances between elements of x and y, e.g. (xj - y²)² or [zg - Vj which are necessarily nonlinear.
We thus propose locally approximate dBB (z, y) with quadratic model, the Mahalanobis distance (x, y) = (- 7)² A(ī 7). where A > 0 is a positive semidefinite matrix and T., ÿ are interpretable representations of x, y (see (Ribeiro et al. 2016) and note that = x. y = y if the features in æ. y are already interpretable). This simple, interpretable approximation is itself a distance between æ and y. In Appendix E.2] we discuss the equivalence between explaining distances and similarities. In Section 5.1 we show qualitative examples for how elements of learned A can explain similarities. We learn by minimizing the following loss over a set of perturbations (I,,, y4) in the neighborhood Nay of the input pair (x. y):
Σ (H
Wzam (dgn (x vi) (z. - (.:TTAAC 7.)))
min A
The loss captures the fidelity of the Mahalanobis approximation to the black box. For non -eggaiiee weights Wz..g. problem equation is convex because 1) the quadratic form (5, ) A(i; V/s
linear in A, 2) this is composed with a weighted least squares objective, and 3) the set of semidefinite matrices is convex. At the same time, the semidefinite constraint A > makes equation | different
from LIME. We use CVXPY (Diamond & Boyd 2016 Agrawal et al. 2018) to solve equation [
The generation of perturbations (x,, y:) and their weighting by Wz..,. mostly follow LIME's approach (and share its limitations (Slack et al.) 2020)), with the following two differences: First and most notably, for perturbing categorical features, we use a method based on conditional probability models that generates more realistic perturbations. This is described in Appendix G along with other perturbation details. Second, we compute the weights Wy.yy, as Wz.,. + Wy.,. where Wa,z, (similarly Wy.y; is computed as in LIME by applying an exponential kernel to distance between æ and z;-
3

---

Under review as a conference paper at ICLR 2022
To further ease interpretation of the Mahalanobis explanation, we also consider version in which A is constrained to be diagonal. Here, the quadratic form can be simplified as (3 - 7) A ( 5) = a
where diag(A) and has components 8 = (i, .,}² Further, the constraint A > 0 reduces to a 2 O simplifying problem (T) into least-squares regression with a non-negativity constraint.
4.2 ANALOGY-BASHY SIMILARITY EXPLANATIONS
We now describe a method of providing analogies as local explanations for similarity learners. Given an input example pair (x, y) and black box model, the goal is to identify set of diverse pairs of examples from the dataset that have the same (or similar) relationship to each other as the input pair. The diversity can help weed out less important factors that one might otherwise think are important. For example, let us say two patients have similar disease conditions (input pair) based on which the model predicted them as being similar. The analogous pairs can be other pairs of patients who are also similar to each other in their disease conditions, but are perhaps aocio-economicaly diverse. This will help ascertain that disease conditions are the reason for the similarity and not socio economic factors. In other words, the true latent factors responsible for the black box's prediction can be uncovered using our analogies. This is not to say that our analogies can never be similar to the input pair, but that they will be only if the true relationship is not obscured. For instance, in Example I in section [. the first analogous pair is very similar to the input pair since the words describing the action (viz. playing) are more important than the object of the sentence (viz. harp, keyboard). As such, analogy based explanations can be seen as a more unbiased way of explaining requiring human judgement, than feature attributions where the reasons are directly provided, making the two somewhat complimentary. Nonetheless, as we will see next our analogy based explanations can take into account the feature attributions to the extent desired (see equation 3
Let pairs of examples in a dataset be 2 = (211. z(2) for i E (1,.... N) and an input instance pair be x (z1, x2). Given a black box model бвB and an analogy closeness function G(z,, x) to be defined, the goal is to find kt analogous pairs to x by solving the following for \1, X2 2 0:
argmin Σ (Onn(zz) (x)) + A Σ G(z,,
A2
min (Z. Zz)
(2)
;=15=1
where, 6min (Z, Zj) min [onn ((2 251)) + 8RB ((z/2, 252)), 8pn ((z), 2Y2)) + Snn ((222 2y1))]
The first term in equation2]u ensures that the analogous pair z; chosen has a similar distance between its members 2,1, 2/2 the input pair (21,,22) according to the black box. The last term encourages diversity in the analogous pairs such that the individual instances are different across pairs, although
the similarity/diffrrenee within a pair is close to that of the input. The function dmin (Zi, 21) determines the best matching between two pairs (Zi, 21). For the analogy closeness term G(zi, x), we use
Gizi, x) = D(z,, x) + a (ōi(z)) - 5(()))?
D(zi, x) = 1 (Ø(z)2 (zz)) )) T (0(x2) 6(x1))
(4)
1 (z:2) o6 2.1 (ll))()))) (()))l
32) is the distance predicted by the feature-base based explanation of Section 4.1 The inclusion of this term with weight a > 0 may be helpful if the feature-based explanation faithful and we wish to directly interpret the analogies. The term D(z, x) is the cosine distance between the directions ((222) - Φ(211) and (xx²)) - Φ(x1) in an embedding space. Here ø is an embedding function that can be the identity or chosen independently of the black box, hence preserving the model- -agnosicc nature of the interpretations. The intuition is that these directions capture aspects of the relationships between 2,1, 2,2 and between IT, 2². We will hence equationn
In equation (x) (01 B2)
refer to this as direction similarity. In summary, the terms in equation - together are aimed at producing faithful, intuitive and diverse analogies as explanations.
Let ((Zz....,,22) denote the objective in equation 2
We prove the following in Appendix B
Lemma 4.1. The objective function f in (equation is submodular:
Given that our function is submodular, we can use well-known minimization methods to find a k-sparse solution with approximation guarantees (Svitkina & Fleischer 2011).
4

---

Under review as a conference paper at ICLR 2022
5 EXPERIMENTS
We present first in Section 5.1 examples of explanations obtained with our proposed methods, to illustrate insights that may be derived. Our formal experimental study consists of both human evaluation to investigate the utility of different explanations (Section (.2) as well as quantitative analysis (Section (.3).. The numerical experiments were with embarrassing parallelization 32 core/64 GB RAM Linux machine, 56 cores and 242 GB of RAM were used for larger experiments.
5.1
QUALITATIVE EXAMPLES
We discuss examples of the proposed feature-based explanations with full A matrix (FbFull) and analogy-base explanations (AbE). The examples are taken from the Semantic Textual Similarity (STS) benchmark datase Cer al. 2017) described further below.
STS dataset: The dataset has 8628 sentence pairs, divided into training, validation, and test sets. Each pair has a ground truth semantic similarity score that we convert to a distance. For the black box similarity model бBB y). we use the cosine distance between the embeddings of x and y produced by the universal sentence encode (Cer al. 2018a). possible learn a distance on top of these embeddings, but we find that the Pearson correlation of 0.787 between the cosine distances and true distances is already competitive with the STS benchmarks (Wang et al. 2019). The corresponding mean absolute error is 0.177. In any case, our methods are agnostic to the black box model.
AbE hyperparameters: In all experiments, we set a = 0 to assess the value of AbE independent of feature- bastee explanations. a and ' were selected once per dataset (not tuned per example) by evaluating the average fidelity of the analogies to the input pairs in terms of the black box model's predictions, along with manually inspecting a random subset of analogies to how intuitive they were. With STS, we get A = 0.5, x2 = 0.01. Analogies with other baselines are in Appendix K
Example 1: To start with simple example, we consider the following pair of sentences.
(a) A man playing harp.
(b) A man playing a keyboard. Sas (2. y) = 0.38.
This pair was assigned a distance of 0.38 by the black box (BB) similarity model. FbFull approximates the above distance by the Mahalanobis distance (æt - T)7A(a - 7). For STS, the interpretable representation E is binary vector with each component Σ, indicating whether word is present in the sentence. We define C' to be the distance contribution matrix C' whose elements C5k == (z, A5) (The- ūK) sum up the Mahalanobis distance. The distance contributions C5k for Example 1 are shown in Figure 2a Since the substitution of "keyboard" for "harp" is the only difference between the sentences, these are the only rows/columns with non-zero entries. diagonal element Cj₃ is the contribution due to one sentence having word j and the other lacking it (e.g. T, 1, y 0). Interestingly, the diagonal elements are partially offset by negative off- off-diagonal elements Cjk. which represent a contribution due to substituting word j (ž,- = 1. J, 0) for word & (3x = 0, Vk 1). We can presume that the offset occurs because harp and keyboard are both musical instruments and thus somewhat similar.
AbE gives the following top three analogies:
1. (a) A guy is playing hackysack. (b) A man is playing a key-board. (1, y) = 0.40.
2. (a) Women are running, (b) Two women are running. dua (I, y) = 0.19.
3. (a) There's rule that decides which players be picked for bowling/batting in the Super Over.
(b) Yes team can use the same player for both bowling and batting in a super over. des (z, y) 0.59.
The first analogy is very similar except that hackysack is a sport rather than a musical instrument. The sentences in the second pair are more similar than the input pair reflected in the corresponding BB distance. The third analogy is less related (both sentences are about cricket player selection) with a larger BB distance.
Example 2: Next we consider the pair of longer sentences from Figure I this pair is 0.19 they are closer than in Example 1. The two sentences are mostly the same but the first one adds context about an additional dolphin scheme.
The BB distance between
5

---

Under review as a conference paper at ICLR 2022
00015
go
0.0300
1412
9469
0.2
ongage
0.08
SCreme
0.1000
dodboa
0.0075
the
payeng
man
0.2
0.025
Went
100
0.0000
200
Na
(a) Example
(b) Example 2
Example
Figure 2: Contributions to distance based on feature-based explanation with full matrix (FbFull).
In addition to the analogy shown in Figure I
the other two top analogies from AbE are:
1. (a) The American Anglican Council, which represents Episcopalian conservatives, said it will seek authorization to create separate provinee in North America because of last week's actions.
(b) The American Anglican Council, which represents Episcopalian conservatives, said will seek authorization to create a separate group. (z, y) 0.18.
2. (a) A Stage episode is declared when ozone levels reach 0.20 parts per million.
(b) The federal standard for ozone is 0.12 parts million. Sas (z, y) 0.44.
The analogy in Figure and the first analogy above good matches because like the input pair, each analogous pair makes the same statement but one of the sentences gives more context (a group North America and because of last week's actions, Singapore being small city-state). The second analogy is more distant (about two different ozone thresholds) but its BB distance is also higher.
The distance contribution matrix given by FbFull plotted in Figure 2b For clarity, only rows/columns with absolute sum greater than 0.01 are shown. Several words with the largest contributions come from the additional phrase about the dolphin scheme. The substitution of the verb "set up" for "engage" is also highlighted.
Example The third pair both more complex than Example and less similar than Example 2:
(a) It depends on what you want to do next, and where you want to do it.
(b) I guess it depends on what you're 'e going to do. dan(I, y) = 0.44.
Figure shows the distance contribution matrix produced by FbFull, again restricted to significant rows/columns. The most important contributions identified are the substitution of "[a]re going" for "want" and the addition of "I guess" in sentence b). Of minor importance but interesting note is that the word "next" in sentence a) would have larger contribution but it is offset by negative contributions from the ("next", " "going") and ("next", "guess") entries. Both "next" and "going" are indicative of future action. Below is the top analogy for Example 3:
prefer the second half 1-2 minutes faster then the first. (b) would definitely go slightly slower first half. ôes (z, w) = 0.45.
Both sentences express the same idea (second half faster than first half) but in different ways, similar the input pair. Two more analogies are discussed Appendix K
5.2
USER STUDY
We designed and conducted human based evaluation investigate five local explainability methods.
Besides the proposed FbFull and AbE methods, the three other approaches evaluated in the user study are: feature-based explanation with diagonal (FbDiag): ProtoDash (PDash) Gurrmoorthy et al. 2019). state-of-the-art exemplar explanation method and Direction Similarity (DirSim). which finds analogies like AbE but using only the direction similarity term D(z,, x) in equation Given the novelty of our setting, we thought these to be reasonable competitors.
Methods:
Setup: For each pair of sentences in the STS dataset, users were instructed to use the provided explanations to estimate the similarity of the pair per black box similarity model. As mentioned in
We created analogies by selecting prototypes for each instance and then pairing them in order.

---

Under review as a conference paper at ICLR 2022
Pergentape
Percentape
Figure 3: % accuracies from our user study (higher values are better). The left, center and right figures are overall results, results for short sentences and results for long sentences respectively. The error bars are one standard error. We observe that overall our proposed AbE and FbbFull methods are (statistically) significantly better than other approaches, P-values based on paired t-tests for our approaches being (statistically) equal to the second best approaches confirm this and are as follows: AbE- is 1.66 x 10-9 FbFull-PDass is 2.92 x 10 10-5 AbE-FbDiag is 3.28 x 10 and FbFull-FbDiag is 3.36 x 10-5, Looking at specific cases we see that for short sentences DirSim was the most competitive (center figure), while for long sentences FbDiag was (right figure). However, our methods (AbE and FbFull) remain consistently better in both of these scenarios.
the black box model produces cosine distances in [0, based a universal sentence encoder (Cer et al., 2018b). To be more consumable to humans, the outputs of the black box model were discretized into three categories: Similar (0 0.25 distance), Somewhat similar (0.25 0.75 distance) and Dissimilar (> 0.75 distance). Users were asked to predict one of these categories or "can't say" if they were unable do so. Screenshots illustrating this are in Appendix M and the Predicting black box outputs is standard procedure to
Section 5.1
full user study is attached in Appendix Q
measure efficacy of explanations (Ramamurthy et al. 2020 Ribeiro al. 2016 Lipton 2016).
In the survey, 10 pairs of sentences were selected randomly in stratified fashion from the test set such that four were similar, four were somewhat similar, and the remaining two were dissimilar as per the black box. This was done to be consistent with the distribution of sentence pairs in the dataset with respect to these categories. Also, half the pairs selected were short sentence pairs where the number of words in each sentence was typically < 10, while for the remaining pairs (i.e. long sentence pairs) the numbers of words were typically closer to 20. This was done to test the explanation methods for different levels of complexity in the input, thus making our conclusions more robust.
The users were blinded to which explanation method produced a particular explanation. The survey had 30 questions where each question corresponded to explanation for sentence pair. Given that there were 10 sentence pairs, we randomly chose three methods per pair, which mapped to three different questions. For feature-based explanations, the output from the explanation model was provided along with a set of important words, corresponding to rows in the A matrix with the largest sums in absolute value. For analogy-based explanations, black box outputs were provided for the analogies only (not for the input sentence pair), selected from the STS dev set. We did this to allow the users calibrate the black box relative to the explanations, and without which it would be impossible to estimate the similarity of the sentence pair in question. More importantly though, all this information would be available to the user a real scenario where they are given explanations.
We leveraged Google Forms for our study. 41 participants took it with most of them having back- grounds in data science, engineering and business analytics. We did this as recent work shows that most consumers of such explanations have these backgrounds (Bhatt et al. 2020).
Observations: In Figure 3 we see a summary of the results from our user study. In the left figure (all sentences), we observe that AbE and FbFull significantly outperform both exemplar-based and feature-based baselines. AbE seems to be slightly better than FbFull; however, the difference is not statistically significant. While the results in Section 5.3 show that both of these methods have high fidelity, this was not known to the participants, who instead had to use the provided reasons (analogies important words) decide whether accept the outputs of the explanation methods. The good performance of AbE and FbFull suggests that the provided reasons are sensible to users.
In the center figure (short sentences), DirSim is the closest competitor, which suggests that the black box model is outputting distances that accord with intuition. FbDiag does worst here, signaling the importance of looking at interactions between words. However in the right figure (long sentences),
7

---

Under review as a conference paper at ICLR 2022
0.30
ALE
AGE
0.23
20.00
Didii
POwch
Hean abenliiee eror
Drdim POwch
0.175
10.59
0.25
0.30
+ Azt
Disam
10:075
0.303
8.65
1Dush
8
8
15
15
Number analegics
Number analogies
Number of unatogics
(a) Iris
(b) MEPS
(c) STS
Figure 4: Mean absolute errors (infidelity) of the analogy explanation methods' predictions with respect to the black box predictions for varying number of analogies. The solid lines are the mean over 5 CV folds, and the shaded areas show standard error of the mean. Lower values are better.
FbDiag the closest competitor and DirSim is the worst, hinting that predicting the black box similarity becomes harder based on intuition and certain key words are important to focus on independent of everything else.
We also solicited (optional) user feedback (provided in Appendix N). From the comments, it appeared that there were two main groups. One preferred analogies as they felt they gave them more information to make the decision. This is seen from comments such "The examples [analogies] seem to be more reliable than the verbal reason [words]." There was support for having multiple diverse analogies to increase confidence in prediction, seen in "The range of examples may be useful, some questions have all three examples in the same class." " While one would expect this benefit to diminish without diversity in the multiple analogies, this aspect the feature-based explanations were better given their precision and brevity. An example comment here was "I find the explanation with the difference between the sentences easier to reason about." " A couple of people also said that providing both the fcature-based and unalogy-based explanations would useful they somewhat complement each other and can help cross-verify one's assessment.
explicitly tested. The second group felt
5.3 QUANTITATIVE EXPERIMENTS
This section presents evaluations of the fidelity of various explanation methods with respect to the black box similarity model's outputs.
Methods: In addition the five local explanation methods considered in Section 5.2 we evaluate globally interpretable model, global reature-based full-matrix explanations (GFbFull), LIME (Ribeiro al. 2016), and Joint Search LIME (JSLIMB) (Hamilton et al. 2021). GFbFull uses Mahalanobis model like in Section b but fit on the entire dataset instead of a perturbation neighborhood Nzy. To run GFbFull
the STS dataset, we chose only the top 500 words in the test set vocabulary according to tf-idf scores to limit the computational complexity. For all methods, explanations were generated the validation
using the test set and for AbE, DirSim, and PDash,
select the analogies.
Data and Black Box Models: In addition to the STS dataset, we use two other datasets along with attendant black box models as described below: UCI Iris (Dheeru Karra Taniskidou, 2017)) and Medical Expenditure Panel Survey (MEPS) (EP). The supplement has more details on datasets, black box models, and neighborhoods for maature-based explanations.
For Iris and MEPS, 5-fold cross-validation was performed. For Iris, pairs of examples were exhaus- tively enumerated and labeled as similar or dissimilar based on the species labels. A Siamese network was trained on these labeled pairs the black box model бBB achieving mean absolute error (MAF) of 0.400 0.044 with respect to the ground truth distances and Pearson's 's of 0.370 0.164.
For MEPS, we found that tree- free-based models worked better for this largely categorical dataset. Accordingly, we first trained a Random Forest regressor to predict healthcare utilization, achieving a test R²2 value of 0.381 + 0.017. The black box function døp (z. y) was then obtained as the distance between the leaf embeddings (Zhu et al.) 2016) x, from the random forest. Note therefore that dgs (z, y) is distance function of two inputs, not regression function of one input. Pairs of examples to be explained were generated by stratified random sampling based on ópn (...) values. For feature-based explanations we chose 10000 pairs each from the validation and test set of each fold.
8

---

Under review as a conference paper at ICLR 2022
Table 1: Generalized infidelity (mean absolute error) of the outputs produced by the feature- explanation methods to the black box models. We show mean + standard error of the mean (SEM) for Iris and MEPS where 5-fold CV was performed. See Appendix for more quantitative results. Lower values are better.
Measure
Dataset
FbFull
FbDiag
LIME
JSLIME
Iris
0.676
0.090
0.922
0.116
1.093 + 0.108
1.208 + 0.146 0.150 = 0.002
Generalized Infidelity
MEPS
0.178 = 0.005
0.140
0.002
0.192 + 0.002
STS
0.245
0.257
0.462
0.321
For AbE, DirSim, and PDash, we chose 1000 pairs to limit the computational complexity. For AbE, we used A 1.0 and A2 = 0.01 for both MEPS and Iris.
For feature-based explanations, we present comparisons of generalized infidelity (Ramamurthy et al. 2020). This tests generalization by computing the MAE between the black-box distance for an input instance pair and the explanation of the closest neighboring test instance pair. T shows the generalized infidelity for FbFull, FbDiag. LIME, and JSLIME with respect to the black box predictions. Since GFbFull computes global explanations, we cannot obtain this measure. Generalized fidelity computed using Pearson's 's r and non-generalized MAE/Pearson's r for all methods including GFbFull are presented in Appendix Appendix also presents more descriptions of metrics used.
FbFull/FbDiag have superior performance. This suggests that they provide better generalization in the neighborhood by virtue of Mahalanobis distance being metric. We do not expect LIME to perform well as discussed in Section [4.] but JSLIME also has poor performance with this metric since it likely overfits because of the lack of constraints on A.
From Table I
Since all the black box predictions are between 0 and 1, it is possible to compare these three datasets. The methods seem to perform best with MEPS, followed by STS and Iris. The MEPS dataset, even though the largest of the three has two advantages. The variables are binary (dummy coded from categorical) which possibly leads better fitting explanations, and the search space for computing the generalized metric is large, which means that the likelihood of finding a neighboring test instance pair with a good explanation is high. For STS, the black box universal sentence encoder seems to agree with the local Mahalanobis distance approximation and to some extent even with the diagonal approximation. Iris has the worst performance possibly because the dataset is so small that Siamese neural network cannot approximate the underlying similarity function well, and also because the search space for computing the generalized metric is quite small.
The infidelity (MAE) of the analogy explanation methods (AbE, DirSim, and PDash) is illustrated in Figure 4 Given a set analogies Zı...., Zk, the prediction of the explainer is computed as the average of the black box predictions dgp (z1).... ........gs. dEg (2x) for the analogies. The proposed AbE method dominates the other two baselines because of the explicit inclusion of the black box fidelity term in the objective. For Iris and STS, the MAE of AbE steadily increases with the number of analogies. This is expected because of the trade-off between the fidelity term and the diversity term in (equation 2) as k increases. For MEPS, the MAE of AbE very slowly reduces and flattens out. This could be due to the greater availability of high-fidelity analogous pairs in MEPS.
DISCUSSION
In summary, we have provided (model agnostic) local explanations for similarity learners, in both the more familiar form of feature attributions as well as the more novel form of analogies. The experimental results indicate that the resulting explanations have high fidelity, appear useful to humans judging the black box's 's behavior, and offer qualitative insights.
For the analogy-bard based method, the selection of analogies is significantly influenced by the analogy closeness term G(zi, x) in (equation 2) Herein we have used direction similarity equation 4 which is convenient to compute given embedding and appears to capture word and phrasing relations well in the STS dataset. It would be interesting to devise more sophisticated analogy closeness functions, tailored to the notion of analogy in a given context. is also of interest extend this work from explaining pairwise relationships to tasks such as ranking. We thus hope that the approaches developed here could become meta-approaches for handling multiple types of relationships.
9

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
Explanation methods can be hugely beneficial to stakeholders and can provide real power to end users to fight unfair automated decisions. However, they are also prone to risks. The risks of using post-hoc explanations for similarity learning are similar to those of using post-hoe explanations for Dlusiffiation/regression. Firstly, explanations may not be completely faithful to the black box and hence misleading. Secondly. there are be many conflicting explanations that can have high fidelity with respect to the black box predictions. This is why we think placing reasonable constraints (such forcing the explainer to approximate distance metric) is essential. Thirdly, explanations may pose privacy/security risks and reveal the inner workings of the black box. With careful understanding of these issues, many of these risks can be controlled and mitigated.
REPRODUCIBILITY DISCUSSION
Experimental details and results are provided in Section of the main paper. All datasets are public and their details are provided in Appendix Hyperparameters used are discussed in Appendix G The metrics used in evaluations are described in The complete user study questions are provided in Section Q
Code will be provided during the discussion phase through an anonymized link.
REFERENCES
Medical Expenditure Panel Survey (MEPS). https: hwwp//wwwww... http:///wwwwbg//o////t/////eeeee html Content last reviewed August 2018. Agency for Healthcare Research and Quality, Rockville, MD.
Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42-60, 2018.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, (0(7):e0130140, 2015.
Osbert Bastani, Carolyn Kim, and Hamsa Bastani. Interpreting blackbox models via model extraction. 2017.
arXiv preprint arXiv:
Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Yunhan Jia Ankur Taly, Joydeep Ghosh, Ruchir Puri. José M. F. Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountabiliry, and Transparency, 2020.
Cristian Buciluă, Rich Caruana, and Alexandru Niculescu-Mizil Model compression. In Proceedings of the 12th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, 2006.
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 2/h ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD "15, pp. 1721-1730 New York, NY, USA, 2015. ACM. ISBN 978- 4503 3664-2, doi: 10.1145/2783258.278888
URL http::///oo acm, http:/aa...aaa.................. 110.011111222111 /1t.111///72228888
Daniel Cer, Mona Diab, Encko Agirre, Inigo Lopez-Gazpia and Lucia Specia. Semeval-2011 task I: Semantic textual arXiv: 2017.
and cross-lingual focused evaluation. arXiv preprint
Daniel Cer. Yinfei Yang. Sheng-yi Kong, Nan Hua. Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedeer Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. Universal sentence encoder for English. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 169-174, Brussels, Belgium, November 2018a. Association for Computational Linguistics. doi: 10.18653/v 1/D/1/ 2029. URL https: www. c web. orrbborgg/anee www.eorrrggaooooooooo http//nttoooll//////////
10

---

Under review as a conference paper at ICLR 2022
Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Suajardo-Cespede Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. Universal sentence encoder. arXiv preprint arXiv: 2018b.
Sanjeeb Dash, Oktay Günlük, and Dennis Wei. Boolean decision rules via column generation. Advances in Neural Information Processing Systems, 2018.
Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository, 2017. URL http:
7/archive. wcs.ui.e.duuuii.euuu.uu
Amit Dhurandhar, Pin-Yo Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems 31. 201 8a.
Amit Dhurandhar, Karthikeyan Shanmugam, Ronny Luss, and Peder Olsen. Improving simple models with confidence profiles. Advances of Neural Inf. Processing Systems (NeurIPS), 2018b.
Amit Dhurandhar, Karthikeyan Shanmugam, and Ronny Luss. Enhancing simple models by exploiting what they already know. Intl. Conference on Machine Learning (ICML), 2020.
Steven Diamond and Stephen Boyd. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17(83):1-5, 2016.
R. Guidotti, A. Monreale, S. Matwin, and D. Pedreschi. Black box explanation by learning image exemplars in the latent feature space. In In Joint European Conference Machine Learning and Knowledge Discovery in Databases, 2019.
Karthik Gurumoorthy, Amit Dhurandhar, Guillermo Cecchi, and Charu Aggarwal. Protodash: Fast interpretable prototype selection. IFEE ICDM, 2019.
Mark Hamilton, Scott Lundberg, Lei Zhang, Stephanie Fu, and William Freeman. Model-agnostic explainability for visual search. arXiv preprint arXiv:2103.003: 2021.
Lisa Anne Hendricks, Zeynep Akata, Mareus Rohrbach, Jeff Donahue, Bernt Schiele, and Trevor Darrell. Generating visual explanations. In European Conference Computer Vision, 2016.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. In https://araiv.orgllas//15...... 02531, 2015.
Elad Hoffer and Nir Ailon. Deep metric learning using triplet network. In Aasa Feragen, Marcello Pelillo, and Marco Loog (eds.), Similarity- Based Pattern Recognition, pp. 84-92, Cham, 2015. Springer International Publishing. ISBN 978-3-319-24261-3.
Tom Hope, Joel Chan, Aniket Kittur, and Dafna Shahaf. Accelerating innovation through analogy mining. In Proceedings of Knowledge Discovery and Data Mining, 2017.
E. Hullermeier. Towards analogy-based explanations in machine learning. arXiv:2005.12800 2020.
Tsuyoshi Idé and Amit Dhurandhar. Supervised item response models for informative prediction. Knowl. Inf. Syst., 51()):235-55)) 257. April 2017.
Purushottam Kar and Prateek Jain. In F. Pereira, C. C. Burges, L. Bottou, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25, pp. 215-223, 2012.
Been Kim, Rajiv Khanna, and Oluwasanmi Koyejo. Examples are not enough, learn to criticize! criticism for interpretability. In Advances of Neural Inf. Proc. Systems, 2016.
Andreas Krause. Sfo: A toolbox for submodular function optimization. Journal of Machine Learning Research, 11 11:1141-1144 2010.
B. Kulis. Metric learning: A survey. Foundations and Trends@ in Machine Learning, 2013.
O. Lampridis, R. Guidotti, and S. Ruggieri. Explaining sentiment classification with synthetic exemplars and counter-exemplass In In International Conference Discovery Science, 2020.

---

Under review as a conference paper at ICLR 2022
Zachary C Lipton. The mythos of model interpretabillity. arXiv preprint arXiv:1606.03490) 2016.
Scott Lundberg and Su-In Lee. Unified framework for interpretable methods. In In Advances of Neural Inf. Proc. Systems, 2017.
Ramaravind Kommiya Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. arXiv preprint arXiv: aaee::/005009990 2019
Bryan A. Plummer, Mariya I. Vasileva, Vitali Petsiuk, Kate Saenko, and David Forsyth. Why do these match? explaining the behavior of image similarity models. In ECCV, 2020.
Karthikeyan Ramamurthy, Bhanu Vinzamuri, Yunfeng Zhang, and Amit Dhurandhar. Model agnostic multilevel explanations. In In Advances in Neural Information Processing Systems, 2020.
Marco Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?" explaining the predictions of any classifier. ACM SIGKDD Intl. Conference on Knowledge Discovery and Data Mining, 2016.
Cynthia Rudin. Please stop explaining black box models for high stakes decisions. NIPS Workshop on Critiquing and Correcting Trends in Machine Learning. 2018.
Matthew Schultz and Thorsten Joachims. Learning a distance metric from relative comparisons. In S. Thrun, L. Saul, and Schölkopf (eds.), Advances Neural Information Processing Systems, volume 16, pp. 41 48. MIT Press, 2004.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient- based local- ization. nternational Journal of Computer Vision, 128:336-38 359, February 2020.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR, abs/ abss/1311.6034 2013. URL
1332 6034
M. Sipser. Introduction to the Theory of Computation 3rd. Cengage Learning, 2013.
Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. In AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (ATES), 2020. URL
1911 0w500.50.... .ddf.
Guolong Su, Dennis Wei, Kush Varshney, and Dmitry Malioutov. Interpretable two-level boolean rule learning for classification. In https://arziivorr/abb///0666 600.05998 2016.
Zoya Svitkina and Lisa Karen Fleischer. Submodular approximation: Sampling-based algorithms and lower bounds. SIAM Journal on Computing, 2011.
Zaid Bin Tariq, Arun Iyengar, Lara Marcuse, Hui Su, and Bülent Yener. Patient-specific seizure pre- diction using single seizure electroenceppallggrahy recording. arXiv preprint urXiv:2011.0898) 2020.
Isee van der I. Linden, Hinda Haned, and Evangelos Kanoulas. Global aggregations of local explanations for black box models. In Fairness, Accountability, Confidentiality, Transparency, and Safety- SIGIR Workshop, 2019.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding, In International Conference on Learning Representations, 2019. URL
net / Forum?iddrr44mme
Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research, 10(2), 2009.
Hongjing Zhang, Sugato Basu, and Ian Davidson. framework for deep constrained clustering algorithms and advances. In Proceedings European Conference on Machine Learning, 2019.
12

---

Under review as a conference paper at ICLR 2022
Meng Zheng, Srikrishna Karanam, Terrence Chen, Richard J. Radke, and Ziyan Wu. Towards visually explaining similarity models. In NxXIv:200080603333 2020.
Sijie Zhu, Taojiannan Yang, and Chen Chen. Visual explanation for deep metric learning. IEEE Transactions on image Processing, 2021.
Z. Zhu, C. Yin, B. Qian, Y. Cheng, J. Wei, and F. Wang. Measuring patient similarities via a deep architecture with medical concept embedding. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 749-758, 2016. doi: 10. 10011090.2222060000666
APPENDIX
A OTHER EXPLAINABILITY METHODS
large body of work on XAI can be said to belong to either local explanations (Ribeiro et al. 2016
Lundberg & Lee 2017
Gurumoorthy et al. 2019), global explanations (Hinton et a 2015 Bastani
et 2017 Bucilu ua et al. 2006; Dhurandhar et al. 20186 2020), directly interpretable models
2018 Sipser 2013) visualization- based methods Hendricks et al. (016). Among these categories, local explainability methods are the most relevant our current endeavor. Local explanation methods generate explanations per example for a given black box. Methods in this category are either Seature-basee (Ribeiro et al. 2016 Lundberg & Lee Bach Bach al., (2015) Dhurandhar et al. 2018a Mothilal al..[2019) or (huran/Durandhas
(Caruana al. 2015 Rudin 2018 Su et al. 2016 Dash
exemplar-based (Gurumoorthy et al.) [019 Kim et al. (2016). There are also a number of methods in this category specifically designed for images (Simonyan et al.) 2013 Bach et al., 2015 Guidotti al. 2019; Lampridis et al., 2020 (Selvaraju et | (020)). However, all of the above methods are predominantly applicable to the classification setting and a smaller number of cases to regression.
Global explainability methods try to build an interpretable model the entire dataset using informa- tion from the black-box model with the intention of approaching the black-box models performance. Methods in this category either use predictions (soft or hard) of the black-box model to train simpler interpretable models Hinton et al. (2015); Bastani et al. (201);; Buciluu et al. (2006) or extract weights based the prediction confidences reweighting the dataset Dhurandhar al. (2018b) 2020). Directly interpretable methods include some of the traditional models such as decision trees or logistic regression. There has been lot of effort recently to efficiently and accurately learn rule lists Rudin (2018) or two-level boolean rules Su et al. (2016) or decision sets Sipser (2013). There has also been work inspired by other fields such as psychometrics IIdé & Dhurandhar (2017) and healthcare @ealtheareCaarunaa it al.) (2015). Visualization based methods try to visualize the inner neurons or set of neurons in layer of a neural network [endriccks et al. (2016). The idea is that by exposing such representations one may be able to gauge if the neural network is in fact capturing semantically meaningful high level features.
Caruana
B PROOF OF SUBMODULARITY (LEMMA 4.
Proof. Consider two sets S and consisting of elements z (i.e. analogous pairs) as defined before, where S C T. Let w be pair @ T and x be an input pair that we want to explain. Then for any valid and T, we have
J(S U w) S(S)
(Ōnn(w) - {x)) + X)G(( - X2 Σ 2 (w.)).
(5)
EES
Similarly,
f(T U w) f(T)
(ōBB(W)= - (x)( + G/G(w - du Σ
8 (w, z). (6)
ZET
13

---

Under review as a conference paper at ICLR 2022
Subtracting equation equation 6 from equation and ignoring X2 as just scales the difference without changing the sign gives us
min (w
min (w, z)
ZET
RES
min W, 2)
Σ
8u (Wz))
Σ ZES
min (w, 2)
xET/S
min ( (w 2 0
TET/S
Thus, the function f()) has diminishing returns property.
C GREEDY APPROXIMATE ALGORITHM FOR SOLVING EQUATION 2
The only reliable software that was available for solving the submodular minimization was the SFO MATLAB package (Krause 2010). However, we faced the following challenges (a) it was quite slow to run the exact optimization and since we had to compute thousands of local explanations, it would have taken an unreasonably long time, (b) we wanted sparse solutions (not unconstrained outputs) (c) the optimization was quite sensitive in the exact setting to the hyperparameters ) and A2. (c) attempts to speed up the execution by parallelizing it would require algorithmic innovations, (d) MATLAB needed paid licenses.
Hence for the purposes having better control, speed, and efficiency, we implemented a greedy approximate version of the objective in (equation The greedy approach chooses one analogous pair (z1) to minimize the current objective value and keeps repeating it until & pairs are chosen. The greedy algorithm is provided in Algorithm
D COMPUTATIONAL COMPLEXITIES
The FbFull method involves solving an SDP which has at least time complexity of (dd) where is the number of features since each iteration usually involves solving linear system inverting matrix of that size. However, it not apparent how the CVXPY package we use sets up and solves this problem, which could alter this complexity.
We implement a non-negative sparse minimization for FbDiag with k non-zeros and for this case the computational complexity if O(N122 where N is the number of perturbations used.
For the proposed AbE method, the objective function is submodular the k-sparse minimization algorithm has an approximation guarantee of / In and runs in (g:4.55 time. Typically, is small. However, there is no software implementation available for this method to the best of our knowledge. Hence we use the greedy method proposed above which has a time complexity of O(N),, where N the dataset size.
E ADDITIONAL REMARKS ON METHODS
E. JOINT SEARCH LIME
Joint Search LIME (JSLIME) (Hamilton al. 2021 is bilinear model akin to x" Ag in our notation, where A is unconstrained and may not even be square, opposed to FbFull/FbDaag which uses a Mahalanobis distance (T π) A(= - - 7), where FbFull/FbDiag and ISLIME approaches have their merits. Mahalanobis distance is metric and interpretations can exploit this by decomposing the Mahalanobis distance into distance contributions due to differences in individual features. On the other hand, JSLIME can be more flexible because of unconstrained Homilton et al. (2021) show that can be used to identify correspondences between parts of two inputs, specifically region of retrieved image that corresponds query image. This is different task from explaining a predicted distance by a decomposition. Also for tabular data, it is not clear how meaningful the correspondences from ISLIME will be.
semidefinite and necessarily square. Both
given region of
14

---

Under review as a conference paper at ICLR 2022
E.2 EXPLAINING SIMILARITIES VERSUS DISTANCES
The feature based explanation methods (FbFull and FbDiag) explain the distance between the points a and y/ given by 8BB(T, (r. y) using the Mahalanobis distance (i y)² A( 5). To understand how this is equivalent to explaining similarities, consider without loss of generality that the maximum value of BBB (z, y) is for any I and y. Now, the explanation model for the simple similarity function
1 - (z, y) is Ej=1 Σi= (1/d² Cste) where C5k = (2, A - Dk) is the distance contribution discussed in Example (Section 5.1) Clearly, a low distance contribution Cyk results in a high similarity contribution 1/d² Czk and vice versa.
F DATA AND BLACK-BOX MODELS
For the Iris dataset we created 5 folds of the data with 20% non-overlapping test set in each fold, and the rest of the data in each fold is divided into 80% training and 20% validation samples. For each partition, we create similar and dissimilar pairs exhaustively based on the agreement disagreement of labels. This resulted in an average of 4560 training pairs, 276 validation pairs, and 435 testing pairs per fold. The black-box -bx model used in this case is a paired (conjoined) neural network where each candidate network the pair has single dense layer whose parameters are tied to the other candidate and is trained using contrastive loss. The mean absolute error (MAE) between the black box predictions, (BB(...) .). and the ground truth distances between the pairs was 0.400 0.044, the Pearson's 's was 0.370 + 0.164 For GFbFull, we chose only the top 500 words in the test set vocabulary according to tf-idf scores to limit the computational complexity.
The Medical Expenditure Panel Survey (MEPS) dataset is produced by the US Department of Health and Human Services. It is a collection of surveys of families of individuals, medical providers, and employers across the country. We choose Panel 19 of the survey which consists of cohort that started in 2014 and consisted of data collected over 5 rounds of interviews over 2014 - 2015. The outcome variable was a composite utilization feature that quantified the total number of healthcare visits of patient. The features used included demographic fcatures, perceived health status, various diagnosis, limitations, and socioeconomic factors. We filter out records that had a utilization (outcome) of and transsreffored the outcome for modeling. These pre- proppsocssiing steps resulted in dataset with 11136 examples and 32 categorical features. We used fold CV using the same approach as in Iris data. When selecting pairs of examples for explanations, we performed stratified random sampling based on d (-, -). For FbFull, FbDiag, and GFbFull, we chose 10000 pairs each from validation and test set for each fold. For AbE, DirSim, and PDash, chose 1000 pairs to limit the computational complexity.
The regression black-box model used for predicting the utilization outcome was Random Forest with 500 trees and 50 leaf nodes per tree. The function 6 y) for a pair (z, y) was obtained as the distance between the leaf embeddings Zhu et al. (2016) from the random forests. The R² performance measure of the regressor in the test set was 0.381 + 0.017.
The STS benchmark dataset comprises of 8628 sentence pairs of which 5749 correspond to the training partition, 1500 to the validation partition, and 1379 to the test partition. Each pair has a ground truth semantic similarity score between O (no meaning overlap) and 5 (meaning equivalence). This can be re-interpreted into distance measure by subtracting it from 5 and dividing the result by 5. The black box model used here was the universal sentence encoder") (Cer al. 2018a), which creates a 512 dimensional embedding for each sentence. 8BB (x, y) is the cosine distance between the embeddings of and y. The Pearson's 's r performance measure of these black-box predictions with respect to the distance between the sentences is 0.787 and the mean absolute error is 0.177.
G HYPERPARAMETERS
In all datasets, FbFull and GFbFull were computed along with very small & penalty on A ||]] added to the objective function (equation [. For FbDiag, we request a maximum of 4 non-zero coefficients for Iris, 10 non-zero coefficients for MEPS, and 5 non-zero coefficients
15

---

Under review as a conference paper at ICLR 2022
for STS. As discussed in Section 5 set the hyperparameters for analogy- based explanations to reasonable values as well and do not perform extensive hyperparameter search.
Perturbations for Local Explanations (FbDiag, FbFull): The input instances z, y are individually perturbed to get the data points (2,, yi) E NNay (see equation T. To obtain the weights Wx.. we first compute weights Wa,z, and Wyy.y for each generated instance x; and y/ respectively. We use the exponential kernel to compute the weight Wr.a; exp(-F((x, /x)/o²) as function of some distance F between the generated instance and the corresponding input instance. F could be бBB- The final weight Wz..0, for the generated pair is then given by summing the individually computed weights of each generated data point with its respective input instance i.c. Wz,,y. = Wa,a, + Wy.y
For Iris, the perturbation neighborhood Nay was generated for each example in the pair by sampling from a Gaussian distribution centered at that example. The statistics for the Gaussian distribution are learned from the training set. For MEPS data, perturbations for the categorical features were learned using the model discussed in the next paragraph, with a bias value of 0.1. For STS, the perturbations were generated following the LIME codebase by randomly removing words from sentences. The sizes of the perturbation neighborhoods used were 100 for Iris, 200 for MEPS, and 10 for STS. The interpretable representation (z, 77) the same as the original features Iris; for MEPS it involves dummy coding the categorical features, and with STS, we create vectorized binary representation indicating just the presence or absence of words in the pair of sentences considered. When computing perturbation neighborhood weights, F is the Manhattan distance for Iris and MEPS, whereas it is the Cosine distance for STS data, o² in exponential kernel was set to 0.5625 times the number of features for all datasets, following the default setting in LIME's code.
For categor- ical features, we develop perturbation scheme that can generate more realistic perturbations. For each example, we estimate the conditional probability of a feature j belonging to different categories given all the other feature values. These conditional probabilities can be used to sample categories for feature j to generate perturbations. To ensure closeness to the original category, a small constant (bias) is added the conditional probability of the original category and re-normalized, similar to an additive smoothing scheme. This can be repeated for all categorical features to obtain perturbed examples. In our experiments, the conditional probability estimator is a logistic regression model that predicts the categories of a feature j using the rest of the features in the dataset.
Realistic Categorical Feature Perturbation using Conditional Probability Models:
H DESCRIPTIONS OF METRICS
Let us denote pairs of examples in a dataset D as X1 = (x;1 for E [l,...,N,, the black box model prediction as ôBB (x). the prediction of the interpretable model at x computed using the explanations obtained at z as og x). Let (Dlcar denote the cardinality of the set D. Let K₂x indicate the neighbors of the pair x from D. We compute both infidelity and fidelity metrics respectively based on mean absolute eтror (MAE) and Pearson's 's r. Lower values indicate higher performance for infidelity metrics, whereas higher values indicate higher performance for fidelity metrics.
Infielity: This is the most commonly used metric to validate the faithfulness of explanation models (Ribeiro et al., (016). Here we define it as the MAE between the black-box and explanation model predictions across all the test points. For MAE, this can be denoted as p ExED |y (x) 6g (x)]. Fidelity can also be computed using Pearson's 's in similar manner. We differentiate the metrics discussed here with generalized (in)fidelity discussed next by just calling it (in)fidelity or sometimes oongeenerll (in)fideliit.
Generalized Infidelity: This metric has also been used in previous works (Ramamurthy et al. 2020) to measure the generalizability of local explanations to neighboring test points. For MAE this is defined as [Dl ExED Tk EzEN, (x) (x)]. Generalized fidelity using Pearson's 's can be computed in similar way. In our experiments we set Kyx as the 1-nearest neighbor of the pair x, computed based on the same weighting scheme described in Appendix
16

---

Under review as a conference paper at ICLR 2022
Table 2: Infidelity and generalized infidelity (mean absolute error) of the outputs produced by the feature-based explanation methods to the black box models. We show mean + standard error of the mean for Iris and MEPS where 5-fold CV was performed. Lower values are better.
Measure
Dataset
FbFull
FbDiag
GFbFull
LIME
JSLIME
Iris
0.676 + 0.090
0.922 + 0.116
NA 1.093 + 0.108
1.208 + 0.146
Generalized Infidelity
MEPS
0.178 + 0.005
0.140
0.002
NA
0.192 0.002
0.150 + 0.002
STS
0.245
0.257
NA
0.462
0.321
Iris
0.100 = 0.002
0.187 = 0.002
0.125
0.003
0.122
0.002
0.026 + 0.000
Infidelity
MEPS
0.027 = 0.000
0.102 + 0.003
0.084 + 0.000
0.164 + 0.004 0.016 + 0.001
STS
0.001
0.077
0.015
0.169
0.000
Table 3: Fidelity and generalized fidelity (Pearson's r) of the outputs produced by the feature-based explanation methods the black box models. We show mean standard error of the mean for Iris and MEPS where 5-fold CV was performed. Higher values are better.
Measure
Dataset
FbFull
FbDiag
GFbFull
LIME
JSLIME
Iris
0.344 + 0.084
0.382 = 0.078
NA
0.018
0.070
0.098 = 0.066
Generalized Fidelity
MEPS
0.716 0.008
0.743 + 0.011
NA
0.461 + 0.039
0.738 = 0.006
STS
0.340
0.333
NA
0.042
0.063
Iris
0.920 + 0.006
0.846 + 0.009
0..988
0.002
0.890
0.007
0.979 + 0.001
Fidelity
MEPS
0.987 + 0.000
0.919 + 0.003
0.927
0.001
0.848 + 0.005
0.996 + 0.001
STS
0.999
0.884
0.957
0.844
1.000
MORE QUANTITATIVE RESULTS
We present comparisons of infidelity in terms of MAE in Table 2
Discussions on generalized infidelity are available in the main paper (Section 5.3
Regarding infidelity. JSLIME performs the best in all datasets, but FbFull follows closely (except for Iris data). This suggests that with STS and MEPS data, the black box universal sentence encoder agrees closely with the Mahalanobis distance approximation. In Iris data, the black box model (Siamese neural network) probably cannot approximate the underlying similarity function well using the small number of examples provided, leading to worse performance using methods. However, ISLIME with its unconstrained A (see Appendix [E.]] is able to provide good fit. LIME performs poorly in all cases pointing to the need to move beyond linear approximations when explaining similarity models. A single global approximation, GFbFull, performs reasonably well suffering only for the Iris dataset, where there is not enough examples.
The above story is reversed with generalized infidelity metrics where our feature- based methods outperform JSLIME handsomely. The performance of JILIME is worst in Iris with generalized infidelity suggesting it could be overfitting local explanations with this dataset.
We also present comparisons of fidelity'generalized fidelity in terms of Pearson's r. Table shows the Pearson's 's of the predictions from FbFull, FbDiag, GFbFull, LIME, and JSLIME with respect the black box predictions. The narrative here is similar to that of infideliiyygeeerrlizeed infidelity, except for some small differences such higher performance of FbDiag with Iris for generalized fidelity.
The Pearson's 's r (fidelity) for the analogy explanation methods (AbE, DirSim, and PDash) and shown in Figure 5 Just like the case of MAE (infidelity) presented in Figure 4 the proposed AbE method dominates the others. With this metric as well, we see that for MEPS data, the performance improved slightly with the number of analogies.
J RUNTIMES
We show the runtimes of the various explanation methods in Tables 4 and 5 The compute configura- tions used are: A core and 64 GB RAM, B 25 cores and 64 GB RAM, C - 50 cores and 242 GB RAM. The black-box models are assumed to be trained and available, and the individual terms in
17

---

Under review as a conference paper at ICLR 2022
Table 4: Approximate runtimes (in seconds) for the Feature Based Explanation methods. If there are multiple folds, the runtimes are summed over the folds. See text for descriptions of the compute configurations A, B, and C.
Dataset Iris MEPS
FbFull
FbDiag 40"
GFbFull
150"
35
18355
7695
1000
STS
43
35
1210
Table Approximate runtimes (in seconds) for the Analogy Based Explanation methods generate 10 analogous pairs. If there are multiple folds, the runtimes are summed over the folds. See text for descriptions of the compute configurations A, B, C.
Dataset
AbE
DirSim
PDash
Iris
8^
8A
460
MEPS
154
15A
1185
STS
84
8^
1091
(equation 2 are assumed to be pre-computed; for Table runtimes reported involves only choosing the 10 analogies.
K MORE QUALITATIVE EXAMPLES -STS
The top three analogies for Example 3 are follows:
Ia) prefer the second half 1-2 minutes faster then the first.
lb) would definitely go slightly slower first half. BB distance:
2a) The pound also made progress against the dollar, reached fresh three-ycar highs at $1.6789.
2b)
The British pound flexed its muscle against the dollar, last pereent at $1.6672. BB distance: 0.55
3a) "I started crying and yelling him, 'What do you mean, what you saying, why did you lie to me?"
3b) Gulping for air, started crying and yelling at him, 'What do you mean? distance: 0.31
The first analogy is most appropriate since both sentences express the same idea (second half faster than first half) but in different ways, similar to the input pair. The second and third analogies are less appropriate because the sentences in each pair are more similar to each other than the sentences in the input pair are to each other (the BB distance for analogy 2 seems high and is likely due to not understanding the idiom "flexed its muscle").
Here are top analogies for the competitors for the same three examples.
Example 1:
ProtoDash a) woman is playing a flute, b) man playing a keyboard
1.0
0.9
Parroas
0.6
0.7
ABC
0.4
Abc
Didim POwsh
Otr
7Dass
15
Number analoglns
Number
anatogies
Number of inatogies
(a)
(b) MEPS
(c) STS
Figure 5: Pearson's is of the analogy explanation methods' predictions with respect to the black box predictions for varying number of analogies. The solid lines are the mean over 5 CV folds, and the shaded areas show I standard error of the mean. Higher values are better.
18

---

Under review as a conference paper at ICLR 2022
INSCOV-3.0
0.12
INSCOV-1
POWCAT-5.0 0
POYCAT-4.0 0
ADSMOK=2.0
ADSMOK
0.06
JIPAIN-2.0 0
0.05
0.04
JIPAIN11.0 0
0.02
MNHITHHl1
-0.05
RACF 0
-0.1
RACE
POVCAT-1.0
02 AALLRAA
OTT-NIVeLL
-3.04
PoVCATw1.
POVCAT=33.
INSCOV=1
InScOVw2.0 0.0
(a) Example (BB dist. 0.022)
(b) Example (BB dist. 0.584)
Figure 6: Contributions to distance according to the feature- explanation with full A matrix
(FbFull) with MEPS data.
DirSim a) Women are running, b) Two women are running
Example 2:
ProtoDash a) The American Anglican Council, which represents Episcopalian conservatives, said it will seek authorization to create separate group. b) The American Anglican Council, which represents Episcopalian conservatives, said it will seek authorization to create a separate province in North America because of last week's actions.
DirSim a) A Stage episode is declared when ozone levels reach 0.20 parts per million. b) The federal standard for ozone is 0.12 parts per million.
Example 3:
ProtoDash a) As I wrote above, it's hard to rate this wall.b) Unlike others, think the route is pretty well described.
DirSim a) Remember, from the Fleet's point of view, the rest of the galaxy is what's moving and experiencing time dilation, b) Well, it really depends on how long he was there, and the exact speed of the Fleet.
L QUALITATIVE EXAMPLES TABULAR MEPS DATA
We provide additional qualitative examples for the MEPS dataset for feature-based and Analogy- based explanations. Please see MEPS feature encodings in Section L. for the key feature encodings used in experiments here.
We consider two pairs of test examples: the first pair that has a BB distance of 0.022 (very similar) and the second pair has BB distance of 0.584 (moderately similar). Considering the first pair, the differences between the two examples are only along the dimensions of insurance coverage (INSCOV) and poverty category (POVCAT). The FbFull approach produces a prediction 0.078, and FbDiag's prediction was 0.255. Clearly, FbFull is able to mimic the black-box model much better locally. The contributions to distance according to FbFull is given in Figure [6a We see that INSCOV and POVCAT are picked up as significant features for explanation. Comparisons of contributions for FbFull and FbDiag are given in Figure 7 For FbFull, the contributions in Figure 7a is obtained by summing the rows columns of the matrix in Figure 6al FbDiag misses the mark giving much importance to INSCOV and hence overpredicting the black- box distance, whereas FbFull assigns reasonable importances to both INSCOV and POVCAT.
19

---

Under review as a conference paper at ICLR 2022
x1
x2
distance contrib.
x2
distance contrib.
INSCOV=1.0
1.0
0.0
0.062192
INSCOV-2.0
0.0 1.0
0.200938
POVCAT=1.0
0.0 1.0
0.028023
POVCAT=1.0
0.0 1.0
0.036757
POVCAT=3.0
1.0 0.0
0.003545
INSCOV=1.0
1.0 0.0
0.017232
(a) FbFull
(b) FhDiag
Figure 7: Contributions to distance according the feature-basee explanation with MEPS data for example (BB dist. 0.022). x1 and x2 are the two points in the example pair shown and value of 1 means the feature is active, and means inactive.
We looked 3 analogies each for AbE, DirSim, and PDash for this example, and found that the mean predicions were 0.046, 0.042, and 0.383 respectively. For this simple example, both AbE: and DirSim competitive performance. However, we found that AbE chose pairs with more diverse set of features compared to DirSim. PDash chose one analogous pair with very low BB distance and two with very high BB distances which is desirable behavior.
We perform similar analyses using the second pair with a BB distance of 0.584. FbFull predicts distance of 0.584, whereas FbDiag predicts a distance of 0.661. Once again see that FbFull more equitably perceives the contributions of the different important variables such as RACE, INSCOV, JTPAIN, and ADSMOK. FbDiag places a lot of weight on INSCOV which is probably not the overwhelmingly important contributor for the dissimilarity given differences in race and other health variables/demooraahhies between the two data points the pair.
For this example as well, we looked at 3 analogies each for AbE, DirSim, and PDash. The distance predictions from these methods are 0.623, 0.407, and 0.776 respectively. AbE close to performance of black-box whereas the other two methods over- under-predice the distance between the data points in the pair. In addition to the high performance AbE also produces diverse analogies whose predictions all individually close to the black-box.
L. SELECTED MEPS FEATURE ENCODINGS
INSCOV=1: Any private insurance
INSCOV=2: Public only insurance
INSCOV=3: Uninsured
POVCAT=1: Poor/negative family income
POVCAT=3: Low family income
POVCAT=4: Middle family income
POVCAT=5: High family income
ADSMOK=-1: Inapplicable smoking status
ADSMOK=1: Current smoker
ADSMOK=2: Not current smoker
RACE=1: White
RACE=0: Non-W White
MNHLTH=1: Excellent (perceived mental health status)
MNHLTH=2: Very good (perceived mental health status)
20

---

Under review as a conference paper at ICLR 2022
x1
x2
distance contrib.
RACE=1.0
1.0
0.0
0.184078
INSCOV=3.0
1.0
0.0
0.136582
JTPAIN=1.0
0.0
1.0
0.120557
ADSMOK=1.0
1.0
0.0
0.1 1.115564
x1
x2
distance contrib.
1N
1.0
0.0
0.066556
INSCOV=1.0
0.0
1.0
0.368997
ADSMOK-2.0
0.0
1.0
0.065655
ADSMOK=2.0
0.0
1.0
0.192357
POVCAT=4.0
1.0
0.0
0.063436
RACE=0.0
0.0
1.0
0.037050
RACE=0.0
0.0 1.0
0.048807
POV
0.0
0.031644
POVCAT=5.0
0.0 1.0
0.027253
MNHLTH=2.0
0.0
0.016656
MNHLTH=2.0 0.0 1.0
0.023411
JTPAIN=2.0 1.0 0.0
0.013785
(a) FbFull
(b) FbDiag
Figure 8: Contributions to distance according to the feature-based explanation with MEPS data for example 2 (BB dist. 0.584). xl and x2 are the two points in the example pair shown and value of means the feature is active, and means inactive.
M USER STUDY SCREENSHOTS
We present example screenshots for the user study Figure 9 examples of analogy and feature based explanations presented in the user study. The bottom figure is the instructions page for the user study.
The top and middle figures are
N USER STUDY PARTICIPANT COMMENTS
liked the second .... providing pairs of sentences that analogous the input pair, as opposed to highlighting important words, but guess that could be subjective. think you hit the nail on head with the analogy based explanation ..... just my biased view.
think the keywords make the 3 pairs easier to understand cuz it attacks it bottom up, while 3 pairs is more top down. So maybe best is to provide both and let user decide.
So this what you want. 2) You may want it, but the process given to you is what you have to work within. To me, they look dissimilar but all the explanations point somewhat similar (so that's what I answered). (Q16) A similar case for Q19. In the case of the 3 pairs of sentences, if all of them were marked with one label only. felt that those examples would always lead to the label used as the answer. I find the explanation with the difference between the sentences easier to reason about.
tricky: 1) You just have base your answer what you do know, which is
Identifying the specific words that marked the differences was a lot more helpful. Parsing the examples and trying determine how they relate to the example was pretty tricky with longer examples.
Some of the analogous examples were hard to map to the original samples, i.e. in what way they were analogous.
The second type of explanation is clearer, but is confusing for the exercise since it tells you directly the option you should choose. The first type is better is the point is to have guesses the answer. For the first type, having either a variety of outputs or multiple "somewhat similar" examples helps.
21

---

Under review as a conference paper at ICLR 2022
Please chcose what the model prodict rollowiig ro sennence part based
12 group mon play soocar cn the seact.
2) group bope are playtng soccer
predictions each theos usalinguus loctd estiedd.
unaloposa Damiladd Two drie the show mikking snow
anallgguus
mone
9) mon moiing neer a strenes meve compantin ea the other elde warsnn
medd
80 wa patling looa onto soerw wonds
Similar
newhee similar
Disemmiar
(a) Analogy-basee based explanation example in the user study.
mocel prodic
explanation proviced balows.
1) man plaping harp. 2) plapling hepfrcard
Explanation: The sentoncce appoar somtewhat similac The words "weybcard"" and "harp" are most imper fant cifferencee bellnenn the tee.
Similar
Summow utt simila
Can't
(b) Feature- -aeturee-baedddde explanation example the user study,
AI Explanations for Pairwise Similarity Models
PLE4SE READ DAREFULLY
you survey queations ahould shoose provide recardoc survey,
you
volurteering
aroune minutes complete the survey and you can also leave additional feedback, Bolow are the details of the suivey.
A blackebos (D0) Al model was trained to predict similantt between petrs of sentences, For seample glven sertences boy ana breakffatt and girl ons breakfistt the BB mozel predict bucket predictions model
valuee between il and purposse somewhot
poir serrenocs, your to pueso would predict tanonymized) All explanation mett odt. Each exp coslanation method provides one of two tpes of coclunutiona. The first saplamus lype higgiigg tu the impor tunt worda surter uaa that it shinks led the BS models prediction along with aesesament of what the simileity lovel is liicly to be. The other type oxplanation provides three para sencences that analogaura the
the catopories the provided explunations. You shown nutput difierent
SENTENCES ACCOREING
(c) Instructions in the user study.
Figure 9: In the top and middle examples of analogy and feature based explanations presented in the user study. On the bottom we see the instructions page for the user study.
22

---

Under review as a conference paper at ICLR 2022
For the explanations with keywords, just follow what the explanation said about similarity. The task is for me to predict what the BB model would do, and trust explanations are faithful to the model
just choose what the explanation says. However, the keywords really didn't 't help me understand why the model think the two are similar or dissimilar, the analogous pair did. It was hard for me to judge what the BB model would predict when analogous pairs are all dissimilar, because you can point out a lot of things that are different without telling me what you consider are similar... Hope this is helpful.
Good survey, except the explanations that were of the format: "Explanation: The sentences appear [somewhat similar]" were confusing. There no indication in how the question was phrased to indicate the answer should be different from [somewhat similar).
The examples seem to be more reliable than the verbal reason. The same pair of sentences gets rated as different results due to different examples for the question. Some key words to differentiate the sentence pair as shown in the verbal reason may not fully align with what I would suggest.
Having an example simple, somewhat similar, and dissimilar on each problem would have been useful, as there were problems where sentence would be provided and only one type was shown.
Additionally, the problems with just the: "the Al would classify this as similar because of X, Y" were weird problems include, they could have acted as calibration at the beginning the test. Instead, found myself skipping the problem and just clicking whatever the explanation said it would classify it to label.
understood the exercise better as answered more questions so went back to redo the initial 10. A slider or list of questions to quickly go back will be great. Also. the initial explanation is too wordy. An demo example would have helped. You should group the questions based on the unique number of input pairs. Dunno if all this is possible google form. But you have resources, assume?
prefer to have one example each of - similar, somewhat similar, and similar. Important keywords are helpful too.
It took me awhile understand what is measure one should for distinguishing between similar and somewhat similar. Sometimes found that even if my intuition was to pick similar, the three analogous examples said somewhat similar so in that case went with somewhat similar as my final answer.
also found myself disagreeing with (a) the choice of analogous pairs and (b) the prediction for the analogous pair, was sure if I should follow my intuition for what I thought was (somewhat) similar.
found the explanation with word highlighting to be not very useful as the words highlighted didn't seem to be indicative of the decision sometimes. So I just used the machine's prediction and my prediction to make my final decision.
O UPDATED FIGURE 2
Figure is updated with reordered rows and columns and shown Figure 10
P PSEUDO CODES
We provide pseudo codes for obtaining feature- and analogy- based similarity explanations discussed in Sections 4.1 and 4.2 respectively.
23

---

Under review as a conference paper at ICLR 2022
Moyboadd
en24ge
0.0350
tar2
0,3
0,2
the
wont
ma
6.0055
gg
keybord
0.005 next
-03
0.0000
1012
( Example
(b) Example
2
(c) Example
Figure 10: Contributions to distance based on feature-based explanation with full matrix (FbFull))
The words are ordered by decreasing order of their contributions the distance. Note that contribution of a word is the sum of all columns (rows) corresponding to that row (column). Also in Figure [Toc the first word 're short of the word are. See Example sentence (b) in Section 5. for details.
P.I. FEATURE-BASED SIMILARITY EXPLANATIONS
For feature-based similarity explanations we provide the detailed pseudo code for both FbFull and FbDiag in Algorithm
Algorithm 1: Feature-Based Similarity Explanations.
Inputs:
Input pair to explain (z, y)-
2. Interpretable representation function p(.) that can create the interpretable representations
Ø = p(z), and y = p(y).
3. Black box model 0BB(Z, y).
Method to create the perturbation neighborhood Nay for the input pair.
5. Exponential kernel parameter for computing weights between original and perturbed pair.
Distance function to used in the exponential kernel.
Method:
Step 1: Create the perturbation neighborhood Nay for the input pair (x., y) containing the pairs
(xi, w.).
Step 2: Create black box prediction d (x,, ye) for pairs Nay-
Step 3: Create corresponding interpretable representations (H,, Vs) using the interpretable representation function p(-).
Step 4: Compute weights for each component in the pair Wy,z, = (xsp(-F(sr, =
Wy.a: exp( (F y1)/o²). Total weight for the pair Way-y = Wa.at Wy
Step 5: Using T obtain that can be used to explain the black box function ( at (x,
This corresponds to the FbFull method.
Note: is constrained be diagonal, (I) reduces to a least squares with non non-negatyvyty
constraint discussed in Section 4.1 This corresponds to the FbDiag method.
Output: Matrix (full or diagonal) that can be used to explain the black box function (B.(..))
at (x. v).
24

---

Under review as a conference paper at ICLR 2022
P.2 ANALOGY-BASEE BASED SIMILARITY EXPLANATIONS
For analogy-based similarity explanations we provide the detailed pseudo code for the greedy version discussed in Section C Algorithm
Algorithm Analogy-Baal Based Similarity Explanations.
Inputs:
Input pair explain (2),
2. Pairs of examples in the dataset = (z11, 2,2) for i E {i,..., N to draw analogies from. These pairs constitute the candidate C.
3. Black box model 8BB (z, y).
Embedding function <(.) used to compute first part, D(z,, x), of the analogy closeness term,
( x) (see equation -equuaion 4).
Distance predicted by the feature- based explainer (x(x) used in equation Use Algorithm
to obtain feature-based explanations for x.
Hyperparameters Å (contribution of analogy closeness term), À2 (contribution of diversity
term), and a (contribution of ceature-based explainer) used in equation P
equuaton 4).
Number analogies to be chosen, k.
8. Analogous pairs set S 0.
Method:
do
argmin (δgB(a) - 6x8(x))² + +1,G(a,x) - de Ebes Emn (a, b)
a
U {a}
while
|S| < k;
Output: Set of pairs analogous to x given by the
Q FULL USER STUDY
The printout of the full user study attached from the next page.
25

---

Al Explanations for Pairwise Similarity Models
Al Explanations for Pairwise Similarity Models PLEASE READ CAREFULLY
Thank you much for volunteering to take this survey. No information other than that you choose provide will be recorded in this survey. There are 30 questions and it should take around 15 minutes to complete the survey. At the end can also leave additional feedback. Below are the details of the survey.
A black-box (BB) Al model trained to predict similarity between pairs of sentences. For example, given pair sentences "A boy ate breakfast' and 'A girl ate breakfast', the BB model will predict similarity value between 0 and 1. For the purposes of this survey, we bucket the predictions of the model into three categories similar, somewhat similar and dissimilar.
Given pair of sentences, your task is guess which of the three categories the BB model would predict based the provided explanations. You will be shown the output of different (anonymized) Al explanation methods. Each explanation method provides one of two types of explanations. The first explanation type highlights the important words in the sentences that thinks led the model's prediction along with its assessment what the similarity level likely to be. The other type of explanation provides three pairs of sentences that analogous the given input pair.
In all cases the question is: HOW SIMILAR ARE THE INPUT SENTENCES ACCORDING TO THE BB AI MODEL BASED ON THE PROVIDED EXPLANATION(S)?
Required
Please choose below what the BB model will predict for following input sentence pair based on the explanation provided below.
1) group men play soccer the beach.
2) group boys are playing soccer on beach.
Explanation: following are three pairs sentences analogous to the input pair sentences. The BB predictions for each of these analogous pairs indicated in parenthesis.
Question
1st analogous pair (similar):
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
of 30
a) Two people snowsuits laying in the snow making snow angels.
b) Two children lying the making snow angels.
2nd analogous pair (similar):
a) sad jumping small stream
companion the other side.
b) is jumpiin
meet his companion the other side.
3rd analogous (similar):
a) puts flour piece meat.
b) is putting flour
1.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the BB model will predict for the following input sentence pair based on the explanation provided below.
1) group of men play soccer on the beach.
2) group of boys are playing soccer on the beach.
Explanation: The following are three pairs sentences analogous to input pair sentences. The BB predictions for each of these anslogous pairs is indicated in parenthesis.
Question 2 of 30
1tt analogous pair (similar)
a) woman bungee jumping.
b) girl is bungee jumping.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
2nd analogous pair (dissimilar)
The aiming gun.
b) boy playing toy phone.
3rd analogous pair (somewhat similar)
a) The religious people enjoying the outdoors.
b) The group of people enjoying the outdoors.
2.
Mark only oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the BB model will predict for the following input sentence pair based on the explanation provided below.
1) A group of men play soccer on the beach.
2) A group of boys are playing soccer on the beach.
Explanation: The following are three pairs of sentences analogous to the input pair of sentences. The BB predictions for each of these analogous pairs is indicated in parenthesis.
Question 3 of 30
1tt ana analogous pair (similar)
a) A group of men playing soccer stadium full of people.
b) group of men playing soccer stadium full of people.
2nd analogous pair (somewhat similar)
a) A pair of men walk along the beach.
b) Two boys in black swimming trunks are holding another boy by his arms and legs on beach.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
3rd analogous pair (somewhat similar)
a) Four swimsuits playing volleyball the beach. b) girl is sandy beach.
3.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the BB model will predict for the following input sentence pair based on the explanation provided below.
1) A man is playing harp.
2) A man is playing keyboard.
Explanation: The following are three pairs of sentences analogous to the input pair of sentences. The BB predictions for each of these analogous pairs is indicated in parenthesis.
Question 4 of 30
1st analegous pair (somewhat similar)
a) A woman playing the flute.
b) A man is playing keyboard.
2nd analogous pair (somewhat similar)
a) A guy playing hackysack
b) A man is playing key-board.
3rd analogous pair (similar)
a) Women are running.
b) Two women are running.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the BB model will predict for the following input sentence based the explanation provided below.
1) A man is playing harp. 2) A man playing keyboard.
Explanation: The following are three pairs of sentences analogous to the input pair of sentences. The BB predictions for each of these analogous pairs is indicated parenthesis.
Question of 30
1t analogous pair (similar) a) Women are running.
b) Two women are running.
2nd analogous pair (somewhat similar)
a) A man is playing flute.
b) A man plays keyboard.
2nd analogous (dissimilar)
a) Police believe Wilson then shot Jennie Mae Robinson once in the head before turning the gun herself.
b) 'What they have done is thinly veiled attempt do an end run around the Constitution, she said.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what model will predict for the following input sentence pair based the explanation provided below.
playing harp.
2) playing keyboard.
Question 6 of 30
Explanation: The sentences appear somewhat similar. The words "keyboard' and "harp' important differences between the
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
Please choose below what BB model will predict for the following input sentence pair based the explanation provided below.
woman swimming underwater. 2) slicing carrots.
Explanation: fallowing pairs sentences analogous input pair The predictions for each these analogous pairs indicated parenthesis.
Question 7 of 30
1tt analogous pair (dissimilar)
a) person wearing scuba gear is swimming underwater
b) cutting up
2nd analogous pair (somewhat similar)
a) relaxing bath
b) slicing carrot.
3rd analogous pair (diasimilar)
a) young girl smiling with her head upside down
b) Nikon and Sigma 10-20mm super-wide shots.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
below model will predict for following input sentence pair based the explanation
provided below.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
1) woman swimming underwater.
Question 8 of 30
2) slicing carrots.
Explanation: The sentences appear dissimilar. The words "some", "carrots" and 'woman' the important differences between the
8.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what BB model will predict for the following input sentence pair based on the explanation provided below.
1) A woman swimming underwater.
2) A man is slicing some carrots.
Question 9 of 30
Explanation: The sentences appear somewhat similar. The words 'man', 'woman' and 'some' are the most important differences between the two.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs http://wwwww..ooowooooo///////tn
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please below what the model will predict following input sentence pair based explanation provided below.
playing guitar.
is playing trumpet.
Question 10 of 30
Explanation: The appear somewhat similar. The word "guitar" the important difference between the
10.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
Please choose below what the model predict the following input pair based explanation provided below.
playing guitar. playing trumpet.
Question 11 of 30
Explanation: The sentences appear differences between the
words 'trumpet" 'guitar" the most important
11.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below the BB model will predict following input sentence pair based on explanation provided below.
1) man is playing guitar. 2) man is playing trumpet.
Explanation: following are three pairs sentences analogous input pair sentences. The predictions each these analogous pairs is indicated parenthesis.
Question 12 of 30
analogous pair (somewhat similar)
a) man is playing guitar.
b) man is playing flute.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
2nd analogous pair (somewhat similar)
panda climbing.
b) climbing rope.
analogous (similar)
advised that certain allegations criminal conduct been interposed against my counsel, said Silver. b) advised that certain allegations criminal conduct have been interposed against my counsel, J. Michael Boxley,
Silver statement said."
12.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the model will predict the following input sentence pair based on explanation provided below.
1) A man eating a food.
2) A man eating a piece bread.
Question of 30
Explanation: The sentences appear somewhat similar. The word 'piece' the most important difference between the two.
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs http://wwww...oooooooooo/////tto
13.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
choose below the BB model will predict for following input sentence pair based the explanation provided below.
1) eating food.
2) eating piece of bread.
The BB predictions for Explanation: The following three pairs analogous the input each of these analogous pairs indicated in parenthesis.
Question 14 of 30
st analogous (similar)
a) An animal chewing on something.
b) animal chewing on key chain.
2nd analogous pair (somewhat similar)
a) The Hare Psychopathy Checklist often used assess psychopathy clinical settings.
b) From an article entitled Can You Call 9-Year-Old Psychopath?
3rd analogous pair (similar)
a) player bounces ball.
b) player catching ball.
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs http://wwww...ooooooooooo//a///n
14.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
choose below the BB model will predict for following input sentence pair based explanation provided below.
1) eating food.
2) esting piece of bread.
The BB predictions for The following three pairs analogous the input each of these analogous pairs indicated in parenthesis.
Explanation:
Question 15 of 30
analogous (similar)
a) An animal chewing on something.
b) animal chewing on key chain.
2nd analogous pair (dissimilar)
a) girl dancing in cage.
b) onion being chopped.
analogous pair (somewhat similar) a) shoots basket.
b) man is playing guitar.
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
https://dccs hTtp://wwww....oooooooo/a//////a
15.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
choose below what the model will predict for following input sentence pair based explanation provided
below.
1) just have your what you which is what you
2) may want it, the process given you what you have work within.
Explanation: The following three pairs each of these analogous pairs indicated in parenthesis.
analogous the input pair The B8 predictions for
Question 16 of 30
analogous pair (somewhat similar) a) English, certainly the
of do Do-Support.
b) traditional grammar, the word doing participle in all your examples.
2nd analogous pair (somewhat similar) a) Stocking very subjective
specific inhabitants the setup your aquarium experience. add fish into tank slowly.
b) You'll always
analogous pair (somewhat similar)
a) is rooftop
b) is holding microphone
14
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
https://dccs http://wwww.....ooooo///////////
16.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the model will predict following input sentence pair based explanation provided below.
1) just have your what you know, which is what you
2) may want it, the process given you what you work within.
Explanation: The following three of anslogous the input pair of The predictions for each these analogous pairs indicated parenthesis.
Question 17 of 30
analogous pair (dissimilar) a) Even though the question has been answered would like add my answer
b) don't know should say this, but your job making you anxious.
2nd analogous pair (dissimilar)
will advise exactly the contrary of what just another answer: go A b) could defer admission, but it's little unusual defer year.
analogous (dissimilar)
a) People good philosophy of mathematics mathematicians philosophers.
My motivation that look things from their roots go up.
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs http://wwww..pooooooooo/a//////a
17.
Mark only one oval.
Similar
Somewhat Similar
Dissimilar
Can't say
Please choose below what the model will predict for following input sentence pair based explanation provided below.
1) just have your what you which is what you
2) may want it, the process given you what you work within.
Explanation:
The following three pairs of analogous the input pair The B8 predictions for each these analogous pairs indicated in parenthesls.
Question 18 of 30
analogous (dissimilar)
a) Since you're being kind vague here, only give you vague
b) Look this: http::/www..cppeettet/ www.:/aawpwwwtttriillccoom co. uk/ You could do stages.
2nd analogous pair (dissimilar)
a) You always then choice the author accept that yes, you this.
b) The short
analogous (dissimilar)
a) define problem before attempting solution.
b) As understand nothing that changed, broken down smaller parts inherently real.
16
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs hTtp://ww.......ooo..a////////..
18.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the model will predict for following input sentence pair based explanation provided below.
1) should just your what you do.
2) should listen your boss, because you're paid tell the boss what do.
Explanation: The following three pairs each these analogous pairs indicated in parenthesis.
analogous the input pair The B8 predictions for
Question 19 of 30
analogous (similar) a) young girls hug each other the grass.
b) young girls holding each other grasay ground
2nd analogous pair (somewhat similar)
a) Artists worried plan would harm those need help performers who have difficult time lining up shows.
b) The artists say plan will harm French culture punish those need help most performers who have hard time lining up work.
analogous pair (somewhat similar) a) The villains absolutely hate selfish and self-serving, they also cowards. down is, his/her core, someone who does the suffering others.
b) villain you
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
https://dccs
19.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what model predict for the following input sentence pair the explanation provided below.
You should just your boss what he wants you You should listen your boss, because you're not paid
what
Explanation: The following each of these analogous pairs indicated parenthesis.
pairs
analogous input pair sentences. The predictions for
Question 20 of 30
1st analogous pair (similar) don't know if should say this, your job making you anxious. don't know should say this, your job making you anxious.
2nd analogous pair (dissimilar)
Eugene suggests, try figure out what you want achieve. supervisor is instructing the male workers.
3rd analogous pair (dissimilar)
woman supervisor instructing the male Basically, the rule that you must play the ball, and man, until catches ball.
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs wTtp://wwww.....ooooooo///////tn
20.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what model predict the following input pair based explanation provided below.
You should just ask your boss what you should listen to your boss, because you're paid tell the what
Question of 30
Explanation: The sentences appear similar. The words 'he', 'the" and "tell" the most important differences between the
21.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
Please choose below what BB model will predict for following input sentence pair based the explanation provided below.
1) need
you like you don't.
2) Yes, you should create portfolio showcase you and what you've done.
Explanation: The following each of these analogous pairs indicated in parenthesis.
pairs sentences analogous input pair sentences. The predictions for
Question 22 of 30
1st analogous pair (dissimilar)
a) I'd say you need distract yourself from the bigger picture.
b) No, you don't need have classes earned degree your
2nd analogous pair (dissimilar) a) little girl and boy reading books.
b) completeness, Apple's Pages has quite few nice poster layouts.
3rd analogous pair (diesimilar)
a) The info you've been given previous b) You'll need check the particular policies of each publisher what allowed what allowed.
good.
22.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose model predict for the following input sentence pair explanation
provided
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
You need lot know what you like and what you don't.
Question 23 of 30
Yes, you should portfolio site showcase what you do and what you've done.
Explanation: The sentences appear dissimilar. The words 'lot", "know' and "done" the important differences between the two.
23.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the model will predict for the following input sentence pair based on the explanation provided below.
You need read lot to know what you like and what you don't. 2) Yes, you should create a portfolio site showcase what you can do and what you've ve've done.
Question 24
of 30
Explanation: The sentences appear somewhat similar. The words "lot", 'read' and "can" are the most important differences between the two.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
24.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose what model predict the fallowing input pair based explanation provided below.
well the dolphin scheme, the chaos has allowed foreign companies engage damaging logging and fishing operations without proper monitoring export controls. Internal chaos has allowed foreign companies up damaging commercial logging and fishing operations without proper monitoring export controls.
Question 25 of 30
Explanation: The sentences appear similar. The words "engage", 'dolphin' and 'the" the important differences between two.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs http://wwww...ooooooooo/o//////a
25.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what model predict the fallowing input pair explanation provided below.
well the dolphin scheme, the chaos has allowed foreign companies engage damaging logging fishing operations without proper monitoring export controls. Internal chaos has allowed foreign companies up damaging commercial logging fishing operations without proper monitoring export controls.
Question 26 of 30
Explanation: The sentences appear similar. important words identified in making different.
26.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
10/2/11, 4:11

---

a) Singapore already the United States' largest trading partner, with two-way trade totaling more than $34 billion. Al Explanations for Pairwise Similarity Models
Please choose below what the B8 model will predict for following input sentence pair based on the explanation provided below.
1) As well as dolphin scheme, chaos allowed foreign companies to engage in damaging logging and fishing operations without proper monitoring or export controls.
2) Internal chaos has allowed foreign companies to set up damaging commercial logging and fishing operations without proper monitoring export controls.
Explanation: following are three pairs of sentences analogous input pair sentences. The predictions for of these analogous pairs indicated parenthesis.
Question 27 of 30
1t analogous pair (similar)
a) The American Anglican Council, which represents Episcopalian conservatives, said will seek authorization to create separate group.
b) American Anglican Council, which represents Episcopalian conservatives, said authorization create separate province North America because last week's actions.
2nd analogous pair (somewhat similar)
a) Stage episode declared when ozone levels reach 0.20 parts per million.
b) federal standard ozone 0.1 parts per million.
analogous pair (similar)
b) Although city-state, Singapore is 12th-largest trading partner of United States, with trade $33.4
billion last year.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
https://docs. wttp://wwww.....owooo/o////////n
27.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
below what the model will predict following input sentence pair based explanation provided below.
1) depends what you next, and where you 2) guess depends what you're going do.
Question 28
of 30
Explanation: The sentences appear somewhat similar. The words 'want" and "going" the most important differences between the two.
28.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
depends you next, where you http://www......oo/oo//////////m 2) guess depends what you're going to
Please choose below what model will predict for the following input sentence pair based the explanation provided below.
Explanation: fallowing
pairs sentences analogous input pair sentences. The predictions for each of these analogous pairs indicated parenthesis.
Question 29 of 30
1tt analogous pair (somewhat similar) second half minutes faster then first.
a) prefer
b) would definitely go slightly slower first half.
2nd analogous pair (somewhat similar)
a) The pound also progress against the dollar, reached fresh three-year highs $1.6789.
b) The British pound flexed muscle against dollar, last percent $1.6672.
3rd analogous pair (somewhat similar)
a) started crying and yelling him, 'What you mean, you saying, why did you lie me?"
b) Gulping for started crying and yelling him, 'What you mean?
29.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
Please choose below what the BB model will predict for the following input sentence pair based on the explanation provided below.
1) It depends on what want to do next, and where want to do it.
10/2/11, 4:11

---

Al Explanations for Pairwise Similarity Models
guess depends what you're going
Explanation: The following each of these analogous pairs indicated parenthesis.
pairs
analogous input pair sentences. The predictions for
Question 30 of 30
1st analogous pair (dissimilar) wrote above, it's hard rate this wall.
Unlike others, think the route pretty well described.
2nd analogous (somewhat similar) Remember, from the Fleet's point Well, really depends how long there, and exact speed the Fleet.
rest the galaxy what's moving experiencing dilation.
3rd analogous pair (similar)
Deaths in rollover crashes accounted for percent of number traffic deaths 2002, agency says. Fatalities rollover crashes accounted 62 percent the increase 2002, NHTSA said.
30.
Mark only one oval.
Similar
Somewhat similar
Dissimilar
Can't say
10/2/11 4:11

---

Al Explanations for Pairwise Similarity Models
31. Please provide any (optional) feedback you might have. For example, did you find any of the explanations helpful, and if why? Did you prefer one type of explanation to another? Was this preference governed by the complexity of the input sentences, etc.?
This content nelther created endorsed Google.
Google Forms
10/2/11, 4:11


Reviews:
[{"review_id": "rS2uA3ypdgi", "reviewer": "Reviewer_Un2j", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- Novel formalization of objective function for finding analogies and feature attribution for BB similarity learners\n\n- Diverse evaluation of approach using both objective metrics and a user study\n\nWeaknesses/questions:\n\n- On objective 1: how can one compare doing LIME over the concatenated (x,z) to having A be diagonal?\n\n- On objective 2: Optimization over $\\lambda_1, \\lambda_2$ is unclear, how can one systematically search over them to get intuitive analogies? Furthermore, why is $\\alpha$ set to 0 in all the experiments? What is the effect of $\\alpha$ both quantitatively and qualitatively ?\n\n- Figure 2: why are the words on the x and y axis shuffled from their original order? Also this kind of visualization is a bit hard to parse, is there a better way to visualize the cross weights (off diagonal elements of A) ?\n\n- Issues with user study: 1) using google forms and non-paid participants raises questions on the effort each put into performing the user study.  2) showing participants the same example with different explanation methods: as I understand, there are 10 test examples, and you show each participant the same 10 with 3 different random methods to explain. Thus the participant has access to 3 explanations when they get to third time they see a given example, this clearly biases the results. A correct methodology is to assign each participant to a condition ( an explanation baseline) and only show them 10 examples with explanations from that condition. Then you compare between conditions. 3) no alignment between objective of user study (replicate BB model scores) and practical use cases: supposedly the explanations are to check if the BB is correct or not, why wasn’t that the use case for the user study? I expect it’s because humans are perfect at judging similarity, then it might have been more interesting to introduce an end task where judging similarity is used. \n\n -  I am not sure why is fidelity the “metric” to compare things across for judging similarity.  Furthermore, it would have been helpful to understand the implications of a low generalized infidelity score and a high score. (Ramamurthy et al., 2020) also relies on comparing the feature importance weights, is it possible to do something here that is similar? \n", "summary_of_the_paper": "Goal: provide local explanations for black box (BB) models that assign similarity scores to two input examples. \n\nApproach: Two explanations generated: 1) feature attribution and 2) similar pair of examples that serve as analogies. \n\nApproach for 1) is:\n\nApproximate the BB model on the instance as if it was a quadratic model of the pair of inputs and learn the weighting matrix A by sampling pairs of points around the input and solving the resulting SDP for the matrix A. The weights of A provide a way to assign value to each element of the inputs. \n\nApproach for 2 is:\n\nObtain K different pairs of examples that serve as analogies by solving a subset selection problem from the data that balances three terms:\n\n1) the analogy pair must have similar score to the query according to the BB model\n\n2) the analogy pair and query should have similar features highlighted (weighted by a HP $\\lambda_1$\n\n3) the K pairs should be diverse\n\nEvaluation: The authors show three different kind of evaluations\n\nQualitative examples: on text data from STS, the authors show three examples with the results of their method\n\nA user study on the STS dataset where participants have to predict if the BB predicted the sentences to be similar given the two kinds of explanations. They show that their method outperforms the baselines\n\nQuantitative results on 3 datasets: STS, UCI Iris and MEPS where authors show that their method outperforms LIME and other baselines in terms of a metric called “generalized infidelity” \n", "summary_of_the_review": "The paper presents a novel approach for obtaining explanations from a black box measure. The method appears sound, however, the evaluation is lacking in certain aspects. The user study has some flaws and the quantitative experiments rely on a single metric. My recommendation is a borderline reject that can be improved if authors can better argue their evaluation approach.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "kd-5M6neydC", "reviewer": "Reviewer_F17R", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper’s proposed explanation form seems to be very interesting. After the problem and the explanations methods are well motivated and introduced, the authors first illustrate them on selected examples. This shows nicely the methods’ purpose. However, the STS dataset's task and the selected examples do not well support the quality of the generated explanations and the benefit of analogies-based explanations. I could imagine that the MEP dataset would be more relatable.\n\nAfter this illustration, the methods are extensively evaluated with a designed user study and quantitative evaluations both taking (adapted) previous methods into account. The results demonstrate the purpose and the benefits of the proposed methods. Summarized, the approaches seem to be very interesting, especially since similarity learners became more and more popular in recent years, even beyond the text and tabular data domain. Applying, evaluating and extending these methods for e.g. the vision domain seems to be interesting.\nTherefore, I'm tending to accept the present work.\n\nMinor comments:\n\nIt took me a while to understand Figure 1 when it is first mentioned. Consider using shorter samples or/and expanding the description in the introduction. After reading section 5.1 it became more clear.\n", "summary_of_the_paper": "This paper introduces a novel form of explanations for similarity-based models. The authors present two methods to generate explanations, called Feature-based similarity explanations and analogy-based similarity explanations, where the latter one is a novel type of explanations explicitly developed to explain similarity learners. However, the authors state that both can be used simultaneously, i.e. the latter can use the output of the first.\nThe authors conduct a user study as well as a quantitative evaluation of the proposed methods with a comparison to previous approaches. Since the proposed setting (analogies) is novel, previous approaches compared with needed adaptations to fit the setting. ", "summary_of_the_review": "I'm tending to accept this paper as it is well written and provides interesting and novel approaches to explain similarity-based methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ZCDYNZBlUZ3", "reviewer": "Reviewer_QDef", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I have summarized the main review into the following pros and cons: \n\nPros: \n\n* The proposed technique is flexible as it can provide two forms of explanations: feature and analogy-based. Moreover, explanations in the form of analogies are intuitive for human users. \n* The study includes human and functionally grounded evaluation experiments to show the usefulness of the proposed explanation technique.\n\nCons:\n\n* Many important design choices behind the proposed method in sections 4.1 and 4.2 are not well motivated.\n* Some of the methods in functionally-grounded evaluation are not included in the human grounded evaluation experiments and vice versa. This makes it difficult to draw a general conclusion in favor of the proposed approach across both types of evaluation methods. \n", "summary_of_the_paper": "The paper proposes a new technique for explaining models that predict the similarities of an input pair. The authors propose two forms of explanations for such models: feature and analogy-based. Feature-based explanations highlight the important features of a predicted similarity for an input pair. For the explained pair, analogy-based explanations provide a new input pair that has a similar relationship to one another. The proposed technique outperforms other similar techniques in human and functionally grounded empirical experiments.\n", "summary_of_the_review": "Overall, I vote for rejecting the paper. Although the proposed technique performs well in both human and functionally grounded evaluation experiments, many important design choices are not well motivated. Overall, I believe that the study needs some further refinements before it can be accepted to ICLR 2022. \n\n\nI have divided my detailed feedback into two categories: “major concerns” and “minor improvements”. I am willing to improve my current score in case the authors can address points raised in the major concerns section.\n\nMajor Concern\n\n* What are the reasons that LIME and JSLIME are performing relatively similar in comparison to the proposed FBFull and FBDiag methods on MEPS dataset (Table 1)? Does that mean that the problem at hand can be solved with LIME and JSLIME formulation as well? If so, what are the benefits and limitations of the proposed explanation techniques in this paper? \n\n* How can the usefulness of the analogy-based explanations be argued for when the result of user studies show that users can get nearly similar accuracies using AbE or FBFull (Figure 3)? \n\n* Can authors provide explanations on the effect of each of five additive components in Equation 2? \n\n* What are the reasons for not performing the human and functionally grounded evaluations on the same set of techniques? In addition, how can this affect the generalized statements about which explanation techniques perform best across both evaluation experiments? (For example, LIME and JSLIME are missing in human studies in Figure 3 whereas PDash is missing in the functionally grounded evaluation in Table 1)\n\n* Why lambdas and alphas are not tuned per example and what is the effect of this on the fidelity of “local” explanations (section 5.1 - AbE hyper-parameters)? \n\nMinor Improvement\n\n* Can authors provide a more detailed explanation for the problems that hinder the extension or use the work of [Zheng et al., 2020; Plummer et al., 2020; Zhu et al., 2021] for the problem at hand? \n\n* I see a potential problem in the additive definition of w_{x_i, y_i} (section 4.1). In the current definition, the loss cannot differentiate between these two cases:  perturbations x_i s are close to x and many y_i points are further away from y and vice versa. This can be problematic since removing and adding terms to the explained pair of instances changes the Mahalanobis distance asymmetrically (see Example 1-3 in Figure 2). Can authors confirm this and provide an analysis on the possible effect this can have on the quality of explanations?\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Fk_pJTB7CLm", "reviewer": "Reviewer_UBTb", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths: \n\nThis paper studies an under-studied problem of explanation for similarity models. Due to the particular natures of the similarity prediction task, methods that do not focus on interaction effects (i.e. pretty much all local explainers for the classification setting) understandably could not perform well. The authors solved this problem by the use of a learned local distance matrix, in which interaction effects are clearly shown. In addition, the proposed method of analogy-based-explanation seems novel. The explicit treatment of diversity sets it apart from other explanation methods that also use whole data point for explanation, such as counterfactuals. \n\n## Weaknesses: \n\nDespite the strengths, I do have serious concerns about the experimental evaluation, which fails to convince me of the quality of the explanation. \n\n### Qualitative analysis\n\nFor the qualitative explanation, and AbE in particular, none of the three examples in Sec. 5.1 are convincing, and they feel more like post-hoc over-explanation on the auhors' part based on the analogy pairs produced. As a concrete example, consider the (author-provided) explanation on the analogy-based explanation provided in example 1: \n\n\"The first analogy is very similar except that hackysack is a sport rather than a musical instrument.\" -- Yes, but does this show that the model is recognizing the same-type-ness of the entity (musical instrument or sport) when making the similarity prediction? \n\n\"The sentences in the second pair are more similar than the input pair as reflected in the corresponding BB distance.\" -- This statement is not about explanation, but rather merely about model prediction. \n\n\"The third analogy is less related (both sentences are about cricket player selection) with a larger BB distance.\" -- Again, this statement is only about model prediction. \n\nFor example 2, despite some Internet search, I could not understand what a \"dolphin scheme\" is. Is it a particular way for economic fraud/exploitation (like \"pyramid scheme\")? As a result, I could not understand the author-provided explanations for this example, and I do not think it is a good opening example for the same reason. \n\nFor example 3, \"Both sentences express the same idea (second half faster than first half) but in different ways, similar to the input pair.\" -- This is a very loose assertion. In fact, if both sentences truly express the same idea, I would expect the similarity to be much higher (i.e. distance much smaller), but this pair is actually the most dissimilar pair among the three. \n\n### Quantitative analysis\n\nI have serious concerns about the simulatability user study in the paper. In summary, this design is easily game-able. Since the users have access to the explanation at \"test time\", a simple AbE explanation for achieving such correct prediction would simply be to produce pairs with similar predicted distances, regardless of any similarity in the reasoning process. To make it even worse, if the users could be \"trained\" for a bit, a \"Trojan explanation\" could easily lead to very high user performance, without the users understanding the model at all. For more details about the \"game-ability\" of this approach and the \"Trojan explanation\" definition, see https://arxiv.org/abs/2012.00893 and https://arxiv.org/pdf/2006.01067.pdf. The authors are suggested to consult an earlier proposal for user study design https://arxiv.org/abs/2006.14779, which (in my opinion) avoids this loophole. \n\nIn addition, a synthetic task with known groundtruths could objectively evaluate various properties of the proposed methods, such as whether the highlighted words are indeed important, or whether the analogical pairs also use the same reasoning pattern. Some ideas are discussed in https://arxiv.org/abs/2104.14403 and https://arxiv.org/abs/2105.06506. \n\nMinor: \n\nThe authors could use \\citet in places such as \"Smith (2019) proposed\", rather than the current \"(Smith, 2019) proposed\". \n\nFor gradient-based explanation, the authors cite GradCAM, but I view it more as an adaptation of the original CAM to non-fully-convolutional architectures, and GradCAM are fundamentally about visualizing maximally activating input regions for certain convolution layers/filters. Instead, I would recommend the original Gradient saliency paper by Simonyan et al. (2013) or its SmoothGrad/IntegratedGradient successor. ", "summary_of_the_paper": "This paper studies the problem of explanation for similarity prediction models. Given a pair of inputs (x1, x2), the model to be explained assigns a distance (or similarity) score. The task is then to explain the model prediction on individual inputs. Two methods are proposed. First, a feature-attribution style explanation is computed by learning a local distance approximation, similar to the LIME objective. Second, an analogy-based-explanation is used, in which a set of existing pairs of data are selected, and the pairs are encouraged to both be semantically diverse and share similar model reasoning process at the same time. In the experiments, qualitative and quantitative results are presented. Qualitative results are delivered mainly for the STS dataset. Quantitative results are provided via a user study demonstrating that the users can better predict the model prediction when given the proposed explanation compared to baselines, and an automatic evaluation showing that a global version of proposed method perform better than existing approaches on the fidelity metric. ", "summary_of_the_review": "Unfortunately, I do not believe that this paper meets the standard for publication. While I like the proposed theory, I am really unconvinced by the experimental evaluations. In fact, the qualitative AbE examples raise more questions than they answer, and make me doubtful that the method is really working as intended. A more careful experimental investigation, perhaps with some revision to the theoretical approach based on the problems found, would make this paper a much better one. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "9m8sSlCtpmi", "reviewer": "Reviewer_Lu2q", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe main reason to accept this paper is empirical results, showing performance on the various methods. The author has done plenty of case studies to verify the explanation and with many examples of the proposed explanation method. \n\nWeaknesses: \nGeneralizability of the proposed method. The author shows results on language tasks.  Can it be applied to other tasks? If yes, the author should show results on vision tasks and shows and compare results with state-of-the-art methods.\n\nTo find better similar and contrasting examples in the vision domain, people use exemplar [1,2] theory to find supporting and opposing examples.\n\nWhat sort of feature attribute did the author consider for the explanation? Do you have a section discussing feature attributes?\n\n“set of perturbations (x _i, y_ i ) in the neighborhood,” what kinds of perturbation is used in the text domain?\n\nThe author should compare results with the SHAPE[3], LOO[4], RISE, and Occlusion-based method for input perturbation and U-CAM method for logit perturbation.\n\nHow is it(analogies) different from paraphrasing a sentence? The author could motivate the paper on why analogies help to improve the explanation.\n\n\nHow did the author measure similarity between two inputs (word or phrase or sentence level)? Do you have an analysis of this?\n\nHowever, the paper misses one of the core aspects of machine learning practice: readability and reproducibility of results. What are the various critical components in the proposed method? The author should provide an algorithm or pseudocode to reproduce the results missing in this paper.\n\n\nRef:\n1. Jäkel, Frank, Bernhard Schölkopf, and Felix A. Wichmann. \"Generalization and similarity in exemplar models of categorization: Insights from machine learning.\" Psychonomic Bulletin & Review 15, no. 2 (2008): 256-271.\n\n2. Patro, Badri, and Vinay P. Namboodiri. \"Differential attention for visual question answering.\" In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7680-7688. 2018.\n\n3. Lundberg, S. M.; and Lee, S.-I. 2017. A unified approach to interpreting model predictions. In Advances in neural information processing systems, 4765–4774.\n\n4. Li, J.; Monroe, W.; and Jurafsky, D. 2016. Understanding neural networks through representation erasure. arXivpreprint arXiv:1612.08220.\n\n", "summary_of_the_paper": "The author addresses the problem of post hoc explanation for the black box model. In this paper, the author discusses the task of explanation for two inputs, and the model provides a similarity score as output. The author proposes a model agonistic local explanation method for tabular and structure data. In the proposed method, the author uses feature attributes to explain the similarity between two inputs. Finally, the author proposed an analogy-based explanation to select diverse analogous pairs of examples for the same similarity. The author claims that using the proposed method can explain state-of-the-art models in healthcare utilization applications.", "summary_of_the_review": "I have worked in this field and published related to this work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents two novel approaches to provide explanations for the similarity between two samples based on 1) the importance measure of individual features and 2) some of the other pairs of examples used as analogies.  The proposed approach to explain similarity prediction is a relatively less explored area, which makes the problem addressed and the proposed method unique. However, reviewers expressed concerns about evaluation methods and there were some concerns about the design choices that were not well motivated. The major issue is, as pointed out by the majority of the reviewers, the evaluation methods. Given the paper, reviews, and responses of the authors and the reviewers, it appears that there is certainly room for improvement for more convincing evaluation methodologies to convince a cross-section of machine learning researchers that the proposed approach advances the field. Overall, this is a good paper, but appears to be borderline to marginally below the threshold for the acceptance.
Decision: Reject