Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an extension of Locally interpretable model agnostic explanations (LIME) called Locally invariant explanations (LINEX).
LINEX is inspired by invariant risk minimization and aims to provide high-fidelity, stable, black-box invariant, unidirectional local explanations.
LINEX is empirically evaluated using tabular, image, and text data and the authors show improvement over LIME and other baselines over several metrics and datasets.
main_review: **Strengths**

The introduction presents a good motivation and explains well the gap in current research. Although you do a great job defining locality later in the paper, a sentence defining what is meant by local decisions and locality would make the intro easier to read. Similar for the terms faithful, stable, and unidirectional, which are technical terms that are used before definition in the intro.

The related work does a great job of summarizing global vs local explainability, exemplar vs feature based methods, as well as robust and causal approaches.

The preliminaries offers a faithful explanation of IRM and does well to distinguish local explainability from IRM.

The description of Nash equilibria and definition of a pure strategy Nash equilibrium as well as the Desirable properties are very well written, but I wonder if this shouldn't be part of the preliminaries? With the exception of unidirectionality, which may be considered a contribution?

The method description well describes and justifies the main contributions of the paper.

The experiments section does well to describe the datasets and metrics used.

**Weaknesses**
The main issues I have are with the experiments section.

1. I'm not sure that Figure 2. demonstrates that the explanations for LINEX are better than MeLIME. They. do look smoother, but does that make them better explanations? I don't like the use of the correlation coefficient here to imply that the LINEX explanations are better as I could just put the original image as the explanation and have a perfect score. For the shoe, why is the LINEX explanation better than MeLIME?

2. There doesn't seem to be a great deal of difference in the top attributed works for the sentiment analysis experiment. Is it possible to look at the top attributed bi-grams rather than just the top attributed words?

3. I appreciate the effort to report standard errors, but I'm not sure it makes sense to take these over different kernel sizes. How is this a sensible approach? Assuming that this is ok, why highlight 1% improvements rather than statistical significance?
summary_of_the_review: I think this is an interesting paper that has technical novelty, however there are issues with the experimental results that need to be addressed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Update:
In our back and forth, the authors have addressed or committed to address the concerns I had with this work. As a result, I have updated my score

---------------------------------------------

This paper introduces a new model agnostic local explanation method based on a game-theoretic formulation to ensure that resulting explanations are high fidelity and unidirectional across nearby examples. The idea is to first form several local environments via random perturbations or using creating realistic neighbors using generative or retrieval methods. Then, using the proposed algorithm, we iteratively learn a constrained least squares predictor for each environment and the final predictor is then the sum of these individual predictors. Experiments on three data modalities (tabular, vision, and text) show that the proposed method leads to better explanations than existing methods such as LIME.
main_review: - I think this paper is well motivated and offers good contributions.
- Experimental results appear to strongly support the efficacy of the proposed method. The proposed method outperforms LIME, S-LIME and MeLime in over half the experiments, while achieving comparable performance on most that remain. However, results on FMNIST and Rotten Tomatoes appear to not be as convincing as the results on tabular data. The paper is also missing discussion on this aspect.
- I didn't see an error analysis of the proposed method, especially for cases where the performance is worse than the baselines. That's something that I think could add to the paper quite a bit.
- Overall the paper is well written and the presentation is quite clear. I do have some minor comments that I have listed below.

------------------------------------------------------------------------
Additional comments and questions:
- Page 3 local explainability setup vs IRM setup: "we want to highlight features in our explanation that may be spurious from the domain perspective, but nonetheless the black-box model uses them to make decisions". I think this is a wrong assertion that the underlying model "uses them to make decisions". These are post-hoc explanations and per my understanding, nowhere in the paper do you actually establish that the model indeed only relies on the explanations you've identified. Am I missing something?
- Section 4.3, Assumption 1: Is this independence across dimensions in the same environment or the same dimension across environments? Current phrasing doesn't disambiguate but I'm assuming it's the latter? Please rephrase.
- Section 4.3 Assumption 2: I'm not quite sure I follow when you say that Assumption 2 ensures "that we closely analyze the role of the l_\inf penalty. Please elaborate.

-------------------------------------------------------
Presentation and typos:
- I'm not quite sure the title ("Towards Causal Explanations") is appropriate. To the best of my understanding, I could not identify any causal estimand being estimated in this paper.
- Introduction Line 6: "based on a image scan" -> "based on an image scan"
- Introduction Page 2 first paragraph: "There have been variants suggested to overcome..." Please expand upon this sentence and give some additional background to the reader.
- Section 4.2.1: "optimized subject two constraints" -> "optimized subject to two constraints"
- Section 4.2.1: "In other words, for features where there is massive disagreement in even the direction of their impact are eliminated by our method" There is an agreement error here. Rephrase it as "In other words, features that have a massive disagreement in even the direction of their impact are eliminated by our method." Further, what does "massive" even mean here? Better to use grounded language or not use adjectives at all and just say disagreement.
- Missing related work: Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason Huang, Yaron Singer, and Stuart Shieber. "Causal mediation analysis for interpreting neural nlp: The case of gender bias." NeurIPS 2020.
summary_of_the_review: Overall I think the paper provides a nice and sound contribution. The results are convincing on the tabular data but I would like some discussion about other modalities where results are somewhat comparable across several metrics. In addition, I have a few points I'd like some clarity on.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an algorithm LINEX for learning a robust LIME-like explanation for a black box model. The algorithm is based on the Invariant Risk Minimization Games framework proposed by [1],[2]. LIME is notorious for being very sensitive to its hyperparameters, and the paper posits that a robust variant of LIME is one which is invariant to these hyperparameters. The primary contribution of the paper is to formulate the perturbation-neighbourhoods of the point to be explained as ‘environments’ in an IRM setup. The paper also notes that characteristics of such invariant predictors (Theorem 2 of [2]) make them well-suited for explanations.
main_review: Strengths

+ LINEX is a robust form of LIME, without losing much fidelity
+ The motivation for choosing the IRM framework is laid out well
+ The experiments are over a wide range of datasets and evaluation metrics.

Concerns:
- LINEX doesn’t seem to be significantly better than MeLime or MAPLE, especially for non-tabular data. MAPLE is competitive on the MEPS dataset, while MeLime shows competitive performance on the FMNIST and Rotten Tomatoes datasets. When considering that LINEX takes 2.5x the training time as LIME (as mentioned in Appendix A), did other methods have the same benefit of additional training time/iterations?
- It is not clear why the paper states ‘Causal’ in the title (and also elsewhere in the paper), this seems unnecessary. The paper uses the word ‘causal’ loosely, and there is no support to the claim that LINEX explanations are causal.
- The paper states that ‘unidirectionality’ is a new property. However, there have been papers in recent years on robust explanations and attributional robustness (see [3],[4],[5],[6] references below). How is this different from these efforts? 
- The novelty of the work seems limited, as the algorithm and theoretical results in the paper are largely based on [1],[2].

[1] Ahuja, Kartik et al. “Invariant Risk Minimization Games.” ArXiv abs/2002.04692, 2020
[2] Ahuja, Kartik et al. “Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions.” ArXiv abs/2010.15234, 2021
[3] Lakkaraju, Himabindu et al. “Robust and Stable Black Box Explanations.” ArXiv abs/2011.06169, 2020
[4] Zhao, Xingyu et al. “BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations.” ArXiv abs/2012.03058, 2020
[5] Chen, et al, Robust Attributional Regularization,  NeurIPS 2019
[6] Sarkar, Anindya et al, Enhanced Regularizers for Attributional Robustness, AAAI’21

summary_of_the_review: Utilizing invariances for robust explanations is an interesting direction, but the paper relies heavily on the algorithms and results presented in [1],[2]. The results, though competitive, are not striking enough to warrant the usage of LINEX over existing methods (and also over other formulations for robust explanations such as [3],[4] in the above references).
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper approaches the problem of locally explaining predictions of black box models. They attempt to do so by creating local environments around a data unit. They propose their method, LINEX to overcome the shortcomings of the method, LIME. LINEX learns locally invariant predictors and aggregates their explanation for locally explaining the original model. The authors show some theoretical properties as well as empirical evidence.
main_review: One concern about creation of the local environments. Faithfulness of the local predictor to the original model has commonly be a concern in local explanation methods. Creating “environments” around a data unit further exacerbates this concern. How can one assure the faithfulness of the predictors in each environment to the original model? Even if one can empirically show that such approach works, without a theoretical guarantee, one cannot rely on LINEX. On the other hand, one other related concern is the answer to the following question: Hoe exactly to create environments? The authors themselves acknowledge that this is important and propose two ways of doing so: Random perturbation, and realistic generation according to the literature that they cite. IMO, one major concern is solving this issue. Pointing to ongoing work about this problem does not suffice. This paper is not clearly stating whether they are proposing a solid solution on how to create local environments.

The title of the paper suggests that this paper provides causal explanations. The main text attempts at slightly maneuvering over the terminology of causality. The abstract claims that the work ascertains explanatory features without using a causal graph. All of these evidence point toward using causal inference and/or explaining the black box model with causal features. However, the paper falls short of providing such explanations. In fact, to my understanding, the authors do not extract causal features. It would be great if the authors clarify on when/where they extract causal features, or alternatively, modify their writing not to reflect causal inference.

 The constraint in the optimization of the proposed algorithm does not necessarily formulate the situation of a Nash equilibrium. I am not clear about the theoretical properties of the Nash equilibrium in this problem. If one wants to interpret the results using concepts of this equilibrium, the whole problem needs to be formally formulated and axiomatically translated.

 The authors have shown their results on more than one environment (<= 5) in the Appendix. I am wondering about the behavior on even more environments? How would the authors recommend choosing the number of environments? I have an understanding that this is, at the moment, determined only empirically, and am not clear if there is any theoretical analysis over the number of environments.

On the application side, the paper is showing interesting results.

summary_of_the_review: The paper lacks in novelty, contribution, and supporting theory of the proposed methodology.
review_title: null

Meta-review: The reviewers are largely in agreement that this proposal would benefit from more clarity and comparison to key papers/findings in this space. While one reviewer is leaning towards acceptance, and their points were considered by the other reviewers, there wasn't a consensus towards aligning towardsa an acceptance. Thus, I recommend that the authors take advantage of the reviewers' comments to further improve their manuscript.
Decision: Reject