Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes LINEX, an explainable AI(XAI) method. In a typical categorization of XAI, LINEX deals with a method to explain a given black-box predictor locally with features, i.e. providing an explanation of a given data with important features. LINEX employs IRM and game theory in its formulation and the optimization of the explanation model fitting objective. In addition to existing desired properties for the explanation model, the authors also introduce unidirectionality, which is persuasive. The LINEX is compared with existing approaches on various modalities and show competitive or better performance with respect to various metric including the proposed unidirectionality.
strength_and_weaknesses: ### Strengths
- The paper combines IRM with game theory to find a local explanation model.
- A novel property for the feature-based local explanation model, unidirectionality, has been proposed.
- LINEX is shown to outperform existing methods with different ways to generate local environments.
- In experiments with various modalities, LINEX generally shows better performance than existing methods on various metrics.


### Weaknesses
- Lack of rigorous proof of the main theoretical contribution.
    - Even though a sketch is provided in the proof, it is quite difficult to check whether the claim is correct. I cannot find proof in the appendix, either. This may be kind of trivial to someone with appropriate knowledge of game theory. If so, I hope that the authors provide some references to check the correctness of the proof.
    - From the sketch, I can get kind of a rough picture of how $l_{\inf}$ affects. However, since $l_{\inf}$ regularization is claimed as one of the core contributions of the paper, the proof at least for the 2-players case should be given rigorously.
    - Since it is a sketch, it doesn't have to be complete. Still, it does not seem to cover cases exhaustively.
    - If the authors provide rigorous proof, I will check it and re-evaluate on this weakness.


### Questions
- Sensitivity to $\gamma$, $l_{\inf}$ norm bound
    - It seems that $l_{\inf}$ regularization seems to play an important role. It seems that $\gamma$ is chosen reasonably as specified in Appendix C.1. Still, I am curious how sensitive the performance of LINEX with respect to the choice of $\gamma$.

clarity,_quality,_novelty_and_reproducibility: Except for the justification of the claim in Thm.1, the paper conveys the main idea properly and the detailed implementation can be depicted with the provided algorithm. A novel metric, unidirectionality, seems to be a meaningful contribution to feature-based local explanation model research. 
summary_of_the_review: Overall, the idea proposed in the paper and its empirical validation are convincing, except for the lack of rigorous proof of the Thm.1. It seems that even without its rigorous proof, there are meaningful contributions of the paper. However, if the authors want to include the theoretical contribution, I think it is better to be self-contained to a certain level. If rigorous proof or a more detailed sketch with a certain level of rigor can be provided, I would increase the score.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The article describes a black box perturbation based explainability approach. It introduces an evolution of the LIME algorithm where the linear local predictor used to identify feature attribution is estimated using an extra $l_\infty$ constraint. The algorithm is justified by a Game Theory interpretation and inspired by an Invariant Risk Minimization approach. In addition to fidelity and stability, it proposes a new evaluation metric for explanation based on computing the sign consistency of attribution values and called unidirectionality. The approach is evaluated on a series of small size datasets containing text, tabular and image data. A large appendix (20 pages) with complementary experiments  is added to the main paper.

strength_and_weaknesses: == Strengths ==

- Simple but efficient evolution of  LIME algorithm for explainability

- Thorough evaluation and good results on five small size datasets for five metrics

== Weaknesses ==

- The novelty is incremental (an evolution of LIME)

- The theoretical justification of the algorithm is interesting but should be improved.

- Evaluation limited to small dimension data.

clarity,_quality,_novelty_and_reproducibility: * Novelty

The paper introduces a new desirable objective for local explanations (unidirectionality) and proposes an algorithm strategy to reach it: but basically, it exploits the same conceptual framework as LIME (linear approximation of local perturbations) but with another cost, and is mostly an incremental work, although interesting.

* Clarity & Quality

I appreciated the good endeavor to provide a theoretical interpretation of the approach but I didn’t find it fully convincing. I found it difficult to understand the logical links between IRM, Game Theory and the actual proposed algorithm, in its written form (loop over environments + iterations until convergence). I also found the Game theory interpretation a little bit pedantic and not fully developed or rigorously expressed. It would be nice to improve this justification (from theoretical principles to algorithm).

The writing also must be checked: several symbols are not defined ($c^x_e$, what is a strategy $S$) or used differently ($u_i$ takes one or two arguments in the definition of Nash equilibrium.)  

Regarding experiments, the appendix provides many details on how they have been conducted, which is a good thing. However, to fully justify the approach, it would be nice to have, at least on examples, how it behaves on more complex data, typically higher dimension image data than FMNIST or CIFAR10. 

* Reproducibility

The algorithm seems simple to implement but relies on many hyper-parameters: the appendix shows many experiments to demonstrate their impact; however a simple rationale describing a way to control them would be helpful.

The authors claim that the Code will be provided during the discussion phase.

summary_of_the_review: The paper describes a well motivated evolution of the classical LIME approach, with a thorough experimental analysis but on small size data. The novelty is incremental but gives good performance. The theoretical justification of the algorithm is interesting  but should be improved.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a new explanation device, i.e. LINEX, that requires only black-box access to the target model, motivated by Invariant Risk Minimization (IRM). To compute the feature attribution score locally at an input x, LINEX iterates over different environments until the overall approximation error is less than the threshold, with constraints encouraging a Nash Equilibrium (NE) over all environments. To show the effectiveness of LINEX, the paper compares it with several LIME-based baselines on 5 different metrics. 
strength_and_weaknesses: ### Strength
Overall I think the methodology, minimizing IRM with constraints incorporating NE, is novel and sound. It provides the means to capture the model’’s behavior at a particular instance in a broader neighborhood (i.e. a set of environments). As a result, the method is automatically justified compared to some other baselines.  

### Weakness
There are two major weaknesses of this work. 

**Presentation.** The current presentation of the story is so confusing and hard to follow. 

In Section 2 (Related Work), the paper gives an overview of explanation methods without defining the term “explanation”. There is a large body of explanation types, e.g. feature attribution, instance-based explanations, rule extractions, etc., and the scope of the paper should be specified. Important setups like “In this work we do not assume availability of such additional information” (access to parameters) should appear somewhere else instead of the section discussing related work. Moreover, I have to read the entire paper before reading the last paragraph of Section 2 because a lot of contexts are missing if I don't read the rest. 

Section 4.1 is most confusing. It introduces a lot of metrics that neither motivates the proposed method nor provides any insight why the new method would do better on them until the experiment section shows up. It is explaining metrics that are only discussed in the empirical part. Why not put this into the experiment section?

Finally, the way LINEX is introduced is strange to me. The authors directly present Algorithm 1 after introducing a number of metrics that do not motivate the algorithm. My questions then follow: (1) what is the motivation of this method. Namely, what are the problems that existing work does not solve and LINEX does?; and (2) why LINEX may do well on the metrics defined in Section 4.1 compared to baselines? Namely, how do IRM and NE constraints relate to these metrics while baselines are not or not relating in a tight way? The theoretical motivations of Algorithm 1 is present after the introduction of the algorithm in Theorem 1 with an example of two environments; Therefore, to understand Algorithm 1 I have to read the paper in a reversed order. 

**Experimental Results.**. Firstly, The family of LIME is not the only one that provides attribution scores. For example, NB and Square in Yeh et al (2019) [1] and Occlusion [2]. Secondly, the paper claims that LINEX is more stable and unidirectional and faithful in the same degree. It seems this argument is only true for IRIS because on other datasets sometimes the results are usually within the ballpark (and there are a lot of NAs in the table or the results on CIFAR10 only works between MeLIME and LINEX). Also, within the axis of fidelity, LINEX is better than MAPLE on GI but worse on INFD on IRIS dataset. Can authors explain what this means? Why for the same axis we need two metrics, e.g. INFD and GI. Lastly, I would appreciate more analytical discussions on why the proposed method should be better on these metrics. As a result, I know what to expect from the empirical results and it helps to convince me that the proposed method solves the problems in the existing ones. The empirical results so far are a bit dense without a clear interpretation.

Also, it would be nice to provide the run-time comparisons. 


[1] Yeh, Chih-Kuan et al. “On the (In)fidelity and Sensitivity of Explanations.” NeurIPS (2019).

[2] Zeiler, Matthew D., and Rob Fergus. "Visualizing and understanding convolutional networks." European conference on computer vision. Springer, Cham, 2014.

clarity,_quality,_novelty_and_reproducibility: ### Clarity 
As explained in the previous part, the presentation of the paper is one of the major weaknesses. The only part that is clear to me is the technical overview of the new method. The motivation why we need LINEX (e.g. the problem in the existing methods and why IRM is a good fit for solving the problem) is unclear. Some sections and paragraphs could be better arranged so that the readers do not need to firstly read the rest of the paper with questions and undefined concepts in mind.  The theorem that motivates the method should be firstly introduced before the actual algorithm appears. 

### Quality
The overall quality of the empirical results is satisfied. The paper includes many recent metrics for evaluating explanation methods and recent LIME-based results. The theoretical part from my perspective is sound. 

### Novelty
The paper is novel in terms of the methodology. It is a very good fit for the problem.

### Responsibility 
The author has well-stated the impact and ethics of this work. 

summary_of_the_review: I vote for rejection for now as the presentation of the work and the lack of sufficient experiments outrun the novelty of the method so that I don’t think the current paper is ready for publication.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new explanation method that improves the robustness of LIME to variations in perturbation methods and sample size. Technically, it leverages the invariant risk minimization (IRM) framework to design a new method for fitting the approximation model. Specifically, it first defines the environment concept for the explanation task, which is an exemplary neighborhood around the example that needs to be explained. Different environments correspond to neighborhoods generated via different algorithms (i.e., random perturbation or realistic generation). Once the environments are generated, it learns its specific linear classifier for each environment and sums these models together to get a final model, and uses the final model as the explanation. The paper gives a theoretical analysis of why such a method could provide high-fidelity explanations that are also stable and unidirectional explanations across nearby examples. Empirically, the paper verifies the effectiveness of the proposed method on five datasets in three different modalities (i.e., tabular, image, and text).
strength_and_weaknesses: Strength:
+ The experimental design is relatively comprehensive.
+ The proposed method is overall clearly presented.  

Weaknesses:
- My first major comment is the overall novelty of the proposed technique. Different from existing works that learn one approximation model on collected data, it just learns separate models and adds them together. Since the linear model is additive, I am wondering why not just learn one approximation model using data collected from different environments. Would that give similar results to the proposed model? Besides, the evaluation shows relatively marginal improvements. In some cases, the proposed method even gives worse performance than the baseline approaches. This makes me doubt the effectiveness of the proposed design. 

- It is not that clear how NE helps with the development of the proposed technique. I understand the proposed technique is developed based on IRM loss in Section 3. But I am not sure how NE guides the development process. 

- In section 4.2.1, the author argues, “features that have a disagreement with even the direction of their impact are eliminated by our method .”I wonder whether this property is good or not for the stable explanation. Intuitively, if some attribute is sensitive, it may vary or even change signs for different environments since different environments correspond to the different neighborhoods we generated around the sample we need to explain. However, according to Algorithm 1, the coefficient of this attribute would be set to zero, which means this attribute is not important anymore for our explanation. Could the authors provide more evidence for this implicit assumption (i.e., why they are unimportant)? 

- In Table3, we could notice that in terms of INFD metric, the proposed method performs not well compared to other baselines in some datasets. Could the author give some explanations about this result?

- I would suggest the authors further highlight the advantages and benefits brought by the proposed method. Say with the proposed method, the number of needed neighborhoods is reduced a lot.  

- I do not fully agree with the authors' argument about the computational expansiveness of using realistic neighbors. One can use Gaussian blur or mean feature value to generate these neighbors. A pretrained generative model can also be used for neighbor generation. All the methods are not that expensive. 
clarity,_quality,_novelty_and_reproducibility: The proposed method is clearly presented and should be able to reproduce given Algorithm 1. IMHO, the technical novelty is limited and the benefits of the new design are not that significant. 
summary_of_the_review: This paper proposes a novel method to provide more robust explanations. Technically, the proposed method makes minor changes to the existing explanation method, and the empirical advantage is not that significant. As such, I do not rank this work as making enough contributions to be accepted by ICLR.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes LINEX, an explainable AI method, based on LINE. Based on the reviews, there was no strong support to accept this paper and several concerns among reviewers: 1) incremental novelty compared to LIME, 2) concerns with the theoretical justification of the algorithm, 3) concerns related to the experimental evaluation. Based on that, I recommend rejecting the paper.
justification_for_why_not_higher_score: There was no strong support for accepting this paper but several concerns.
Decision: Reject