Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a post-hoc explainability algorithm that is global and model-agnostic. The global comes from training from batches of data while the model-agnostic property is achieved by a gradient estimator.  Experiments show the proposed method outperforms IG and LIME from aspects of sparsity, sufficiency, and time. Some case studies also validate its effectiveness.
main_review: **Strengths**
1.	The proposed explainer is model-agnostic and fairly efficient, which has great potential in applications. Moreover, such universality is validated by various models, including Random Forest, BERT, and CNN.

**Weakness**
1.	I find the paper hard to read, and lots of its parts seem to be wordy and lack clarity. I suggest the authors thoroughly reorganize the lines of writing, especially for the introduction and method section. 
2.	As suggested in the introduction part, the novelty of this works partly comes from the global feature of datasets instead of a single instance, which is not new in the literature, e.g., [1]. 
3.	Regardless of the variety of models in the experiments,  more competitive baselines are necessary to justify the superiority of MAGNEX.  
4.	I think more clarifications for the experimental conclusions are needed. For example, why would the sparsity values be the same for all the methods? Plus, it would be nice to provide a more in-depth study about the performance gain of MAGNEX.


[1] Parameterized Explainer for Graph Neural Network. Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, Xiang Zhang.
summary_of_the_review: I think this work is not ready for publication, thus I lean towards rejection.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a global explainers for black-box models. Original
inputs/features are fed in parallel with the black-box model to an
explainer model that is in charge of deciding important scores for
each feature. Depending on the scores, features are dropped and the
remaining input is also bed to the original model. The outputs for the
original input and the trimmed version with the explainer's help are
compared. The explainer is trained such that the original input and
the trimmed version produce similar scores when fed to the original
model. In addition, the sparsity of the trimmed input is maximized.

The new explainer is compared to LIME and Integrated Gradients in
terms of how faithful the produced explanations are (how close the
results of the original input and the trimmed one are when passed
through the black-box model) and the execution time on three different
tasks: image classification, sentiment analysis and question
answering. The new explainer produces the results much faster and with
similar and sometimes higher quality (as measured by faithfullness).


main_review: I think the approach in this paper is fairly intuitive. The results
are considerably better wrt execution time. One thing that was not
clear to me: is prediction time for the trimmed input going through
the original model included in the total execution time?

In the introduction, there is a statement that I'm not sure I agree
with. Last phrase of the first paragraph: "Furthermore, explainability
is an important mechanism to ensure black-box models act fairly and
without bias". I'm not sure that once we understand model predictions
we understand its fairness characteristics. Perhaps undersstanding
model predictions could help with understanding its fairness.

In Figure 4, it's not clear what the question is.

I think the paper would have a much stronger contribution if a small
sample of the results for the explainer and LIME would be judged by
humans for how helpful they are for the human to appreciate the
results of the model. As it stands now, the execution time is the main advantage (albeit, a strong one). 

summary_of_the_review: I think this paper introduces a straightforward technique for global explainers that has clear execution time advantages over existing explainers.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes a global model-agnostic neural network-based explainer.
main_review: The proposal of the paper is interesting. However, it has some weaknesses especially in the experimental part that should be fixed before publication. In particular, the evaluation is not convincing because: a) the sparsity metrics takes always the same values for all the methods tested and there are not tests of statistical significance; b) the evaluation is done with respect to two measures optimized by the MAGNEX (it would be odd that it get low results on them). However, it should be tested also with respect to other evaluation measures; c) the main advantage of MAGNEX seems to be the low runtime but this aspect is not the focus of the paper; d) there are no examples of explanations for images or question-answering; e) the approach should be tested also on the most simple data type, i.e., tabular data; f) I recommend adding a large array of baselines such as surrogate decision trees and random forests returning as explanation the features importance, SHAP, Trepan. Finally, I recommend investigating how changes the performance of these baselines when varying instead of assigned to it the same retrieve by MAGNEX.

Minor. Some mission related works:
- Setzu, Mattia, et al. "GLocalX-From Local to Global Explanations of Black Box AI Models." Artificial Intelligence 294 (2021): 103457.
- ElShawi, Radwa, et al. "ILIME: Local and Global Interpretable Model-Agnostic Explainer of Black-Box Decision." European Conference on Advances in Databases and Information Systems. Springer, Cham, 2019.
- Spinner, Thilo, et al. "explAIner: A visual analytics framework for interactive and explainable machine learning." IEEE transactions on visualization and computer graphics 26.1 (2019): 1064-1074.
summary_of_the_review: Experiments are not sound nor convincing.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a global model-agnostic explanation method. The method relies on a neural model that learns to predict which input features are important for the original model’s predictions. Through substantial experimentation, the authors demonstrate that their approach outperforms LIME and Integrated Gradients. They compare between explanation methods 
in terms of the faithfulness of its explanations and in computational complexity.

The main contribution of this paper is MAGNEx, a neural model-agnostic explanation method that is substantially faster than popular methods on high-dimensional inputs.

main_review: This is an interesting paper that discusses an important and timely topic, generating fast and reliable model-agnostic explanations. The authors propose a novel and interesting approach that compares favorably with popular methods, and present experiments across many modalities and tasks. They also discuss and analyze the methods performance, and provide a detailed review of their technical approach.

While I do find the paper useful and clear, I have some issues that I would like to see addressed for it to be published. 

First and foremost, I find the lack of any discussion about causal explanation and especially counterfactuals to be highly problematic. Masking parts of the input and observing the model’s output does not causally explain the model’s decisions, even if the prediction of the model is the same.

The authors note that they intentionally do not compare to humans as we should care only about what the model is doing. However, they do not extend this reasoning to trying to causally estimate how decisions are made. I recommend the authors address the causal explanation literature, and explain why masking tokens or replacing existing pixels with black pixels is sufficient in terms of counterfactuals [1,2,3].

Second, I don’t see how is Figure 3 helpful for analysis. As I read it, this seems to be just the authors’ opinion regarding the model’s internal state, without any comparison to counterfactuals. Also, MAGNex mostly highlights “great” in this particular example, so I don’t fully agree with the intuitive explanation. Same critique stands for Figure 4 as well.

Third, the empirical results are not as convincing as the authors claim. Apart from time, MAGNEx doesn’t seem to be improving much in the first three experiments, and the fourth has huge standard deviations. In the BERT experiment, if tested on significantly shorter sentences, even the time advantage might not be true. Also, the CNN experiments should be done on a substantially more challenging dataset


[1] Explaining Classifiers with Causal Concept Effect (CaCE)
[2] Causalm: Causal model explanation through counterfactual language models
[3] Amnesic probing: Behavioral explanation with amnesic counterfactuals

summary_of_the_review: While I do find the paper interesting and find some of the contributions meaningful, I have some issues with the paper in its current form. First and foremost, I find the lack of any discussion about causal explanation and especially counterfactuals to be highly problematic. Second, the qualitative analysis of the output of MAGNEx compared with LIME is unconvincing. Third, the empirical results are not as strong as the authors claim, apart from the important and substantial gain in time efficiency.
review_title: null

Meta-review: This paper proposes a global model-agnostic explanation method. The method relies on a neural model that learns to predict which input features are important for the original model’s predictions. Using experimentation, the authors demonstrate that their approach outperforms LIME and Integrated Gradients. They compare the explanation methods in terms of the faithfulness of explanations and computational complexity. While the premise of the work is interesting, reviewers have suggested several areas for improvement: i) writing can use more clarity. writing seems verbose and hard to understand especially in intro and methods section ii) several points have been raised about empirical evaluation including lack of user studies, a more extensive set of baselines and data modalities. Given this, we are unable to recommend an acceptance at this time. We hope the authors find the reviews helpful.
Decision: Reject