From: NeurIPS.cc/2022/Workshop/TEA/Program_Chairs
title: Paper Decision
decision: Accept
comment: This paper provided an interesting and unique perspective to review and unify existing explainable AI approaches. While the reviewer pointed out a few points that were missing, it is understandable for a workshop paper. But still, please try to incorporate the reviewers' comments in the final version to make the paper more clear to the audience.

From: NeurIPS.cc/2022/Workshop/TEA/Paper10/Reviewer_DKMf
title: A Paper Providing a Survey of Various Explanation Properties
review: Summary:
In this paper, the authors provided a survey of various explanation properties in the existing literature of explainable artificial intelligence. In particular, the authors focused on two properties: sensitivity and faithfulness, and described various formulations of the two properties in the literature.

Strengths:
- The survey is quite comprehensive, and provides much needed links among various works in explainable artificial intelligence.
- The paper is, for the most part, clearly written and easy to follow.

Weaknesses:
- The paper is primarily focused on posthoc explanation methods. However, there is a whole class of inherently/intrinsically interpretable models where explanations are part of the models. For inherently/intrinsically interpretable models, their explanations are naturally as sensitive as their model outputs, and their explanations are naturally faithful to the underlying models. The authors seemed to have ignored the presence of inherently/intrinsically interpretable models in their survey.
- To me, it seems that if an explanation mechanism is faithful to the underlying model, then the explanation sensitivity should be the same as the model sensitivity. In other words, faithfulness should be the more (or most) important property of an explanation mechanism. The authors are encouraged to draw more links among various explanation properties.
- Please rename the title. Instead of "A Unified View of Properties of Interpretable ML," "A Unified View of Properties of Explanations" more accurately describes the paper.
rating: 6: Marginally above acceptance threshold
confidence: 3: The reviewer is fairly confident that the evaluation is correct

From: NeurIPS.cc/2022/Workshop/TEA/Paper10/Reviewer_rtzF
title: It is good to unify properties of explanations in XAI
review: [Summary]

This paper summarizes properties used by previous studies to evaluate explanations. The authors classify the existing properties into four types, and analyze each type in detail. The paper shows that although different works use different ways to formulate these properties, their essence is usually the same.


[Strengths]

This paper focuses on a novel perspective in XAI, i.e., unifying the properties of explanations. As mentioned by the authors, there are various ways in previous studies to formulate a property, which hinder the standard evaluation of explanations. Therefore, I think this paper focuses on an important issue in XAI.

[Weakness]
- How to distinguish the robustness of explanations against perturbations from the case when the explanation changes on an adversarial example? Given a small adversarial perturbation on the input, the model output may be changed, and in this case, the explanation is also supposed to change. Then, how to evaluate whether the explanation is robust?
- The discussion about sensitivity via group-based perturbations is a bit confusing. It would be better if the authors provided more detailed definitions of the group, and the mathematical formula of the sensitivity.
- In Equation 11, it is claimed that the min operation is used to ensure that a few important features do not impact model output more than the set of all positive attributions. However, I think this is a prior assumption about the explanation by users. In fact, due to interactions between input features, the impact of a few important features may be larger than the set of all positive attributions. Therefore, this operation cannot faithfully evaluate the explanation.
- Some important properties/axioms in XAI are not discussed. For example, [cite 1] has proposed the SHAP method to estimate the attribution of input features, which is proven to satisfy the linearity, dummy, efficiency, and symmetry axioms. Many studies also use these axioms to evaluate their attribution methods. It would be better if the authors added these axioms into their discussion.
- The current paper only covers two kinds of properties, and I look forward to the discussions about more properties in the full version.

[cite 1] Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In NeurIPS 2017.

rating: 6: Marginally above acceptance threshold
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct

From: NeurIPS.cc/2022/Workshop/TEA/Paper10/Reviewer_KWQQ
title: Survey on explanation properties in ML
review: This paper surveys explanation properties in the ML literature and describes trade-offs between different formulations of these properties. Explanation properties are categorized in 4 categories: sensitivity, faithfulness, complexity and homogeneity. I think this paper is a really nice contribution to the workshop. I have a few suggestions for future iterations of this work. First, for a survey paper, there's a surprisingly low amount of citations. Please consider expanding the literature review in the future. Second, please consider adding an open questions section at the end of the paper. Given the amount of time the authors have spent reading and categorizing these works, I think they have a unique perspective to offer on what's missing in the field and what are important next steps to pursue under the lens of the survey.
rating: 7: Good paper, accept
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct