Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The paper considers the problem of training black box models for improved interpretability, and  proposes to penalize black-box models at training time using two regularizers that correspond to fidelity and stability explanation metrics. As computing the regularizers exactly is computationally intensive they propose two approximating algorithm. In addition, as the one for fidelity is still prohibitive, a randomized variant is proposed. Connections are established between the regularizers and the model's Lipschitz constant or total variation. A generalization bound is presented for local linear explanations. The proposed approach is evaluated on a variety of datasets. 

The paper deals with an important problem and the exposition is clear.  While regularizing deep learning models is a pertinent  direction, I feel the paper makes a couple of overstatements, and overall I am not fully convinced by the approach and empirical evaluation, as outlined below.

- The paper states "recent approaches that claim to overcome this apparent trade-off between prediction accuracy and explanation quality are in fact by-design proposals that impose certain constants on the underlying model families they consider" and that they are addressing this shortcoming. But in fact, the proposed regularizations do exactly the same: they impose certain constraints. Indeed the regularizers encourages models with lower Lipschitz constant or with small total variation across neighborhoods.

- I also find that it is unsurprising that regularizing via fidelity or stability will lead to models with better fidelity/stability so it's an artificial way to yield "improved interpretability" and this is more of an issue because fidelity and stability are kind of proxy metrics to evaluate interpretability.

- It would be important to investigate further the difference between regularization and explanation neighborhoods. This might not be a bad thing which in fact help with generalization. 

- Proposition 1 supports algorithm 2, but it is not a given at all that Algorithm 1 will have smaller local variances across neightborhoods and hence might generalize well.  It would be important to proceed with an empirical study of the local variance across neighborhoods for Algorithm 1.
 
- Computational complexity remains an issue as ExpO-1D-fidelity is performing much worse.

- Comparison against alternative approaches beyond SENN are lacking (e.g. Lee et al 2019, Wang and Rudin,2015 etc).
 
 Overall I feel that more work is needed to convincingly demonstrate the importante of the proposed approach.
 
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper proposes an approach for local post-hoc explanation with introduction of a new regularization that helps regulate the "fidelity" and "stability" of the output explanation (in the style of LIME).  The fidelity regularizer is essentially the squared error of the explainer as compared to the given model in the local neighborhood, whereas the stability regularizer measures the total squared differences between the explanation outcomes between the sample in question and other samples in the local neighborhood. 
In the experimental evaluation section, the authors evaluates the performance fo the proposed regularizers, used as part of both LIME and MAPLE, against three interpretability metrics: point fidelity, neighborhood fidelity and stability. The results verify that indeed the use of the regularizers improve the performance of both LIME and MAPLE over the unregularized versions, with respect to the corresponding metrics. This is in a way "expected", since the regularizer used in the method and that in the metric are closely related, and is an unsatisfying aspect of the work. 
Using image data, they also demonstrate that qualitatively the use of stability regularizer seems to significantly improve the saliency of the output explanation. 
The paper is well written, the proposal is reasonable, but the contribution is modest and experimental evaluation is not entirely convincing. 

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: 
The paper proposes a new type of regularizer to improve explainability in neural networks. The proposed regularizer is largely based on two metrics, namely fidelity and stability. It optimizes for fidelity and stability as a regularization objective in a differentiable manner.

I would recommend for accept, as the paper shows in its experiments that the explainability of neural networks can be improved with the two proposed regularizer, which outperforms simple baselines of l1 and l2 regularizers. The paper's method is generic, and can be applied to almost all machine learning models with gradient-based optimization, making it helpful to building explainable machine learning systems.

However, I would also like to note that the results in this paper are somewhat unsurprising. The ExpO-Fidelity and ExpO-Stability regularizers can be seen as (almost) directly optimizing the fidelity and stability metric for explainability, so one would naturally expect that models trained with these regularizers will do better on the two metrics above. In addition, I do not see much value in the derivations of Section 3.2. The conclusion that "explainable models with smaller local variances ... are likely to have explanations of higher fidelity" is unsurprising and almost a straightforward claim.
review_title: Official Blind Review #1

Meta-review: This paper investigates a promising direction on the important topic of interpretability; the reviewers find a variety of issues with the work, and I urge the authors to refine and extend their investigations.
Decision: Reject