Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The authors propose to add a regularizer to the loss function when training a prediction model. In particular, the regularizer considers explanations during the model training; if the explanations are not consistent with some prior knowledge, then explanation errors will be introduced. 

The motivation for the proposed research is interesting and has some merit. However, I am a bit worried that the proposed approach is somewhat ad hoc. I can imagine there are various explanations that can be generated for the same model. There can also be different prior knowledge available for a particular problems. Which prior knowledge and explanations to use seem to affect a lot about the learned model. But there is no principled approaches for making the selection.

In some sense, standard regularizers such as L1 or L2 are are intrinsic regularizers, while the proposed regularizer is extrinsic regularizer. I think the extrinsic regularizer certainly has some merit, but it is also hard to regulate.

For instance, consider the example in Figure 2 about the presence of patches. Isn't that a too specific knowledge about the dataset, which in turn makes the proposed approach not general? I have doubts on how useful a method is if it relies on such specific prior knowledge about the data.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper presents a way of using generated explanations of model predictions to help prevent a model from learning "unwanted" relationships between features and class labels. This idea was implemented with a particular explanation generation method from prior work, called contextual decomposition (CD). For a given feature, the corresponding CD can be used to measure its importance. The proposed learning objective in this work optimizes not only the cross entropy loss, but also the difference between the CD score of a given feature and its explanation target value. Experiments show that this new learning algorithm can largely improve the classification performance.

I like the high-level idea of this work and agree that there is not much work on using prediction explanations to help improve model performance. However, there are two major concerns of the model and experiment design. 

First, it seems like the proposed method requires whoever use it already know what the problem is. For example, 

- in section 3.3, the model inputs include a collection of features and the corresponding explanation target values.
- in section 4.1, it is already known that some colorful patches only appear in some non-cancerous images but not in cancerous images. 
- it is even more obvious in section 4.2 and 4.3, because in both experiments, the training and test examples were altered on purpose to create some mismatch. 

My question is that if we already know the bias or the mismatch, why not directly use this information in the regularization to penalize some features? Is it necessary to resort to some explanation generation methods?

My second concern is more like a personal opinion. In the experiment of section 4.2, if the colors are good indicators of these digits in the training set, I don't it is wrong for a model to capture these important features. However, the way of altering examples in the same class with different colors in training and test sets seems questionable, because now, the distributions of training and test images are different. On the other hand, if we already know color is the issue, why not simply convert the images into black-and-white? A similar argument can also be applied to the experiment in section 4.3

Overall, I like the idea of using explanations to help build a better classifier. However, I am concerned about the value of this work. 

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper presents a method intended to allow practitioners to *use* explanations provided by various methods. Concretely, the authors propose contextual decomposition explanation penalization (CDEP), which aims to use explanation methods to allow users to dissuade the model from learning unwanted correlations. 

The proposed method is somewhat similar to prior work by Ross et al., in that the idea is to include an explicit term in the objective that encourages the model to align with prior knowledge. In particular, the authors assume supervision --- effectively labeled features, from what I gather --- provided by users and define an objective that penalizes divergence from this. The object that is penalized is $\Beta(x_i, s)$, which is the importance score for feature s in instance $i$; for this they use a decontextualized representation of the feature (this is the contextual decomposition aspect). Although the authors highlight that any differentiable scoring function could be used, I think the use of this decontextualized variant as is done here is nice because it avoids issues with feature interactions in the hidden space that might result in misleading 'attribution' w.r.t. the original inputs.

The main advantage of this effort compared to work that directly penalizes the gradients (as in Ross et al.) is that the method does not rely on second gradients (gradients of gradients), which is computationally problematic. Overall, this is a nice contribution that offers a new mechanism for exploiting human provided annotations. I do have some specific comments below.

- I am not sure I agree with the premise as stated here. Namely, the authors write "For an explanation of a deep learning model to be effective, it must provide both insight into a model and suggest a corresponding action in order to achieve some objective" -- I would argue that an explanation may be useful in and of itself by highlighting how a model came to a prediction. I am not convinced that it need necessarily lead to, e.g., improving model performance. I think the authors are perhaps arguing that explanations might be used to interactively improve the underlying model, which is an interesting and sensible direction.

- This work, which aims to harness user supervision on explanations to improve model performance, seems closely related to work on "annotator rationales" (Zaidan 2007 being the first work on this), but no mention is made of this. "Do Human Rationales Improve Machine Explanations?" by Strout et al. (2019) also seems relevant as a more recent instance in this line of work. I do not think such approaches are necessarily directly comparable, but some discussion of how this effort is situatied with respect to this line of work would be appreciated.

- The experiment with MNIST colors was neat. 

- The authors compare their approach to Ross and colleagues in Table 1 but see quite poor results for the latter approach. Is this a result of the smaller batch size / learning rate adjustment? It seems that some tuning of this approach is warranted. 

- Figure 3 is nice but not terribly surprising: The image shows that the objective indeed works as expected; but if this were not the case, then it would suggest basically a failure of optimization (i.e., the objective dictates that the image should look like this *by construction*). Still, it's a good sanity check.



review_title: Official Blind Review #3

Meta-review: The paper contains interesting ideas for giving simple explanations to a NN; however, the reviewers do not feel the contribution is sufficiently novel to merit acceptance.
Decision: Reject