Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The authors address two important issues: semi-supervised learning from relatively few labelled training examples in the presence of many unlabelled examples, and visual rationale generation: explaining the outputs of the classifiier by overlaing a visual rationale on the original image. This focus is mainly on medical image classification but the approach could potentially be useful in many more areas. The main idea is to train a GAN on the unlabeled examples to create a mapping from a lower-dimensional space in which the input features are approximately Gaussian, to the space of images, and then to train an encoder to map the original images into this space minimizing reconstruction error with the GAN weights fixed. The encoder is then used as a feature extractor for classification and regression of targets (e.g. heard disease). The visual rationales are generated by optimizing the encoded representation to simultaneously reconstruct an image close to the original and to minimize the probability of the target class. This gives an image that is similar to the original but with features that caused the classification of the disease removed. The resulting image can be subtracted from the original encoding to highlight problematic areas. The approach is evaluated on an in-house dataset and a public NIH dataset, demonstrating good performance, and illustrative visual rationales are also given for MNIST.

The idea in the paper is, to my knowledge, novel, and represents a good step toward the important task of generating interpretable visual rationales. There are a few limitations, e.g. the difficulty of evaluating the rationales, and the fact that the resolution is fixed to 128x128 (which means discarding many pixels collected via ionizing radiation), but these are readily acknowledged by the authors in the conclusion.

Comments:
1) There are a few details missing, like the batch sizes used for training (it is difficult to relate epochs to iterations without this). Also, the number of hidden units in the 2 layer MLP from para 5 in Sec 2.
2) It would be good to include PSNR/MSE figures for the reconstruction task (fig 2) to have an objective measure of error.
3) Sec 2 para 4: "the reconstruction loss on the validation set was similar to the reconstruction loss on the validation set" -- perhaps you could be a little more precise here. E.g. learning curves would be useful.
4) Sec 2 para 5: "paired with a BNP blood test that is correlated with heart failure" I suspect many readers of ICLR, like myself, will not be well versed in this test, correlation with HF, diagnostic capacity, etc., so a little further explanation would be helpful here. The term "correlated" is a bit too broad, and it is difficult for a non-expert to know exactly how correlated this is. It is also a little confusing that you begin this paragraph saying that you are doing a classification task, but then it seems like a regression task which may be postprocessed to give a classification. Anyway, a clearer explanation would be helpful. Also, if this test is diagnostic, why use X-rays for diagnosis in the first place?
5) I would have liked to have seen some indicative times on how long the optimization takes to generate a visual rationale, as this would have practical implications.
6) Sec 2 para 7: "L_target is a target objective which can be a negative class probability or in the case of heart failure, predicted BNP level" -- for predicted BNP level, are you treating this as a probability and using cross entropy here, or 
mean squared error?
7) As always, it would be illustrative if you could include some examples of failure cases, which would be helpful both in suggesting ways of improving the proposed technique, and in providing insight into where it may fail in practical situations.
review_title: Novel approach addressing important problems

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The main contribution of the paper is a method that provides visual explanations of classification decisions. The proposed method uses 
 - a generator trained in a GAN setup
 - an autoencoder to obtain a latent space representation
 - a method inspired by adversarial sample generation to obtain a generated image from another class - which can then be compared to the original image (or rather the reconstruction of it). 
The method is evaluated on a medical images dataset and some additional demonstration on MNIST is provided.


 - The paper proposes a (I believe) novel method to obtain visual explanations. The results are visually compelling although most results are shown on a medical dataset - which I feel is very hard for most readers to follow. The MNIST explanations help a lot.  It would be great if the authors could come up with an additional way to demonstrate their method to the non-medical reader.

 - The paper shows that the results are plausible using a neat trick. The authors train their system with the testdata included which leads to very different visualizations. It would be great if this analysis could be performed for MNIST as well.


From the related work, it would be nice to mention that generative models (p(x|c)) also often allow for explaining their decisions, e.g. the work by Lake and Tenenbaum on probabilistic program induction.
Also, there is the work by Hendricks et al on Generating Visual Explanations. This should probably also be referenced.

minor comments: 
- some figures with just two parts are labeled "from left to right" - it would be better to just write left: ... right: ...
- figure 2: do these images correspond to each other? If yes, it would be good to show them pairwise.
- figure 5: please explain why the saliency map is relevant. This looks very noisy and non-interesting.


review_title: Review of "Thinking like a machine — generating visual rationales through latent space optimization "

final_score: rating: 4: Ok but not good enough - rejection
review_content: * This paper models images with a latent code representation, and then tries to modify the latent code to minimize changes in image space, while changing the classification label. As the authors indicate, it lies in the space of algorithms looking to modify the image while changing the label (e.g. LIME etc).

* This is quite an interesting paper with a sensible goal. It seems like the method could be more informative than the other methods.  However, there are quite a number of problems, as explained below.

* The explanation of eqs 1 and 2 is quite poor. \alpha in (1) seems to be \gamma in Alg 1 (line 5). "L_target is a target objective which can be a negative class probability .." this assumes that the example is a positive class. Could we not also apply this to negative examples?

"or in the case of heart failure, predicted BNP level" -- this doesn't make sense to me -- surely it would be necessary to target an adjusted BNP level? Also specific details should be reserved until a general explanation of the problem has been made.

* The trade-off parameter \gamma is a "fiddle factor" -- how was this set for the lung image and MNIST examples? Were these values different?

* In typical ICLR style the authors use a deep network to learn the encoder and decoder networks. It would be v interesting (and provide a good baseline) to use a shallow network (i.e. PCA) instead, and elucidate what advantages the deep network brings.

* The example of 4/9 misclassification seems very specific. Does this method also work on say 2s and 3s? Why have you not reported results for these kinds of tasks?

* Fig 2: better to show each original and reconstructed image close by (e.g. above below or side-by-side).

The reconstructions show poor detail relative to the originals.  This loss of detail could be a limitation.

* A serious problem with the method is that we are asked to evaluate it in terms of images like Fig 4 or Fig 8. A serious study would involve domain experts and ascertain if Fig 4 conforms with what they are looking for.

* The references section is highly inadequate -- no venues of publication are given. If these are arXiv give the proper ref. Others are published in conferences etc, e.g. Goodfellow et al is in Advances in Neural Information Processing Systems 27, 2014.

* Overall: the paper contains an interesting idea, but given the deficiencies raised above I judge that it falls below the ICLR threshold.

* Text:

sec 2 para 4. "reconstruction loss on the validation set was similar to the reconstruction loss on the validation set." ??

* p 3 bottom -- give size of dataset

* p 5 AUC curve -> ROC curve

* p 6 Fig 4 use text over each image to better specify the details given in the caption.




review_title: This paper models images with a latent code representation, and then tries to modify the latent code to minimize changes in image space, while changing the classification label. As the authors indicate, it lies in the space of algorithms looking to modify the image while changing the label (e.g. LIME etc).

Meta-review: The paper proposes a semi-supervised method to make deep learning more interpretable and at the same time be accurate on small datasets. The main idea is to learn dense representations from unlabelled data and then use those for building classifiers on small datasets as well as generate visual explanations. The idea is interesting, however, as one reviewer points out the presentation is poor. For instance, Table 2 is not understandable. Given the high standards of ICLR this cannot be ignored especially given the fact that the authors had the benefit of updating the paper which is a luxury for conference submissions.
Decision: Reject