Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposed a method called SVD-RND to solve the out-of-distribution (OOD) detection problem. The proposed SVD-RND is under the Random Network Distillation (RND) framework, but make use of blurred images as adversarial samples. SVD-RND outperforms state-of-the-art methods in several OOD tasks.

In general, this paper is well-structured. It is not difficult to understand the problem this paper focuses on and the proposed method. I believe the proposed method is interesting, and I have not seen a similar approach before. It is a simple method, but it achieves excellent performance in multiple OOD tasks.

Although the authors try to explain why SVD-RND performs well, I am not entirely convinced and believe that more investigations might be necessary. Are the images from different datasets of similar average effective rank? I am also wondering how the proposed model will perform if we use other images as the adversarial examples. For example, we can use random low-rank images, which can be generated via Equation (3) by first randomly sampling $\sigma_{jt}$, $u_{jt}$, and $v_{jt}$, and then let a certain number of singular values be zero.

In summary, I am inclined to accept this paper because it proposes a simple method that gives high performance. However, I would suggest the authors include more ablation studies to help the readers understand why the proposed method works.

Minor:
I suggest the authors explicitly state that we do not update the network $g$ in Equation (1) and (4), such that the readers are less likely to be confused. The authors might also need to briefly explain why RND works.

Is a specific strategy applied to initialize the network $g$ randomly? Are weights in $g$ initialized using Gaussian distribution, uniform distribution, or via other initialization strategies?

In Table 1, all the datasets in columns 2, 3, and 4 are OOD samples, correct? 

To ensure that the TPR  is $95\%$ as described in Table 2, we need to tune the threshold for $|| f(x) - g_0(x) ||_2^2$, right? If a sample $x$ gives a $|| f(x) - g_0(x) ||_2^2$ that is higher than the threshold, it is considered as an OOD sample, correct?

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Summary: They tackle the problem of out-of-data distribution by leveraging RND applied to data augmentations. They train a model f(x) to match the outputs of g_i(aug_i(x)), where g_i is a random network and aug_i is a particular type of augmentation. An example with high error in this task is treated as an out-of-distribution example. This work focuses on exploring blurring through SVD, where the smallest K singular values are set to 0, and K varies between different aug_i calls. They find that their method of consistently can achieve strong detection rates across multiple target-dataset pairs.

Comments:
* The experimental results in this work are impressive, which introduces many more questions.
* The model used for f and g is not mentioned in the text.
* Figure 4 (left) suggests that the SVD-RND performs about the same between 10K and 50K examples. The level of robustness is surprising, but doesn’t seem to square with intuition that more data ought to help. How little data can be used? In other words, extend the graph to the left. 
* The geometric transforms baseline is not fair, since SVD-RND uses multiple SVD transforms (b_train > 1) whereas the geometric transforms only have one. Please run a model with all the geometric transforms. This result is important for understanding whether the gains come from the particular transform (SVD) or the number of transforms used.
* Following the spirit of the previous comment, what other data augmentations can be used in place of SVD? Typical image classification pipelines use a large variety of augmentations. I would suggest taking some augmentations from AutoAugment [1] and running RND on top of them.
* An experiment that is missing is RND trained on blurred images. Is the blurring itself the important component, or is having multiple different heads important?
* In general, I am confused about how a single head RND does not converge to 0 loss by learning the weights of g. This seems to be a simple optimization problem. The original RND paper avoided this problem by also using the network to learn a policy, but this does not exist in this approach.
* Furthermore, a comparison with Ren et al. [2] and Nalisnick et al. [3] would be useful. [2] also uses data augmentation to create a background model that is compared against the real model. One can probably simulate this approach by comparing the error rates of each head of RND.

In general, this work seems promising, but lacks proper ablations that elucidate what components of the method are important. I am happy to increase my score if the experiments suggests are added to the work.

[1] AutoAugment: Learning Augmentation Policies from Data. Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, Quoc V. Le
[2] Likelihood Ratios for Out-of-Distribution Detection. Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark A. DePristo, Joshua V. Dillon, Balaji Lakshminarayanan
[3] Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality. Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Balaji Lakshminarayanan
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: UPDATE: 
I acknowledge that I‘ve read the author responses as well as the other reviews. 
I appreciate the clarifications and improvements made to the paper. I‘ve updated my score to 6 Weak Accept. 

####################

This paper presents the idea to use blurred images as regularizing examples to improve out-of-distribution (OOD) detection performance based on Random Network Distillation (RND). The paper proposes to generate sets of such blurred images via Singular Value Decomposition (SVD) on the training images by pruning the lowest K non-zero singular values. The proposed method, SVD-RND, then extends the standard RND objective, which is to train a predictor network f to minimize the L2 loss to the output of some randomly initialized network over the (original) training data, with an additional regularization term that minimizes the L2 loss to the outputs of further multiple randomly initialized networks over the sets of blurred images. In OOD experiments on CIFAR-10, SVHN, TinyImageNet, LSUN, and CelebA, the proposed SVD-RND consistently outperforms baselines and recent competitors which are demonstrated to be vulnerable to blurred images.

I find it hard to make a definitive evaluation for this work at this point and would like to take the authors' responses into account for my final recommendation. The main idea of the paper to generate adversarial examples for training via blurring is rather simple and thus the novelty of this work is somewhat minor. I think the quality of the paper also suffers from many statements in the text that draw too general and too bold conclusions at this point in my mind. The presentation overall is rather unpolished (see comments below). However, I find the empirical results itself quite strong and convincing and think they would make a relevant and significant contribution to the community. I do have questions left open though that need clarification:

(i) I don’t see why different techniques for blurring (SVD, DCT, GB) should lead to such different results as the approach remains conceptually similar. Do you have a reason/intuition why SVD gives the best results? Might SVD just be the easiest to tune method?

(ii) What do you think is the key reason that SVD-RND also appears to generalize too non-blurry OOD samples? Could you elaborate more on the two reasons you give in the paper? (1. RND performance on samples orthogonal to the data; 2. Discrimination between data and its low-rank projection)

(iii) The generation and tuning of multiple sets of blurred images (how many samples per set?) may get quite extensive for large datasets. Could you be specific on the computational cost?

(iv) Might the deep generative models (e.g. GPND) fail to detect blurred images due to insufficient model capacity of the decoder which results in blurry reconstructions? Have you varied the network capacity or latent space dimensionality of such models?

(v) What is the idea behind choosing the log effective rank in such an equidistant manner as proposed?


####################
*Additional Feedback*

*Positive Highlights*
1. SVD-RND shows strong OOD detection performance in OOD experiments on a variety of image dataset combinations (CIFAR-10, SVHN, TinyImageNet, LSUN, and CelebA).
2. The related work includes major works from all the related lines of research (Deep anomaly detection; OOD detection using side information; Adversarial examples/training)
3. Useful hyperparameter selection criterion based on effective rank if no OOD validation data is available.

*Ideas for Improvement* 
4. I think the tone of the paper would greatly benefit from not drawing too general conclusions and too bold implications. Keep statements precise and evidence-based. Declare hypotheses as such.
5. I would appreciate a plot showing samples before and after blurring in the appendix to see the most effective degree of blurring. Maybe also compare the different blurring baselines here to see differences.
6. Report std. devs. with your performance results to infer statistical significance.
7. Compare to the specific geometric transforms method as proposed in the paper [3] and not only using those transformations within your RND approach.
8. Add missing deep anomaly detection related work [6, 2, 5, 1].
9. Expand the sensitivity analysis in Figure 3 (left) over a greater range of K. Especially, I would like to also see K = 0 (unblurred, original images) as a sanity check which may only improve over RND due to ensembling over multiple randomly initialized networks.
10. Consider the one-vs-rest anomaly detection evaluation benchmarks from Ruff et al. [4] or Golan and El-Yaniv [3] to further infer the generalization performance of the proposed method.
11. I think the paper spends too much time on introducing previous work. Section 2 Related Work and Section 3 Background might be combined into one section.

*Minor comments*
12. In the abstract: “... VAE or RND are known to assign lower uncertainty to the OOD data than the target distribution.” is a bit strong. Rather “have been observed” etc. This is a working hypothesis in the community, but there is recent work (https://openreview.net/forum?id=Skg7VAEKDS) indicating (at least for VAEs) those are effects of poor model design.
13. In the abstract: “... efficient in test time ...” » “... efficient at test time ...”
14. In the abstract: “... in CelebA domain.” » “... on the CelebA dataset.” or just “... on CelebA.”
15. Section 1: “However, such models show underwhelming performance on detecting OOD, such as detecting SVHN from CIFAR-10. Specifically, generative models assign a higher likelihood to the OOD data than the training data.”. I think those are way too general conclusions at the moment. Rather something like “OOD detection performance of deep generative models has been called into question” and “have been observed to assign ...”.
16. Section 1: “Such results clearly support the degeneracy of deep OOD detection schemes”. Again, I find this way too bold of a statement at this point in time.
17. Section 2: “... a recently proposed paper ...” » “... a recent paper ...”
18. Section 2: “... outlier data independent of OOD data.”. What would outlier data not being out-of- distribution be?
19. Section 2: “Golan et al. (2018) design geometrically transformed data and regularized the classifier ...”. Not regularized. They trained a classifier on labels identified with these transformations.
20. Section 2: “..., resulting in OOD detection in each labeled data” » “..., resulting in OOD detection on labeled data”
21. A subsection title directly following a section title is bad style. A major section should be introduced with a few sentences on what this section is about.
22. Figure 2, right plot: These loss curves are rather strange... Increasing, then sharply decreasing again. Is there a drop in learning rate at epoch 80?
23. Many axis labels are too small and hard to read.
24. In Section 3.3.: “.. in Section 7.2.” » “.. in Section 4.2.”?
25. The definition of the log effective rank in Eq.~2 is weird. It's the entropy over the singular value distribution, i.e.~$H\left(\sigma_1 / \sum_j \sigma_j, \ldots, \sigma_N / \sum_j \sigma_j \right)$. Also, the parameters/notation involved are not introduced.
26. Make clear you apply SVD on single images. The paper alters formulations between data matrix and images...
27. Several instances where citet is used instead of citep.
28. In Table 1: Separate the target column from the three OOD columns more clearly. (e.g. vertical separator, center OOD, target in bold, etc.)
29. In Section 5.1: “area of the region under the ... curve” » “area under the ... curve”.
30. Figure 3: Better explain the plots. Are the three curves the respective target classes? What is the OOD set?
31. Figure 4: Maybe use subfigures with individual titles/captions.
32. Section 6: “For evidence, we fine-tune the classifier ...” » “For evidence, we fine-tune a classifier ...”
33. Plots and Figures are somewhat scattered and not referenced chronologically.
34. Introduce the effective rank at the point where it is used (Section 6.2). Somewhat unclear why to introduce this in Section 3 already.
35. Visually speaking, I find the RND examples in Figure 4 actually more anomalous than the top-anomalous SVD-RND samples (flashy colors, weird angles, borders, high contrast, ...)


####################
*References*
[1] R. Chalapathy and S. Chawla. Deep learning for anomaly detection: A survey. arXiv preprint arXiv:1901.03407, 2019.
[2] H. Choi, E. Jang, and A. A. Alemi. Waic, but why? generative ensembles for robust anomaly detection. arXiv preprint arXiv:1810.01392, 2018.
[3] I. Golan and R. El-Yaniv. Deep anomaly detection using geometric transformations. In NIPS, 2018.
[4] L. Ruff, R. A. Vandermeulen, N. Görnitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, and M. Kloft. Deep one-class classification. In International Conference on Machine Learning, pages 4393–4402, 2018.
[5] L. Ruff, R. A. Vandermeulen, N. Görnitz, A. Binder, E. Müller, K.-R. Müller, and M. Kloft. Deep semi-supervised anomaly detection. arXiv preprint arXiv:1906.02694, 2019.
[6] T. Schlegl, P. Seeböck, S. M. Waldstein, U. Schmidt-Erfurth, and G. Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In Proceedings International Conference on Information Processing in Medical Imaging, pages 146–157. Springer, 2017.

review_title: Official Blind Review #3

Meta-review: The paper proposes a new method for out-of-distribution detection by combining random network distillation (RND) and blurring (via SVD). The proposed idea is very simple but achieves strong empirical performance, outperforming baseline methods in several OOD detection benchmarks. There were many detailed questions raised by the reviewers but they got mostly resolved, and all reviewers recommend acceptance, and this AC agrees that it is an interesting and effective method worth presenting at ICLR. 
Decision: Accept (Poster)