Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper proposes to detect inputs that are from a slightly shifted distribution (eg images of houses in CA instead of KY) or from a very shifted distribution (eg imagenet images) by fitting a density model to the last layer h(x) of an MLP trained with squared error, and using the likelhiood score p(h(x)) as a metric. This is a reasonable idea. However, it is not novel eg Aigrain'19 did essentially the same thing for classification models. (The difference between classification and regression is a trivial change to the loss function, and does not change the fundamental idea.)

In addition to lack of novelty, the experimental methodology is very weak. First, the toy 2d example is too trivial to be informative, since there is essentiallly no overlap between the two distributions of features, p(x) and q(x) - even a density model on input space could detect this. More importantly, the results on the two image datasets are suspect. First, it seems that using the predictive variance sigma(x) as the reliability metric (the "var" method) - which is totally standard approach known as 'heteroskedastic regression'. - works very well in several cases. I suspect when it fails it is due to  implementation problems (eg trying to predict sigma instead of log(sigma)). Also ensembles are known to be very robust to distribtution shift (see eg Ovadia'19), so  I am surprised at their poor performance. Another problem is that the datasets used are not standard, so it is impossible to compare to other papers. Finally, no error bars are reported, so it is hard to know if any of the results are statistically significant. 



J. Aigrain and M. Detyniecki, “Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection,” in ICML Workshop on Uncertainty and Robustness in Deep Learning, 2019 [Online]. Available: http://arxiv.org/abs/1905.09186


Y. Ovadia et al., “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift,” arXiv [stat.ML], 06-Jun-2019 [Online]. Available: http://arxiv.org/abs/1906.02530




review_title: Official Blind Review #4

final_score: rating: 1: Reject
review_content: ===Summary===

The authors propose to perform out of distribution detection for regression models by fitting a generative model in the feature space of the regression model. An input example is deemed to be out-of-distribution if it has low likelihood under this generative model.

===Overall Assessment===

I recommend that the paper is rejected. There are a number of aspects that need to be improved. You should fix these and resubmit to a future conference. 
The paper focuses on the difference between regression and classification tasks and claims that the paper's method addresses an unmet need for OOD for regression. However, both the proposed method and the analysis justifying it are generic enough to be applied to both regression and classification.
The paper handles technical claims far too casually in sec 4 and does not provide sufficient justification that the claims are true.
There are natural baselines, such as using a generative model on the raw input space, that are ignored.

===Comments===

Remark 1 feels to me like it was added for the sake of having more math in the paper, not because it is crucial to the paper's argument. 

You remark at various places that existing methods don't naturally generalize from classification to regression. However, you never fully explain why. Also, your proposed method can be applied out-of-the box to classification problems. Your analysis in sec 4 trivially applies to binary classification tasks, and could be naturally extended to multi-class classification where w is not a vector but a num_classes x num_features matrix. 

The parallel should be between classification and heteroskedastic regression, since there you have a distribution per example.

The logic in "In-distribution features are intrinsically low dimensional" is insufficient

The connection between section 4 and your proposed method is not particularly precise. You also have lots of technical claims in 4 that are unsubstantiated. For example, you write "this new network will likely have less discarded information than the shallower network". What does 'likely' mean? In what sense are you making an actual technical statement? Each of the subsections in sec 4 has similar issues.
"The CNNs are pre-trained on ImageNet (Denget al., 2009) and the last layer is replaced with a linear layer that produces a single output."
Why did you do this? Did you fine tune or just retrain the top layer?

"For these two baselines, the variance of the forward passes is used as a metric for detecting OOD inputs" 
   Can you explain why these are reasonable baselines for OOD? Why no baseline that fits a generative model in input space?

You should cite Ren et al. "Likelihood Ratios for Out-of-Distribution Detection"

review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This work is tackling the problem of doing out-of-distribution detection in regression. The core idea is to fit a simple generative model to hidden activations and use thresholded likelihood to decide whether an input is in or out of distribution. Using a simple generative model for this is motivated by the intrinsic low-dimensionality of hidden activations.

My issue with the narrative of this paper is two-fold. 

First, the intrinsic low-dimensionality of the hidden activation distribution seems entirely problem-dependent. One can train networks with orthonormal layers and therefor flat spectra that work fine and there are also invertible discriminative models that do not discard any information from layer to layer, albeit being very similar to standard ResNets. It might be that this heuristic works well for the problems discussed in the paper, but I don't see any reason for this to hold in general.

Second, Nalisnick et al. don't show that deep generative models are often overconfident (as stated in the paper here). What they show is that likelihood can be higher for datasets with lower entropy than the one the deep generative model has been trained on. They also show that "graying" inputs increase the likelihood and they can even show this analytically for flow models with data-independent determinant of their Jacobian. The GMM approach proposed here is thus a special case of what Nalisnick et al. described and should have the same problem. Reducing the variance of the features should increase the likelihood.

All in all, the presented method relies too much on heuristics that seem specific to the analyzed architectures and datasets and there is no theoretical reason for this to hold in general as far as I can see.

The originality of the paper is also somewhat limited, as generative models for OOD detection are well-studied. Thresholded likelihoods are problematic in many settings, so I am not convinced this approach will work well across the board.

----

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don’t know? In ICLR, 2019.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: The authors present the empirical observation that regression models trained using weight decay tend to produce high-level features that lie on low-dimensional subspaces. They then use this observation to propose an algorithm for detecting out of distribution data by fitting a simple GMM to the features produced during training.

The main contribution of the paper is a way of calculating the "OOD scores" that are reported in the different figures. However, after reading the paper 3 times I'm still unable to find a definition of these scores. I'm guessing it's negative log-likelihood under the fitted GMM, but I'm unsure.

The observation (features on low-dimensional manifolds) is interesting, and the proposed algorithm is potentially useful. However the experiments are quite limited, with only 2 training datasets. The analysis of why/when the algorithm is expected to work and how this depends on the model is shallow. The two main results are for models pre-trained on Imagenet: presumably this has a huge impact on the features that end up being used for OOD detection, but this is not discussed in the paper at all.

Questions to the authors:
- Why focus only on regression? It seems to me that your analysis for the features lying on a low-dimensional subspace should also apply to classification models.
- How do your results depend on the specific models that are used? I can come up with models with very few final features where your method would not work. Do you have any analysis or guidance?
- What do all abbreviations in the tables mean? Please make the captions more informative so the reader does not need to search in the main text.
review_title: Official Blind Review #2

Meta-review: The paper investigates out-of-distribution detection for regression tasks.

The reviewers raised several concerns about novelty of the method relative to existing methods, motivation & theoretical justification and clarity of the presentation  (in particular, the discussion around regression vs classification). 

I encourage the authors to revise the draft based on the reviewers’ feedback and resubmit to a different venue.

Decision: Reject