Paper:
Neural network out-of-distribution (OOD) detection aims to identify when a model is unable to generalize to new inputs, either due to covariate shift or anomalous data. Most existing OOD methods only apply to classification tasks, as they assume a discrete set of possible predictions. In this paper, we propose a method for neural network OOD detection that can be applied to regression problems. We demonstrate that the hidden features for in-distribution data can be described by a highly concentrated, low dimensional distribution. Therefore, we can model these in-distribution features with an extremely simple generative model, such as a Gaussian mixture model (GMM) with 4 or fewer components. We demonstrate on several real-world benchmark data sets that GMM-based feature detection achieves state-of-the-art OOD detection results on several regression tasks. Moreover, this approach is simple to implement and computationally efficient.

Review:
===Summary===
The authors propose to perform out of distribution detection for regression models by fitting a generative model in the feature space of the regression model. An input example is deemed to be out-of-distribution if it has low likelihood under this generative model.
===Overall Assessment===
I recommend that the paper is rejected. There are a number of aspects that need to be improved. You should fix these and resubmit to a future conference. 
The paper focuses on the difference between regression and classification tasks and claims that the paper's method addresses an unmet need for OOD for regression. However, both the proposed method and the analysis justifying it are generic enough to be applied to both regression and classification.
The paper handles technical claims far too casually in sec 4 and does not provide sufficient justification that the claims are true.
There are natural baselines, such as using a generative model on the raw input space, that are ignored.
===Comments===
Remark 1 feels to me like it was added for the sake of having more math in the paper, not because it is crucial to the paper's argument. 
You remark at various places that existing methods don't naturally generalize from classification to regression. However, you never fully explain why. Also, your proposed method can be applied out-of-the box to classification problems. Your analysis in sec 4 trivially applies to binary classification tasks, and could be naturally extended to multi-class classification where w is not a vector but a num_classes x num_features matrix. 
The parallel should be between classification and heteroskedastic regression, since there you have a distribution per example.
The logic in "In-distribution features are intrinsically low dimensional" is insufficient
The connection between section 4 and your proposed method is not particularly precise. You also have lots of technical claims in 4 that are unsubstantiated. For example, you write "this new network will likely have less discarded information than the shallower network". What does 'likely' mean? In what sense are you making an actual technical statement? Each of the subsections in sec 4 has similar issues.
"The CNNs are pre-trained on ImageNet (Denget al., 2009) and the last layer is replaced with a linear layer that produces a single output."
Why did you do this? Did you fine tune or just retrain the top layer?
"For these two baselines, the variance of the forward passes is used as a metric for detecting OOD inputs" 
   Can you explain why these are reasonable baselines for OOD? Why no baseline that fits a generative model in input space?
You should cite Ren et al. "Likelihood Ratios for Out-of-Distribution Detection"