Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper studies the problem of providing calibrated predictions for out-of-distribution data. They propose algorithms for both calibrating predictions given a single image from the unknown distribution as well as given multiple images from the unknown distribution. They propose an algorithm that estimates which “calibration distribution” the novel image came from, and then use calibrated predictions for this distribution. They evaluate their approach on a standard image datasets including CIFAR-10 and ImageNet, and show that their approach outperforms existing work.

Pros
- Important problem

Cons
- The approach claims to work on out-of-distribution data, but assumes the possible novel distributions are known
- Missing related work

The approach proposed by the authors is fundamentally flawed: while they do not directly assume to know which “unknown distribution” the novel image is from, they assume it is from one of a small set of possibilities. This information is not assumed in existing work, and fundamentally alters the problem, making it simple to address and uninteresting.

The proposed approach is also very simplistic, which is not a flaw in and of itself but is a consequence of the extra knowledge they assume. In particular, given this extra information, the authors simply predict which shifted distribution the novel example is from, and then use the calibrated prediction for that distribution.

In practice, this problem is important for handling unanticipated distribution shifts in production. If the distribution shift is known and anticipated, then a much more natural approach would be to simply use data augmentation to generate data from the shifted distribution and train the model on this extra data.

In addition, there recent work in this area that the authors do not cite, for instance:

Park et al., Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptation. In AISTATS 2020.

Wang et al., Transferable Calibration with Lower Bias and Variance in Domain Adaptation. In NeurIPS 2020.

-------------------------------------------------------------------------------------------------------------------------------

Post rebuttal: I have updated my score based on the clarification provided by the authors. My remaining concern is that I still think the baselines considered by the authors is incomplete. In particular, the calibration under distribution shift techniques can still be applied, just using either just a single test image or their set of multiple test images. Admittedly, this approach would probably not perform well for a single image, but in Table 5, it seems like oftentimes multiple images are needed to even beat Ovadia et al. (2019).
review_title: Clearly flawed approach relies on extra information

final_score: rating: 6: Marginally above acceptance threshold
review_content: The submission proposes a very simple and seemingly effective method for improving uncertainty estimates of predictions for corrupted data. The main idea is to calibrate predictive confidences assuming access to an “exposure” set of corruptions. In particular, the paper uses contrast-corrupted data to calibrate predictive confidences and shows improvements for the types of corruptions discussed in [1] (leaving out contrast). 

(+) The improvements seem to be fairly consistent, and the method is simple and intuitive. It is interesting to know that such improvements can be had, i.e. calibration on one type of corruption is transferrable to other types to some extent.

(-) While the 'single image method’ is more or less intuitive, the motivation for the ‘multiple image method' is less clear. For example, why not compare the distribution of p_maxes in S^deploy to the distribution of p_maxes in each p^CAL? Or, why not do a similar weighted averaging as in the single image method, with divergences between these two distributions providing the weights?

(-) While this paper mostly builds on prior work, it would be interesting to see calibration advantages on more than just the artificially corrupted sets. The prior work [2] that this submission builds upon reports results across a fairly wide range of tasks and out-of-distribution types. It would be more interesting to see if such calibrations with a specific set can be relevant for more realistic OOD cases as well, as in [2].

The sentence "Heuristically we always want clean images in our calibration set while having different shifted means” is not clear, could the authors elaborate? Why might we prefer such a calibration set? This might be an important point for the reader, since it motivates the particular choice of contrast-corruption.

An obvious baseline would be to perform the same recalibration procedure, but with the validation set (i.e. non-corrupt data), to figure out if corrupt sets in particular are required for calibration. I suspect they are, since performance at such sets are likely to be poorer, which would allow for more calibration room over a larger “error”-space.  

Overall I think this paper could be interesting in that it informs us of the possibility of transferring recalibration from a corruption-exposure procedure, which to my knowledge is a novel reporting. More experiments as described above would improve the paper, by making a more compelling case for such exposure-based recalibration techniques.

[1] Benchmarking neural network robustness to common corruptions and perturbations, Hendrycks and Dietterich

[2] Can you trust your model’s uncertainty? Ovadia et al.


Post Rebuttal:
Thanks for the response, and the new experiments. I continue to think that this is a nice simple method that works well enough to be interesting. I retain my initial rating.
review_title: Simple method, provides improvements to calibration

final_score: rating: 4: Ok but not good enough - rejection
review_content: In this work, the authors propose a post-hoc calibration method for potentially OOD data that relies on estimation of the "degree" of corruption for new test data. The rely on the benchmark provided in Ovadia et al. as a basis for their analysis. Ovadia et al. assessed common measurements of uncertainty such as Brier score, ECE, and entropy over a variety of datasets including MNIST + translations/rotations, CIFAR-10 and CIFAR-10C, and ImageNet and ImageNet-C for a variety of models such as vanilla neural networks, SVI, ensembles, and Dropout. By using corrupted versions of these common datasets, Ovadia et al. could evaluate how uncertainty estimates vary under dataset shift. In this work, the authors aim to improve the calibration of probabilities obtained from these models. They start by establishing a calibration set where they derive $p_{correct}$ from a sample of $p_{max}$. Then, depending on how many test images they are evaluating, they use a single image or multiple image method to attempt to determine which calibration set (of which there can be many depending on the number of corruption levels considered), the test images are closest to. Then they "correct" the model's probability estimate by weighting over the calibration sets. They show on CIFAR-10 and ImageNet that their method results in lower ECE over varying levels of corruptions. 

Strengths: 
* This method could be applied post-hoc to a variety of models 
* Seems fast to compute 
* Results in better calibration estimates

Weaknesses: 
* The terminology of the paper is ill-defined. For instance, I don't think the authors ever explicitly define $p_{max}$. I understood it by context, but the presentation could be much clearer. 
* When is it clear to use the "Multiple Image Method"? How can one be sure that $\{x_1,...x_m\}$ come from the same distribution? 
* It was not explained why "contrast" was chosen as the calibration set. 
* How can this method be applied when the OOD corruption is very far away from "contrast"? Could you evaluate how your method performs on corruptions such as translations and rotations? 

Ultimately, I don't think that this method was presented in a clear enough fashion or has sufficiently demonstrated an ability to generalize to new types of corruptions and am rating this a 4 because of these reasons. 
review_title: Post-hoc calibration looks promising, but many open questions remain about applications and generalization

final_score: rating: 6: Marginally above acceptance threshold
review_content: Thank you for the additional experiments. Especially, Figure 7 and 8 look promising.
My conclusion from the experiments is that the "contrast" corruption (used for validation) seems to be general enough, in the sense that for many other corruptions, encountered at test time, the performance is good.
However, as AnonReviewer1, I am not sure about why the methodology seems to work well for very different types corruptions at test time, and completely OOD data (Figure 7 and 8). 
More empirical/theoretical analysis would be nice.
Increased rating to 6.

------

Summary:
The paper addresses the important problem of over-confident predictions on out-of distribution (OOD) data. 
Focusing on image classification, they propose to calibrate probabilities on a validation dataset which contains images that were artificially corrupted in various ways (noise, blur, brightness etc.). This is different from traditional post-hoc methods which use an uncorrupted validation data set (iid assumption). This way they can explicitly calibrate probabilities for the non-iid assumption.
Their proposed methods (Single Image and Multiple Image Method) seem to be new, and their experiments suggests that for certain types of corruptions their method is effective (as measured by ECE and shown by calibration diagrams in Figure 2).

Strong points:

- Even when the type of corruption in the validation dataset is different from the corruption in the test set, the proposed method can considerably improve over existing methods.
- The proposed method, similar to other post-hoc calibration methods, can be used in combination with any model. 
- The illustrations, in particular Figure 2, are well done.

Unclear/Weak points:

- The proposed method relies on set of pre-specified set of corruptions that are used to generate the validation data. Therefore, I think it is important to throughly evaluate the  impact when the corruption used for validation is different from the one in the test data.
However, results are only reported when the validation corruption is "Contrast".
Furthermore, it seems the results reported are the ones on the test data averaged over several types of corruption. However, it would be interesting to see for what types of test corruption the method work/not works.

- It would be interesting to see results when the test data is completely OOD data, like using SVHN dataset for testing predictions of a model that was trained on CIFAR-10 (see Ovadia et al, 2019). 

- Some intuition/analysis should be given about the proposed method.
For example, in "Single Image Method" (i) and (ii): 
My understanding is that the authors want to achieve that 
if a test sample is corrupted by type A, then q_i(max) should be close to one for the calibration set which type of corruption is similar to type A.

- At least in the Appendix:
Some more formal description of how Equation (3) is calculated should be given

- The formula for computing $L_j(p_{max}(x))$ in "Single Image Method" (i) should be given.

- The notation is slightly unclear, what does the "m" in $P^{CAL, j}_m$ mean?

- I am not sure what this means:
"on each of the calibration sets determined by combinations of intensity given by: {0}, {5,0}, {5,4,0}, {5,4,3,0}, {5,4,3,2,0}, and {5,4,3,2,1,0}."

- What are sizes of each calibration set in the experiments? How many calibration sets are there?

- I am also not sure about this sentence:
"Heuristically we always want clean images in our calibration set while having different shifted means." Does it mean that you would like to have both clean images and corrupted images in the validation dataset?


Minor points:
- it should probably be $L_i(p_{max}) / ...$ and not $L_j(p_{max}) / ...$  in the equation in paragraph "Single Image Method" (i).

- It might be better to rename $P^{deploy}_m$ as $P^{test}_m$, to confirm to the standard terminology of train/validation/test data.

- in "of p_max under each of the calibration models, using the probability density",
I think it is easier to understand if "calibration models" -> "calibration sets".

review_title: New method for preventing OOD detection in image classification, but unclear when and why method works

Meta-review: This paper presents a method to improve the calibration of neural networks on out-of-distribution (OOD) data.

The authors show that their method can be applied post-hoc to existing methods and that it improves calibration under distribution shift using the benchmark in Ovadia et al. 2019.

However, reviewers felt that the theoretical justification for why this works is unclear (see detailed comments by R1 and R4), and some of the choices are not well-justified. Revising the paper to address these concerns with additional theoretical and/or empirical justifications should improve the clarity and strengthen the paper.

I encourage the authors to revise and resubmit to a different venue.

Decision: Reject