Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper introduces a new loss function for training deep neural networks, which show good performance with respect to well-calibrated, trustworthy probabilities for samples after a domain shift. The authors conduct experiments with multiple datasets and multiple forms of perturbations, where the proposed method achieve superior performance. 

I have the following major concerns with the paper:

1. Presentation: The paper's presentation is very weak. It contains repetitive, long, convoluted statements and paragraphs all throughout making it difficult for the reader to understand anything. The first time I read this paper, I couldn't process what is happening. The introduction is more like realted work with less focus on what they are trying to pitch in the paper. However, the latter is the less severe concern.

2. While I appreciate the use of reliability diagrams and the loss function inspired from it, I do not completely understand what are the entities in equations 1 and 2 or how they are used later. Please give them a name, and elaborate on them.

3. While I was reading the paragraph before section 2.2 the first time, it seems to me that ECE is defined for those 10 pertubations specifically. I believe this is not the case; hence, the authors should make it more general and divert the specific details to experiments. 

4. I am not sure what the authors mean by, "... while the loss surface remains largely unchanged", in paragraph above section 2.2.2. I believe the authors have constructed a new loss function by adding a new term to it. That makes it difficult to understand the advantage the authors are talking about after this statement. Overall, I am not really convinced how in contrast to Bayesian Deep Learning, their approach can be EASILY applied to LSTM's and GRU's.

5. What is L_{adv}? There is no equation number, no discussion around where this is defined. Again a presentation issue.

Minor: Please clarify the reference for supplementary materials. For example, it is not clear that Table S1 is in supplementary material.

Overall, this paper is good and has an interesting idea. The experiments are also extensive useful. However, I have reservations regarding the presentation of this paper at this moment. 

--- After Rebuttal  ---

I thank the authors for providing a detailed response to my questions and editing the paper. I am now more positive about the paper; however, I still feel the presentation of the paper could be further improved. At this point, I will not move to acceptance range and keep the same score. 
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The paper presents a method for calibrating neural networks on in- and out-of-distribution data using two additional loss terms: entropy-encouraging loss term which maximizes softmax probabilities for wrong classes and adversarial calibration loss term which pushes confidences to match accuracy on adversarial examples. The idea of using adversarial calibration training is interesting and promising, however, the clarity of the paper needs significant improvement and there are several issues which need to be addressed; for this reason, I recommend a weak reject for the current version.

1. One of the contributions listed in the paper is that authors “illustrate the limitations of entropy as measure for trustworthy predictions and introduce a new metric to quantify technical trustworthiness based on the concept of calibration”; the section 2.1 discusses this in detail. The proposed metric is expected calibration error (ECE) averaged over different levels of noise perturbations applied to data during test time. This metric assumes that for a given dataset we know in advance what kind of noise perturbations can be implied to cause out-of-domain/domain-shift scenarios during test time. This assumption most likely doesn’t hold in real-world applications since domain shifts may be of various kinds. Moreover, calibration may not make sense at all for out-of-domain data if test inputs don’t belong to any of the train classes (as opposed to entropy of predictive distribution which still can be computed and is expected to be higher for such inputs). The metric is also dependent on the considered noise level range; the plot of ECE vs noise level can be illustrative and informative while the averaged value of ECE over all noise levels can be misleading (at least, we may want to have some discount factor to account for the fact that with noise level 0 we strongly care about calibration, while for very high noise levels calibration doesn’t give us much information since the useful patterns in data may be corrupted and it may be impossible even for humans to classify such objects, e.g. on Figure 1-top-left for noise level 100, the digit 6 is corrupted so much that we wouldn’t care about calibration for such inputs but just want to maximize the entropy).
The paper also claims “Recent efforts in terms of evaluating predictive uncertainty have focused on entropy as measure for uncertainty-awareness for predictions under domain shift.” In previous work addressing uncertainty in domain shift [1], not only entropy but other various metrics are considered for out-of-domain detection (Brier score, thresholded confidences, etc), these metrics don’t depend on particular perturbations and are very informative.
The introduced metric, ECE for different noise labels, makes sense only in the toy scenarios where we can control the noise level, however, it would still be better to either look at the plots or to use some discounted averaged ECE over different noise levels, but not equal average. So the significance of this contribution (introduction of new metric) is limited.  

2. The clarity of the paper could be significantly improved.

(a) Figure 1: the top left plot suggests that for high noise level “wrong predictions are often made with high confidence” (so the entropy of this distribution is low) while the top right plot shows that on average predictive entropy gradually grows and is high for high noise levels, so the top right plot is probably not a representative example and it might be misleading to claim this is an “often” case.

(b) Could you please clarify what you mean by “after removing non-misleading evidence” in section 2.2.1? In the next sentence, the remaining probability is probably distribution uniformly across C-1, not C, classes. The predictive entropy loss term essentially maximizes the probabilities of wrong classes with a lower coefficient. How would you support the claim that the loss surface is unchanged?
The loss term is “parameter-free” but still we need to tune a coefficient lambda_S for it.

(c) The adversarial loss equation in section 2.2.2 is written as L2-norm of a scalar value, why is L2-norm needed? The acc(B_m) is not differentiable so it is probably just considered as constant in the paper? Please comment on that. 

(d) Section 3.2 and Figure 4. On Figure 4 Middle and Right plots have different colors than those listed on legend, please, fix this. On the middle plot, both FALCON and EDL have high ECE for noise levels <50 which indicates that they are both highly underconfident for in-distribution data and low noise levels, while other methods have close to 0 ECE on low noise levels. The authors only comment on EDL underconfidence: “it is worth noting that EDL has a substantially higher ECE for in-domain predictions, reflecting under-confident predictions on the test set”, and not on FALCON underconfidence. However, using the proposed score, ECE averaged over all noise levels (Table 2), it may look like the method is doing a good job, while underconfidence problem is revealed when looking at Figure 4. This is also an illustration of the concern about the proposed metric I raised in point 1.

3.  It would help to have an ablation study in the main text of the paper showing the significance of each loss term: in the appendix Figure S1, it is shown that adding adversarial loss to standard and entropy loss helps. Is the entropy loss needed at all? How does performance change if we only have standard and adversarial loss? How sensitive is performance to the choice of lambda_avd and lambda_s?


[1] Ovadia, Yaniv, et al. "Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift." arXiv preprint arXiv:1906.02530 (2019).
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposed FALCON, a simple method to produce well-calibrated uncertainty estimation. The idea is to introduce two additional terms, one that directly encourage lower confidence for all negative classes of all data points, and another one that optimizes the ECE for adversarial samples. Experiments show that FALCON outperforms several state-of-the-art methods for calibrating neural network predictions.

Although the first term, L_S, affects only negative predictions, it is still somewhat strange to uniformly operate on all data points. It would help to perform an ablation study to see how L_S affect the results.

Some description in Section 2 could use more mathematical rigor (e.g., when describing L_{adv}).

The authors use EDL as a baseline, I was wondering why not use the more commonly used, and possibly more effective, temperature scaling (TS) method from Guo et al. 2017. Note that the EDL paper does not seem to explicitly compare EDL and TS. 

Figure 4 (middle and right) is confusing. The line style is not consistent in the figures.

There are a few places where the text is rather vague and confusing. For example, what do you mean by ‘non-misleading evidence’ when describing L_{adv}? It would also be better to provide more insight more L_S to help the readers out. For example, it would help to state that L_S operate only on negative predictions.

Most baselines are rather simple non-probabilistic (non-Bayesian) methods. Besides, MNF, It would also be interesting to see how FALCON compare other probabilistic NN method such as natural parameter networks, where they also explicitly evaluated uncertainty estimation.

review_title: Official Blind Review #1

Meta-review: This paper proposes an algorithm to produce well-calibrated uncertainty estimates. The work accomplishes this by introducing two loss terms: entropy-encouraging loss and an adversarial calibration loss to encourage predictive smoothness in response to adversarial input perturbations. 

All reviewers recommended weak reject for this work with a major issue being the presentation of the work. Each reviewer provided specific examples of areas in which the paper text, figures, equations etc were unclear or missing details. Though the authors have put significant effort into responding to the specific reviewer mentions, the reviewers have determined that the manuscript would benefit from further revision for clarity. 

Therefore, we do not recommend acceptance of this work at this time and instead encourage the authors to further iterate on the manuscript and consider resubmission to a future venue. 

Decision: Reject