Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a simple yet effective technique for estimating a classifier's accuracy on test data which exhibits distribution shift from the training data. Given a score function for the model and a data point, the technique finds a threshold such that such that the relative proportions of validation data above and below the threshold are the same as correctly vs incorrectly classified points, respectively. This threshold is then used to predict the model's accuracy on test data, and the paper demonstrates that this predictor is significantly more accurate than prior methods on a range of distribution shift benchmarks. There are also some theoretical and empirical results on a toy problem that seem to indicate that the proposed technique is well-suited for at least this problem.
main_review: Strengths
---
I am not an expert in this particular area, but to the best of my knowledge, the proposed technique is novel. Moreover, it is demonstrably more effective than prior methods and particularly simple to implement and use.

The paper is of relatively high quality. An extensive number of experiments are carried out on different distribution shift benchmarks, and it seems like the proper prior methods are compared to, though again, I am not an expert.

The paper is well-written and well-structured. This is aided in particular by the fact that the proposed technique is so simple to begin with.

Finally, the paper is likely to be of practical interest for researchers and practitioners studying and dealing with distribution shift.


Weaknesses
---
There are still a few substantive ways to improve the quality of the paper. First, it seems that neither of the theoretical sections (3 and 6) tie in well with the rest of the paper. In my view, the paper is primarily an empirical contribution. Section 3 ostensibly says nothing specific about the proposed technique, it just presents some general results that seem relatively obvious (though I am unsure whether they have been explicitly stated in prior work). But does the theory here motivate the proposed technique in any way? This is unclear to me. Section 6 is more closely related to the empirics, but this connection could still be improved. What are the key insights we can take away from the toy problem? How does it allow us to better understand for which real world problems the technique may succeed or fail? In my view, the paper could do a better job at addressing these questions.

Regarding empirics, the paper is significantly stronger here, but there is still some room for improvement. Most concretely, evaluations on the other WILDS datasets could be quite interesting. This is not (just) a generic "add more experiments" comment: all of the current experiments focus on natural (with the exception of some rendition and sketch shifts) object-centric images, and WILDS contains other modalities such as medical and satellite images, which makes for a rather interesting additional testbed. WILDS also contains language tasks, which would be separately interesting to evaluate on, though I acknowledge this may take more time to set up.

Regarding clarity, there are some minor ways to improve the paper. Section 4 could be expanded to provide more intuition and formal arguments for why the proposed technique makes sense and is justified, ideally from first principles. In fact, I may go as far as to say that this type of exposition would be more useful than Section 3 entirely, to the concerns I have noted above. Figures 2 and 3 were confusing to parse at first -- I think the lines are regression fits, but this should be made explicit if it currently isn't. Table 1 is a bit confusing in the first two columns because the datasets are listed but not necessarily the test sets. Finally, some of the comparisons such as IM should be explained in more detail.
summary_of_the_review: Primarily due to my concerns above, I am initially recommending a weak accept of this paper. I am happy to engage in discussion with the authors and other reviewers in order to reach a more confident final recommendation.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work proposes a simple method, Average Thresholded Confidence (ATC), to predict the OOD accuracy of a classifier based on a labeled source distribution and an unlabeled target distribution. First, the authors show a theoretical lower bound that such task is impossible without assumption on the nature of the shift. Then, the authors proposes ATC. Experiments show that ATC empirically performs better than baselines on different datasets and models. Theoretically, they show that ATC is a consistent estimator for a toy setting with spurious features.
main_review: Strengths:
1. The experiments are thorough, conducted on a variety of types of shifts and models.

Weaknesses:
1. The proposed method lacks theoretical rigor. The threshold for source and target would differ based on different types of shifts, and there's no reason why the source threshold should be used to predict target accuracy. The fact that it works empirically is not a good justification.
2. No theoretical analysis are are done on NE vs. MC. When should we use one over the other, and why?
3. Section 3 doesn't fit with the rest of the paper. Since there's no general-purpose estimator, why does this paper still propose such an estimator? The impossible results are not surprising. It is perhaps more meaningful to study principled estimators under specific settings, just like the covariate shift or label shift settings.
4. I wish to see the theoretical analysis on the toy setting generalized. What are the necessary and sufficient conditions for ATC to work?
5. I wish the authors compare with other calibration methods beyond temperature scaling.
summary_of_the_review: The paper is strong empirically but lacks theoretical rigor.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a new scoring mechanism, Average Thresholded Confidence (ATC), to estimate out-of-distribution performance (i.e. accuracy) of a trained classifier using only labelled data from the source distribution (train distribution) and unlabelled data from the target distribution (test distribution). 

The threshold is based on the logit output of a classifier and can thus be used on unlabelled data.

main_review: Overall, I find the paper quite interesting and intuitive. The mechanism is simple and seems to be effective compared to prior work. Given that this is an important real-world problem, I think the paper deserves to be accepted at ICLR. 

However, I am not a 100% familiar with the related work in order to judge the novelty of the paper. 

## Strengths

1. an appropriate amount of experiments
2. interesting discussion of related work
3. relevant theoretical results to motivate the problem

## Weaknesses and other Remarks

1. **Derivation and/or Interpretation of ATC**:
The authors did a decent job of providing some related theoretical results in the beginning. However, when the authors continue to introduce ATC this seems fairly ad-hoc without any kind of deeper explanation. I think this would merit a longer discussion.

2. **Proper contextualization of theoretical results**:
As I said before, I appreciate the theoretical results as motivating examples for considering empirical scores. However, I don't think they can be considered actual strong contributions of the paper. The main contributions are a heuristic-based score and accompanying experimental results showing the effectiveness. So please make sure that this is reflected in the writing. 

3. **Clarification around some experimental settings**: 
   a. Could you provide some confidence intervals around the scatter plot in Figure 2 and related figures, e.g., using bootstrapping. Why are there so many more sampled points around higher accuracies? 
   b. Could you add more discussion around temperature scaling (cf. _For all methods, we implement post-hoc calibration on validation source data with Temperature Scaling_?) Why is this relevant and what's the intuition? If such an approach improves your performance, could also try improving your performance using more sophisticated calibration methods? That would be interesting to see. 


## Ways to Improve My Score

Please address the weaknesses and I have listed above. I would be happy to engage in discussions during the rebuttal period.

--- 


## Update after rebuttal

The authors addressed my concerns in their responses. Consequently, I am raising my score from 6 to 8. My confidence score will remain at 3 given my missing knowledge about some of the prior work. 
summary_of_the_review: The paper is well-written and I enjoyed reading it. I cannot 100% verify the novelty of the paper since I am not so familiar with the related work but given what the authors wrote in their related work section and their results it seems novel. 

Overall, I didn't have any major complaints about the paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper introduces Average Thresholded Confidence (ATC), a method that aims to predict a model's accuracy on an unlabeled test distribution that varies from the original training distribution by some form of distribution shift. 
The paper first gives a theoretical analysis that shows that, in general, it is impossible to estimate test accuracy without assumptions on the particular distribution shift and that each method that tries to solve this problem has some underlying assumption on the distribution shift.

Their method ATC selects a threshold t on a scoring function (max confidence, or negative entropy) based on the error in the original source domain. The estimated error is then equal to the number of points in the test set that have a score lower than t. Importantly, this is done without using any label information from the test set. 

In their experiments, they demonstrate improvements over various baseline methods on a large selection of datasets and distribution shifts. 
Finally, they have a toy example with very strong assumptions on which ATC-MC is a consistent accuracy estimator. 
main_review: -The proposed method is simple and easy to understand. I also think that their setting where no labeled data from the test distribution is available is highly relevant.

-Their evaluation contains a large number of datasets and distribution shifts and clearly demonstrates that ATC is superior to baseline methods. The only thing that is not quite clear is which scoring function should be used in practice, as there exist certain configurations where ATC-MC and ATC-NE perform the best. 
They also compare to a regression model that is trained with labeled data and are able to achieve similar performance without this extra information. 

-Is there an intuitive understanding why temperature rescaling helps? While temperature rescaling preserves the classification of the model, it can vary the ordering of the scores on different samples, however I would not have expected such large changes, especially in the order of ATC-NE pre and post T. Are there specific logit configurations that clearly show how temperature rescaling helps? 

-I liked the initial theoretical discussion of the problem. The authors state that "every method of estimating accuracy on target data is tied up with some assumption on the nature of the shift", is it possible to elaborate on the assumptions of ATC in this context? 


summary_of_the_review: The paper offers some good theoretical insight into the general problem and introduces a simple method that clearly outperforms the baselines. The problem statement is interesting and very relevant to the community. The evaluation is extensive and clearly demonstrates the strengths of this paper. While the theory does not cover all aspects (eg temperature rescaling), I liked the paper overall.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies how to predict the target domain accuracy using only labeled source data and unlabeled target data. They propose Average Thresholded Confidence (ATC) to learn a threshold on the model's confidence. In particular, ATC predicts accuracy as the fraction of unlabeled examples for which model confidence exceeds the threshold. Extensive experiments have been conducted to show that ATC outperforms prior work across multiple model architectures, types of distribution shifts, and datasets. Theoretically, they prove that identifying the accuracy is as hard as identifying the optimal predictor, and the efficacy of any method depends on the assumptions on the nature of the shift.
main_review: The problem of predicting accuracy of the models is interesting and novel to me. How to identify the accuracy of a model under distribution shift is a practical challenge. The proposed method of ATC is extremely simple and makes intuitive sense. It is also reasonable to claim that identifying the accuracy is as hard as identifying the optimal predictor, and the efficacy of any method depends on the assumptions on the nature of the shift. 

Quality: The submission is technically sound. The claims in the contribution are well-supported by theoretical analyses and empirically results. It is a complete piece of work that outperforms prior work empirically.

Clarity: This paper is well-written and easy to follow. The problem is also well-motivated. The experimental details are also very specific, such that reproducing the results should be possible. 
summary_of_the_review: This paper proposes a novel method, Average Thresholded Confidence (ATC) to learn a threshold on the model's confidence. It is well-written and well-motivated. The proposed idea is simple and technically sound. The claims are well-supported by theoretical analyses and extensive experimental results.
review_title: null

Meta-review: The authors propose a simple method to estimate the accuracy of a classifier on an unlabeled dataset given an in-distribution validation set. In extensive experiments the authors show that the proposed method is significantly more accurate than previous methods and other baselines. 

The reviewers are quite consistent in their judgement, just the weighting of the different aspects is different.
After the rebuttal four out of five reviewers recommend acceptance.

Strong points:
- simplicity of the method
- strong experimental results for various tasks and domain shift problems

Weak points:
- there is no clear theoretical statement when the method is supposed to work
- the discussion in Section 3.1 is pretty obvious and seems a bit like a waste of space whereas the motivation for the actual method is very short

While I agree with the reviewers that there is little theoretical justification for the method, the strong experimental results on various datasets, tasks and different domain shifts make this paper interesting for a large audience. Thus this paper is a nice contribution to ICLR and I recommend acceptance.
However, I strongly recommend to the authors to add more motivation in Section 4 and add a limitation section where the cases are discussed where the method is definitely not working. Section 3 is pretty obvious and could be significantly shortened or integrated into the limitations section. 

One case which is highly relevant for this limitations section is the provable asymptotic overconfidence of neural networks which is discussed in

Hein et al, Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem, CVPR 2019

This would definitely lead to a failure of the presented method as all predictions would get a score above the threshold. I would also assume that the method would predict high accuracy values for out-of-distribution tasks which are semantically similar e.g. training on CIFAR10 and then using CIFAR100 as unlabeled dataset. In that context it would be interesting to evaluate OOD-aware classifiers using ATC such as discussed in

Hendrycks et al, Deep Anomaly Detection with Outlier Exposure, ICLR 2019

Also it would be helpful to understand better the influence of the classifier performance on the original task on the performance of ATC on unlabeled data.
Decision: Accept (Poster)