Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: - The paper considers two problems: unsupervised accuracy estimation and error detection, both in out-of-distribution settings.
- The authors propose a practical approach, where they train a ‘check model’ h using self-training and ensembling. Points on which the prediction model and the check model disagree on are predicted as misclassified.
- The theoretical analysis identifies conditions under which the approach should work and presents a bound on the accuracy estimation error
- The authors empirically show that their algorithm outperforms baselines for both unsupervised accuracy estimation and error detection
main_review: - Overall, the proposed approach is well-motivated, supported by theoretical analysis, and has thorough experiments. My main comments are about potential areas of improvements in the experiments.
--------------

- Empirically, the proposed approach outperforms many existing baselines on multiple datasets. Beyond the benchmark numbers, the paper also includes thorough experiments with additional empirical analysis, ablations, multiple versions of the algorithm, etc. But I think there are potentially a few caveats with the experiments as described below.
- The paper has an ablation experiment, where they study the effect of ensembling by changing the number of models in the ensemble. One potential confounding factor is that the check model may be more accurate with more models in the ensemble, and this may be particularly relevant since better-performing check models are expected to work better. Is ensembling important, even when controlled for check model accuracy/size? 
- The paper compares the proposed approach with multiple baselines, but it seems like many of the baselines do not use unlabeled test examples, whereas the proposed approach does (through self-training). Is this correct? Do you think it would be more appropriate to compare with other methods that use unlabeled test examples? For example, https://arxiv.org/abs/2012.05825 also considers detection of misclassified points in OOD/transductive setting, using an ensembling technique.
Another ablation I’d be interested in is on the specific self-training procedure proposed. In contrast to the standard self-training methods, the proposed approach pseudolabels only a specific subset of the dataset. How does this compare with standard pseudolabeling + ensembling? 
- The paper says the approach works when the ensemble check model h is either accurate or diverse on the points misclassified by f. It also mentions that self-training can sometimes improve the accuracy of h, or increase the diversity of the predictions on the points misclassified by f but not yet identified by h. To what extent are the empirical gains explained by the high accuracy of h, as opposed to the diversity in the ensemble? 
- The last few bullet points would give me more confidence about the novelty of the paper. My understanding is that the proposed algorithm is a nice combination/modification of well-known techniques, and the work considers a relatively unexplored setting, outperforms existing baselines, and has theoretical analysis that’s new to my knowledge. However, I think the contributions would be more incremental if it turns out that the high accuracy of h is the main reason why the approach works, and the high accuracy of h is achieved by standard self-training and ensembling; training a check model has been proposed before, and self-training and ensembling are well-known ways to improve model performance.
- In general, how do the prediction accuracies of f and h compare?
--------------

- The paper is clearly written, and I enjoyed reading it.
- Additional discussion on how the three assumptions may or may not hold up in practice could be helpful. The paper describes them as minor assumptions that’s likely to hold in practice, but it’s not clear to me why/if that’s the case for most datasets/models.
- It seems like there is a difference between the analyzed framework and the practical approach, and I wasn’t sure why this was the case. Why does algorithm 1 use majority voting to select R (lines 4 and 5) as opposed to following the procedure in Framework 1 (line 4)?
----------------------
Minor comments
- N seems overloaded between Algorithms 1 and 2
- I found the discussion of DANN-arch confusing. What’s the difference between typical DNN and DANN-arch without the discriminator head, and what’s the rationale? Is the important part the fact that h is trained with DANN in one, but not the other, rather than the architectural differences? 
- How did you select the hyperparameters? The paper describes the chosen values (e.g., “we set T=5,...”), but it’s unclear how they were selected.
- Line 301: Figure 3 → Figure 1

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper considers the problem of unsupervised accuracy estimation and error detection of pre-trained classifiers. The contributions consist of (i) a bound on the generalization performance on estimated accuracy and on detected errors and (ii) a solution based on ensemble models. Specifically, the bound on the generalization error takes into account three quantities, namely (i) the error of the ensemble on correctly-classified test instances, (ii) the agreement of the ensemble and the pre-trained model on potentially wrong test instances and (iii) the diversity of the ensemble on mis-classified test instances. The solution based on ensemble models consists of an iterative training strategy, where at each iteration the ensemble is trained to minimise a training surrogate for quantity (i) combined with quantity (ii) and ensuring diversity among ensemble predictions across iterations. Experimental analysis on several benchmarks (including digits, Office-31, CIFAR-10, iWildCam and Amazon Reviews) show the potential of the proposed solution.

Overall, the paper is well written and well structured. The idea of using ensemble strategies in the context of unsupervised accuracy estimation and error detection is novel. However, there are some issues in terms of ORIGINALITY and QUALITY, which need to be addressed and/or discussed (see below for further comments).
main_review: ORIGINALITY:
The idea of using ensemble methods is interesting and novel in the context of unsupervised accuracy estimation and error detection. However, the work is incremental (extending [1] to the ensemble setting) and it is not clear up to which extent we can consider it novel. Can the authors highlight the main algorithmic and theoretical differences with respect to [1], as there is no discussion in the main article?

QUALITY:
Overall, the paper looks sound (I haven't checked the proof though).
Regarding the theoretical bound (Theorem 1), the assumptions on the three quantities, namely small error rate, low agreement and high diversity over all training iterations of the ensemble, are quite strong. How can these conditions be guaranteed in practice and for contexts where the test domain has small relation with the training one? Further experimental analysis on benchmarks other than digits (currently shown in the Supplementary) would provide a better support to the theory.

Regarding the experiments, most of the claims about the superiority of the proposed strategy over competitors are overly and improperly stated. In fact, note that in Table 1, the work of [1] has high variance in almost all cases. Are the performance of your solution statistically significant compared to the ones of [1]?
Furthermore, how are the hyperparameters of the proposed solution (i.e. T, gamma, alpha, N) chosen?

CLARITY:
The paper is well written and well structured.

SIGNIFICANCE:
The problem is significant and relevant for future deployment of deep learning models to real-world cases. However, the significance of the proposed solution is low, as it is not clear up to which extent the theoretical findings and the experimental observations can provide additional knowledge to the understanding of the problem.

[1] Chuang et al. Estimating Generalization under Distribution Shifts via Domain-Invariant Representations. ICML 2020

limitations_and_societal_impact: There is no paragraph/section addressing the societal impact. Is there a specific reason for that? As far as I understand, the considered problem has a security impact, as being fundamental for the deployment of deep learning models to real-world scenarios.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: 
The paper proposes a framework to tackle two tasks: 1. unsupervised accuracy estimation (estimate accuracy on unlabeled data) and 2. error detection (identify the misclassified examples in the unlabeled data). The framework uses an ensemble of networks to identify mis-classified points as points where the current classifier disagrees with the ensemble. Then the mis classified points are used for self-training to improve the ensemble. The paper gives a theoretical analysis which depends on assuming that the ensemble is mostly correct where the classifer is correct, the classifier and the ensemble disagree on pseudolabeled data and that the ensemble is diverse in its predictions, which is claimed to be true empirically. 
main_review: Quality

The framework is simple and makes good sense, the experiments are reasonably extensive, and there is some theoretical understanding, although it depends on strong conditions on the ensemble. As long as the ensemble obeys good properties on possibly OOD data, the framework seems to make sense, but there seems to be an art to this. For example, the conceptual understanding doesn't address the distribution shift between training and unlabeled examples, which is perhaps why Algorithm 3 (RM) incorporates a domain adaptation method and in general surpasses Algorithm 2 (random init ensemble) by a large margin consistently. 


- Self-training seems to improve the performance a lot, as in Figure 2. If self-training is also combined with Proxy Risk, does the ensembling method still improve over it on the 5 datasets? This would make clearer the role of ensembling vs self-training.
- Although the theoretical assumptions are somewhat strong, the paper does show empirically that the values in the conditions are favorable on the digit datasets. What about in the more complex datasets?
- In general, the ablations of ensembling and self-training parameters are only on Digits - do the same trends hold for other datasets?
- When do the conditions hold? For example, what if the pre-trained classifier itself is a majority vote over an ensemble? 
- There are some observations that aren't reported with numbers, such as in L147. 


Significance

Unsupervised error detection and estimation are interesting tasks that, if we can make progress on them, it would help in model development for domain adaptation/OOD generalization, as well as a score for model selection, and as a tool in algorithmic frameworks for improving the model such as the one presented in this paper. 


Originality
The main novel idea is to use ensembles + self-training in a similar framework to the Proxy Risk paper. The use of domain invariant representations, a check model, and disagreement is in similar spirit to Proxy Risk. 
- Neural network ensembles have been used for uncertainty quantification / OOD detection, but the error detection task here is slightly different as it takes into account the performance of some classifier, and the goal is to detect examples with errors.
- The ensemble method in Algorithm 3 is reminicient of other domain adaptation + semi-supervised learning methods, for example DIRT-T. 

Clarity

The paper is mostly clearly written.
- the definitions of the various sets R_X / W_X / B_X / G_X etc are scattered throughout and hard to remember. 
- In Figure 1, the color/shapes change from the legend. I think this is denoting new train/unlabeled pairs, but this is pretty subtle.
limitations_and_societal_impact: The authors note that some prior knowledge is needed to achieve the best performance, and this will influence the design of the ensemble method, which remains an art. Social impact is not discussed.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper studies the problem of error detection and accuracy estimation of unlabeled data points. Particularly, the paper proposes to use a diverse ensemble of models and detect errors by checking the inconsistency between the ensemble prediction and the target model prediction. Moreover, the paper proposes a self-training procedure that iteratively identifies incorrect predictions and adds their pseudo labels as new training targets to detect more errors. The paper also gives some theoretical analysis given assumptions about the ensemble models' quality.
main_review: The proposed method matches my intuition about an ensemble of deep learning models. The empirical performance also looks very promising. The presentation of the paper is also very clear.

It may be worth investigating specific wrongly predicted data points and compare their true labels with their different predictions from f, models in the ensemble and across different self-training stages. 

For the empirical verification about the theoretical analysis. I think the authors should also provide the final bound calculated using the estimated parameters.

Minor comments:
I don't quite agree to use "linear neural networks" to refer to the layers with fully connected network structures. I think it's more appropriate to call them fully connected layers as there are ReLU non-linearities. The linear neural network should refer to the networks without non-linearities.

## Post Rebuttal Update
Thanks to the authors for answering my questions in the rebuttal. After reading the rebuttal and discussing it with other reviewers, I decide not to change my rating of the paper. The major concern is that the approach is quite related to the Proxy Risk work. I suggest the authors give more analysis about the similarities and differences between the two works in the next version of the paper.
limitations_and_societal_impact: They are properly addressed.
review_title: null

Meta-review: The paper proposed a novel approach that leverages ensembles and self-training for unsupervised accuracy estimation and error detection. All reviewers find the problem setup interesting and the paper well written, with theoretical justification (although relying on strong assumptions) and reasonable empirical support. There are some useful suggestions during the discussion phase, in particular for improving the experimental results.

After a few rounds of interaction during the discussion phase, the committee reached a consensus on the technical contributions: Reviewers agree upon the technical novelty as using ensembles and self-training for improving model disagreement -- a subtle but different approach to the existing work of ProxyRisk, which warrants the novelty of this work; it is worth mentioning that reviewers also note that the use of domain invariant representations, a check model, and disagreement are in a similar spirit to prior work.

An initial disagreement among the committee was on the scope of the experiments: whether a modified version of ProxyRisk with an ensemble is necessary to be included. Although the modification suggested in the discussion phase (by Reviewer 6eDi) deviates from the ProxyRisk algorithm proposed in the original work, the committee agree that this could be viewed as an additional ablation of the role of ensembling vs self-training, which would have made the work stronger. The authors are encouraged to take into consideration of such feedback when preparing a revision.
Decision: Accept (Poster)