Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper describes the agreement-on-the-line phenomenon: instead of relying on accuracy-on-the-line, the paper shows that it is possible to use the agreement of a pair of models as a proxy for the ID vs OOD performance tradeoff. Agreement-on-the-line can be estimated solely from unlabelled data and can be used to predict potential OOD performance. Empirical results based on CIFAR-10, ImageNet, and WILDS OOD data show good OOD performance predictions.
strengths_and_weaknesses: ## Strengths

- Estimating OOD detection and generalization performance of ML models is an important topic.
- The paper is well written and easy to follow.
- The proposed method of considering the disagreement between multiple models is simple and does not require labeled OOD data access.
- Disagreement ensembling can also be performed over checkpointed models instead of multiple fully trained models.

## Weaknesses

- Although it seems unlikely that a broad set of models would all agree on the same prediction for an OOD data point, this event probably has non-zero probability, especially for a low number of models. This means that the proposed method can probably not guarantee to be a good OOD performance predictor under all circumstance. This difficulty seems to stem from the fact that behavior on in-distribution data is well understood while this is not the case for OOD data.
- Baseline methods for experimental section are not well discussed. It would be helpful to introduce these methods in the related works section.
- Limitations are not well discussed.
questions: - Under what circumstances would we expect an OOD data point to have a widespread label agreement across models? Is it enough for accuracy-on-the-line to hold for this to be true? If so, why?
- What happens for $0.75 < R^2 < 0.95$?
- Agreement (equation 2) is defined pairwise between two models. Would it be more appropriate to directly formulate it as a quantity (potentially) involving more than just two models and thereby bypass estimation of individual $A_{ij}$'s and $b_i$'s?
- Can you provide more details as to why Prop (ii) only holds for neural networks? Is this merely an empirical finding or can you provide a theoretical justification?
limitations: The paper does not appear to explicitly discuss major limitations of the presented work. I was not able to find other limitations explicitly mentioned. It would be great if the authors could comment on the first weakness (and question) listed above.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper identifies and extensively analyzes an interesting phenomena relating the agreement and accuracy of models on in-distribution and out-of-distribution data. In particular, they find that when there is a linear correlation between in-distribution test accuracy and out-of-distribution test accuracy across a set of distinct models trained on this data (a concept discovered in an earlier paper), there is also a linear correlation between the *agreements* of pairs of models trained on that data. This phenomena is identified to be unique to neural networks, and somewhat surprisingly, can be used in the reverse direction as a method of unsupervised estimating out-of-distribution accuracy by measuring agreement - even in cases where in-distribution and out-of-distribution agreement is only roughly linearly correlated. 
strengths_and_weaknesses:    * **Originality**
     While other papers have used agreement to determine how well a model performs on unlabeled data, to the best of my knowledge, this is a completely novel phenomena that has been identified comparing agreement-on-the-line with the recently discovered accuracy-on-the-line. It is clear how this work differs from previous contributions, and the analysis shows new insights that were not described in previous work. 
   * **Quality**
     The claims are mostly well-substantiated, with extensive experimentation supporting the major claims. The methods used are appropriate, and both the strengths and the weaknesses of the work is remarked on. A possible area of improvement could be in the theory that agreement-on-the-line and acccuracy-on-the-line having the same slope and bias for neural networks but not other models - for example, this could be rigorously tested with a hypothesis test to give further evidence to that claim. Another area of improvement could be increased evaluation with respect to the methods for predicting OOD accuracy - for example, the ALine method that only uses a single model would be useful to have in Table 2, especially if it is in the main body of the paper. 
   * **Clarity**
     The paper is very clear, both in its organization as well as its figures. It is clear how to run the experiments and reproduce these results, as well as exactly what the two different algorithms are doing. A minor nitpick would be to remove the citation in the abstract so that it can be read completely stand-alone. 
   * **Significance**
     The paper is significant both from a theoretical and a practitioners point of view. From the theoretical viewpoint, it provides further evidence that some domain shifts have very particular properties that create both agreement and accuracy-on-the-line phenomena. It is very unclear what that phenomena is or how one might be able to predict whether it is occuring, but it clearly has some interesting implications. For the practitioner, this method provides a method for unsupervised prediction of performance on OOD data that is significantly better than other methods (especially when agreement-on-the-line holds), which is a very valuable contribution that I expect will be used. 
questions:    * Is there a statistically significant difference between the slopes and biases of neural networks accuracy-on-the-line vs agreement?
limitations: The paper does a nice job describing its limitations, both in the amount of models that need to be trained as well as its reliance on the accuracy-on-the-line phenomena. The paper could benefit from further discussion on kinds of domain shift where agreement-on-the-line is guaranteed to not hold, e.g. if there were a label shift.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces a finer-grained version of accuracy-on-the-line called agreement-on-the-line, capable of estimating OOD performance without access to OOD labels. Rather than analyzing the correlation between ID and OOD performance metrics like accuracy-on-the-line, the authors analyze the predictions made by pairs of models, and observe that strong agreement between model predictions on ID data correlates with prediction agreement on OOD data. Moreover, agreement-on-the-line can be used as a test for when accuracy-on-the-line is appropriate to use due to the apparent co-occurrence of the two phenomenon. The main advantages of their method over previous works is not having to rely on assumptions about data shift magnitude, and the ability to aggregate information from many pairs of models. also The authors validate their claims empirically on a variety of datasets, and for several model classes.
strengths_and_weaknesses: S1: Agreement between models is a clever surrogate for whether accuracy-on-the-line holds. It is fast and simple to compute.

S2: Results are impressive, and consistent across the several datasets considered, particularly for the Camelyon and iWildCam datasets. 

S3: The ability to use multiple checkpoints for a single model rather than having to train many independent models is very appealing. This was something not investigated by the accuracy-on-the-line paper, so it is a novel and intriguing result.

W1: An analysis of agreement vs accuracy would be welcome. For example, what is the biggest possible discrepancy in agreement for two models M1 and M2 that have the same ID accuracy Acc? This could be used to bound the difference in slope between the agreement and accuracy lines, when accuracy is not computable.

W2: The definitions of weak (R^2 < 0.75) and strong (R^2 > 0.95) correlation seem a bit arbitrary. Such definitions vary throughout the literature, though an R^2 of 0.75 is generally considered to be quite strong.

W3: Some of the claims in section 5 are a bit hand-wavy. For example, "Interestingly, the other benchmarks also did not perform very well on these datasets, suggesting that perhaps the success of these prediction methods could also partially be attributed to accuracy-on-the-line" needs some more justification. 
questions: Related to W1, it's not obvious if Proposition 1 is unique to neural networks, or just highly accurate models. Specifically, if the ID and OOD accuracy are strongly correlated across the set of models considered, and the models are also accurate, how do you think minimum accuracy affects agreement correlation? The lower bound on agreement is Acc(M1) - (1 - Acc(M2)), so just by following this lower bound, agreement correlation would be strong. 

There is a bit of a contradiction in the claim that "In fact, ALine-D supercedes previous methods even when accuracy-on the-line does not hold". Namely, agreement-on-the-line is introduced as a way of determining when accuracy-on-the-line holds, and then leveraging accuracy-on-the-line to predict OOD performance, so if accuracy-on-the-line does not hold, how can it be the case that ALine-D performs so well when the underlying conditions break down?
limitations: There is almost no discussion of limitations, but a couple come to mind. For example, the baselines used in Figure 2 such as linear models and SVMs are not appropriate for CIFAR10 classification. Understandably the authors wanted to see if their results hold for classical ML models, but such models should be evaluated on datasets where they can achieve good performance since conclusions drawn from poorly performing models can be misleading. Also, the experiments section does not investigate why agreement-on-the-line is so effective. It would be nice to see if there is something particular about the optimization procedure used, i.e. the inductive bias of SGD, that enables agreement-on-the-line to work when using model checkpoints. Since the premise of the paper is that the observed phenomenon is peculiar to neural networks, the conclusion should at least discuss what role architecture plays.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper investigates empirically the important and practically very relevant question of OOD performance. The approach taken builds on previous work of 'accuracy-on-the-line' (which observes the occurrence of strong linear (transformed) correlation of ID vs OOD accuracy in some datasets) and adds the simultaneous evaluation of agreement between pairs of models in the OOD vs ID.
On a varied set of ID/OOD Image classification tasks, the work empirically finds that a similar linear (transformed) relationship exists or does not exists jointly for accuracy and agreement. Further the findings show a strong similarity between the slopes and bias of the agreement and accuracy ID vs OOD lines on a significant subset of the studied datasets. Interestingly outside of neural networks this similarity breaks down.
Through these observations the authors address an important practical problem of assessing existing models' performance on OOD data, critically, without the availability of labels for the OOD data. 
The proposed method relies on the observed joint existence (or lack there-off) of accuracy-on-the-line and agreement-on-the-line phenomena --- such that if there is no agreement-on-the-line one can test (verify) it.
Then, by exploiting the similarity of the lines, they show how to estimate the expected OOD accuracy.

The paper proposes both a pair-model based assessment as well as a multi-model based assessment which allows for noise and potential bias reductions (e.g. from the transform mismatch).
Finally and importantly, the paper shows for a specific case that agreement-on-the-line holds also across training. 



strengths_and_weaknesses: $\textbf{Strengths:}$


The paper adds simple, novel and very powerful set of observations:
* The existence (or lack there-off) of strong linear (transformed) correlation of accuracy is empirically found to go hand in hand with a strong linear (transformed) correlation of agreement between ID and OOD.
* When both exist the correlation coefficient are 'similar' (in a large subset of the cases) for neural networks examined.

These observations are interesting both theoretically (for further research) and practically --- for the pertinent question of how one may assess OOD expected accuracy of a given model given OOD, unlabeled, data.

By building on the above observations together the authors demonstrate how this practical question may be addressed:
OOD accuracy feasibility can be verified (if linear correlation is high) and carried out (assuming identical correlation coefficients between accuracy ID vs OOD line and agreement ID vs OOD line hold).

The method is straightforward and clear given the assumptions. 

Throughout the paper is clearly written.

$\textbf{Weaknesses / areas of improvement:}$

1. Most important --- assessment of single model OOD performance:

In practice, it is most often the case that the practical question is for a given model --- what would be its OOD accuracy.
The authors introduce a path to address that (this is a strength of the paper) though the throughout-training agreement vs accuracy experiment, however, the authors only cursorily touch on the same-model OOD performance with a single example.
Since this a very powerful and practical setting, seeing that is generalizes across the datasets is important. 
It is advisable the authors expand this experiment across the datasets and in particular for the imagent/V2 datasets.

2. Generalizability and Implications --- Is agreement-on-the-line a fundamental phenomena, does it hold outside of image classification?
It would increase the confidence in the generalizability of these finding (and their applicability) if demonstrated in the NLP context.
It would be very good to add for example a checkpointed pre-trained transformers (single model across training for simplicity) and evaluate loss and agreement loss for ID vs OOD.
Even if the phenomena does not hold in this setting that would increase the insight provided by this paper for further research.

Adding these items would change this to an even stronger paper

$\underline{Minor:}$

stylistic / typos:

line 112 --- reference 42 duplication

line 231 --- the $\Phi^{-1}$ probate transform is undefined (left for the reader to piece together)

line 233 --- summation index typo, should be $i \neq j$  

line 242 --- should read 'substituting $b=...$ into (5)'

In practice (and in the appendix) the 'almost the same slope and bias' observation seems to hold to various degrees across the data examined and is left unquantified, or only quantified indirectly (e.g. through the MSE of the accuracy estimation in a subset of the cases).
consider adding a systematic quantification in the paper across the datasets (with details in the appendix) in terms of MSE OOD accuracy prediction.


questions: see above
limitations: yes
review_title: null

Meta-review: metareview: This work addresses the “agreement-on-the-line phenomenon” by extensively analyzing a phenomenon relating to the agreement and accuracy of models on in-distribution and out-of-distribution data. In particular, one of the findings is that when there is a linear correlation between in-distribution test accuracy and out-of-distribution test accuracy across a set of distinct models trained on this data, there is also a linear correlation between the agreements of pairs of models trained on that data. Agreement-on-the-line can be estimated solely from unlabelled data and can be used to predict potential OOD performance. The main advantages of their method over previous works are not having to rely on assumptions about data shift magnitude, and the ability to aggregate information from many pairs of models. The paper proposes both a pair-model-based assessment as well as a multi-model-based assessment which allows for noise and potential bias reductions. Empirical results based on CIFAR-10, ImageNet, and WILDS OOD data show good OOD performance predictions.

The paper convinces in all four categories (originality, quality, clarity, and significance), and the reviewers all agree on accepting this work for publication. For the camera-ready version, it would be great if the authors could include a short description of the baseline methods and briefly discuss the reasoning behind choosing the R2 threshold.
confidence: Certain
Decision: Accept