Title: Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
Venue: NeurIPS 2025
Accepted: yes
Keywords: OOD Generalization Benchmarks
Abstract: Benchmarks for out-of-distribution (OOD) generalization often reveal a strong positive correlation between in-distribution (ID) and OOD accuracy across models, a phenomenon known as “accuracy-on-the-line.” This pattern is commonly interpreted as evidence that spurious correlations—relationships that improve ID but harm OOD performance—are rare in practice. We show that this positive correlation can be an artifact of aggregating heterogeneous OOD examples. Using a simple gradient-based method, OODSelect, we identify semantically coherent OOD subsets where accuracy-on-the-line breaks down. Across widely used distribution-shift benchmarks, OODSelect uncovers subsets—sometimes comprising more than half of the standard OOD set—where higher ID accuracy predicts lower OOD accuracy. These results suggest that aggregate metrics can mask critical failure modes in OOD robustness. We release code and the identified subsets to support further research.


Peer reviews:
comments: SUMMARY:
This paper uses a gradient-based method, OODSelect, to partition benchmarks’ OOD split into semantically coherent subsets where accuracy on the line does not hold. The authors tested the method in six datasets and demonstrated that the method uncovers subsets where higher ID accuracy predicts lower OOD accuracy, with sometimes up to 77% of the usual OOD split. They also provide the identified subsets for five datasets for future research.

STRENGTHS AND WEAKNESSES:
**Strengths**

- The paper is well written, and the OODSelect method is well supported by the theoretical analysis as well as extensive experiments.
- The paper explores selection consistency and coherence within the subset and demonstrates the semantic soundness of each partition. It also investigates the use of LLMs and VLMs to generate semantic concepts in the dataset.
- Model pool size selection, subset size, and other design details are clearly illustrated in the main text or appendix, with rationale and supporting experiments.

**Weaknesses**

- The experiments are only conducted in the image domain, and the semantic explanations can be challenging. But I understand that running additional experiments in other domains (like text or tabular data) during the rebuttal period may be difficult.

QUESTIONS:
- Can the method be applied to non-image domains (e.g., text or tabular)? What modifications would be necessary? In these domains, could the semantic explanations of the OODSelect set be easier?
- Can the authors provide concrete computational costs with dataset size and model pool size for a better understanding of the extensive computational cost in the main paper?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper revisit a well-acknowledged concept of ‘Accuracy on the line’, that is, the in-distribution performance and the out of distribution performance of a model will highly correlate. In this paper, author design a test-time adversarial gradient based optimization, to reveal that that exists a subset of OOD data that is inverse correlated to the ID accuracy. Even the overall correlation is positive, a subset up to 77% of the OOD split can lead to negative correlation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Author design a neat method to reveal that Accuracy on the line could just be an artifact of aggregating heterogeneous OOD examples, where a subset of OOD might break this hypothesis. This is a interesting idea and novel to the community, it also indicate that average metric can be misleading especially in safety-critical application like medical.\n2. The paper conduct experiment on diverse dataset, including medical dataset and conventional classification dataset, shows that this behavior is universial. \n3. This paper also dive deeper into their experiment and provide good discussion. I find many insights interesting, for instance, the OOD select examples are not corresponding to the most misclassified example (which often lead to 0 correlation).\n\nWeaknesses:\n1. This work ask an interesting question of revisiting benchmarking, but not provide sufficient actionable item. For instance, there’s attempt in finding semantic pattern among OODselect data using VLM, which could be a actionable item for future benchmarking design (like debias, etc), but the result and insight is limited. There’s more discussion on Broader Impact section but mostly appeal for future direction. A complete discussion on how can this idea could inspire future algorithm design(especially training time), benchmark design will be great. \n2. The algorithm is computational expensive. Author should include table showing how will the computation scale with number of model & data.\n\nQUESTIONS:\n1. There're previous paper also criticize the universality of accuracy on the line, like [1] and [2].\n2. This work mostly adopt non-VLM model for experimenting the correlation between ID and OOD dataset. However, as pointed out in [2], VLM model behavior differently to non-VLM in ID-OOD correlation. How would the method/conclusion of this work apply to VLM? ( I assume it's applicable on zero shot inference, as if I understand correctly only model prediction is required for OOD subset selection).\n3. Why not use ImageNet and ImageNet-OOD dataset in the experiment (which is more widely adopted by the community), is it due to size of the data?\n4. In table 1, I am not sure I follow some of the data. for Chest Xrays, full OOD size is 129838, OODselect subset is 50000 (70%), but 50000/129838=38.5%? \n\n\n[1] Effective robustness against natural distribution shifts for models with different training data, NeurIPS 2023. [2] LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies, ICML 2024.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes that out-of-distribution (OOD) performance may be on average positively correlated with in-distribution (ID) performance for models of increasing capacity, but there may be a subset of OOD data that is actually negatively correlated with ID performance. To determine if this is the case, a method for splitting an OOD test set to minimize correlation with the ID training set is presented (OODSelect). The method is applied to a number of application-focused datasets with OOD subsets, and a large number of models of varying capacities are trained and evaluated to find that there is indeed a subset for which ID performance does not predict OOD performance. In at least one dataset, the subset that performs worse is associated with certain identifiable subpopulations that are unrelated to the data labels.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\nThe paper is very relevant to practical applications, and also addresses an important issue with evaluating models: namely their generalization properties in real-world applications. The results are also very interesting and relevant - especially to the fairness and bias of models. The experiments cover a wide range of datasets and models.\n\n## Weaknesses\n\nMy main issue with the paper is it is not as well presented as it could be, and there are some omissions and odd details in the methods/results. The method also seems a bit overcomplicated and I would like to see the paper focus more on exploring its findings in greater detail (which are quite interesting). The actual methods, although original in execution, are not entirely original in purpose (see the additional citations in Questions). Having said that, I still think this paper addresses an important problem with insightful results, and will be happy to raise my score if the authors address the points below.\n\nThere is something strange going on in fig 3a and 4: the intercepts for OODSelect are sometimes negative, which shouldn't be possible since the slope is always negative and the accuracies are always positive. Are the regression lines applied to the probit transformed accuracies?\n\nThe proof sketch for Proposition 1 is not entirely convincing, and a counterexample to submodularity would help greatly.\n\nI don't understand how the quantity of models was chosen, because it is the diversity of the models that really matters - e.g. if all of the models have the same performance up to noise, then there would probably be little or no correlation between ID and OOD test accuracies.\nAs an example of why this is an issue: in figure 4d there is a clear line of correlated All/OODSelect accuracies on the right side. One possible explanation is that there is an architecture for which All/OODSelect accuracy are positively correlated, and if one were to only sample models from that architecture, then the OODSelect slope would become positive (just like All), invalidating the findings for fig 4d.\n\n Minor issues:\n- Definition 1 should explain $acc_{P_{ID}}$, etc. Actually, maybe definition 1 should follow the problem setup\n- Problem setup should make clear whether OOD examples are from train or test set\n- notations are somewhat unwieldy: long tilde, long text variables like $acc_{P_{ID}}$, abbreviations like AoTIL (why not just say positive/negative correlation?)\n- Typo in table 1 (missing % sign)\n- Proposition 1 $\\subset$ isn't well defined since $\\bf{s}_i$ is a vector of 0s and 1s\n- Paragraph at line 178 needs a reference to figure 2\n- Appendix is missing from the submission\n\nQUESTIONS:\nWhat is the reason for applying the probit transform, as opposed to e.g. Spearman (rank) correlation?\n\nCould the authors clarify the meaning of the second sentence in the quote below (line 67)?\n> Influence functions [33] may appear suitable at first glance, but they rank training points by leave-one-out influence, rather than partitioning the test/OOD set. Thus, applying influence functions in this context would still require an additional heuristic to define coherent subsets, while also inheriting known fragilities in modern deep networks.\n\nIn CXR 70% of OOD examples are negatively correlated. Where is the strong positive correlation between ID and all OOD examples coming from then? Surely the model must be doing significantly better on the remaining 30% of examples to counteract the OODSelect subset.\n\nParagraph at line 186 (selection consistency and coherence) - this seems intuitively sensible, and if anything, the opposite is surprising. Could the authors comment on how inconsistency can occur when increasing subset size (maybe in relation with proposition 1)?\n\nThe discussion of influence functions as well as the subset discovery method presented seem similar to influence scores computed by Feldman & Zhang (https://proceedings.neurips.cc/paper/2020/hash/1e14bfe2714193e7af5abc64ecbd6b46-Abstract.html). There is also a lot of related work in this area (sometimes called example difficulty scoring, e.g. https://arxiv.org/abs/2401.01867), some of which is relevant to the bias/fairness angle (e.g. https://arxiv.org/abs/2010.03058). Is there a way to leverage the scores of individual examples, such as influence scores from Feldman & Zhang, to do the same OOD selection?\n\nLIMITATIONS:\nI think the main potential limitation is the range of models evaluated, but I cannot say either way whether the authors have adequately addressed this point from the main text alone (without Appendix A). As the authors discuss, the datasets chosen are also limited in the metadata available, and perhaps here the authors may want to examine datasets similar to the chest X-rays datasets, which have labelled subpopulations, in order to determine if the negatively correlated OOD subsets have bias/fairness implications.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the OOD split in the current benchmarks and propose an approach to select subsets of these OOD samples before they are used for evaluating methods. This is useful to address the concerns of spurious correlations (higher ID accuracy reduced OOD accuracy) in machine learning models. In some of the datasets such as Chest Xrays, they extracted around 70% of OOD splits.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n-  Originality: The paper presents an interesting approach to select OOD split from datasets.\n\n-  Quality: The study is well-executed, with a thorough theoretical and empirical analysis to show the validity of the proposed approach. The authors provide clean analysis for each dataset graphically. The authors also provided detailed appendix with proofs, dataset analysis and anonymized codes.\n\n-  Clarity: Definitions of key concepts, such as the objective, proof of non-submodularity, are well-explained. The practical recommendations on each dataset and benchmarks provide clear and actionable insights for both researchers and practitioners on their selection.\n\n-  Significance: the paper is useful for the study of hidden OOD data in the current datasets and this will improve the current distribution shoft benchmark to have better ID-OOD split which aids in the evaluation of generalizability of models.\n\n\nWeaknesses \n\n- Can the selection of OOD samples (OODSelect) as per objective in Equation 2 bring out ID samples (potentially adversarial samples) where the model incorrectly classifies, apart from original OOD samples. How can we differentiate between where the model is going wrong in ID and incorrectly predicted OOD samples? Is the above problem addressed as we use N number of models and OODSelect samples perform poor in large subset of these models?\n\n- Missing baselines: The paper proposes a subset selection of OOD samples from a dataset considering an ID distribution. How about we see some baselines for the same problem. For instance, we plot latent feature distribution of data samples using a pretrained model (e.g. CLIP) and we pick OOD samples based on some distance metric thresholds. There are some discussions on these lines using foundational models in Appendix C. However, it will be nice to ground the numbers as in Table 1.\n\n- Alternate to correlation metric: These simple distance metric (e.g. Euclidean dist) can make the objective submodular and the subset selection problem can be solved using greedy algorithm. This can be a good comparison?\n\nSome related works that could strengthen related works:\n-  Nagarajan et al. Understanding the failure modes of out-of-distribution generalization. ICLR 2021\n-  Lin et al. Spurious Feature Diversification Improves Out-of-distribution Generalization. ICLR 2024\n-  Deng et al. Robust Learning with Progressive Data Expansion Against Spurious Correlation. NeurIPS 2023.\n-  Shi et al. LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies. ICML 2024\n\nQUESTIONS:\n-  Are there any analysis done on the OOD splits that obtained from each dataset? What are the semantic coherence with ID data?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}