Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Through an empirical study on vision benchmarks and medical images, this paper explores the concepts of spurious and shortcut features and their links to prediction depth and V-information. They show empirically that shortcuts can be detected early in training. They also show a link between prediction depth and V-information.
strength_and_weaknesses: Strength:
 - The idea of using the PD in order to identify potential spurious correlations is new and interesting. The experiments on toy and medical images provide interesting new insights.

Weaknesses:
 - It feels like the statement "not all spurious correlations are shortcuts" is somewhat trivial
 - I was not able to fully grasp the theoretical contribution in section 3. Arguably, this could be because I am missing some background on the subject, but I think that this also means that the presentation of your proposition 1 could be improved, even if still informal.

Other suggestions:
Figure 6: I suggest also looking at epoch 0 or in other words at initialization before any training.
Section 4: "shorcut is harder" why is it a shortcut then ?
clarity,_quality,_novelty_and_reproducibility: Clarity:
The presentation (section 3) and the proof (A.1) of the relationship between prediction depth and V-information should be improved.

Novelty and Quality:
The method of using the PD in order to detect that a trained network is relying on spurious correlations is novel and relevant as far as I know. The experiments on medical images are convincing.
summary_of_the_review: I liked the method of using the PD in order to examine training and identify potential spurious correlations. I think that the paper could be improved by clarifying the theory part.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies the relationship between spurious features, shortcuts and learning dynamics in a novel way. By effectively using the prediction depth (PD, Baldock et al.) as its main tool, the paper concludes that only spurious features that are “easier” than core features (instead of all spurious features) can be shortcuts, and the learning of such shortcuts can be identified early during training via PD and GradCAM visualization. The proposed workflow is validated on both synthetic and real (chest X-ray) datasets and shown effective in spotting the shortcuts. Additionally, the paper theoretically connects PD and usable information (PVI, Ethayarajh et al.) to explain its empirical success.
strength_and_weaknesses: Strengths
+ (Clarity) The paper is well organized and clearly written. Fig 1 (prime number vs white patch) is a great example explaining the paper’s main finding.
+ (Novelty) The paper’s main finding, effective tool usage (PD) and theoretical analysis (connection between PD and PVI) are all novel contributions as far as I know.
+ (Significance) Given the importance of the topic, the novelty of the contributions, and the overall effectiveness of the proposed workflow (although can be better validated, see below), I believe this paper can be a strong, potentially impactful baseline for future researchers & practitioners to use and/or improve upon.

Weaknesses
- (Quality) While the paper’s technical details are all correct as far as I can tell, the empirical validation still can be improved in the following aspects.
1. Although medical imaging indeed is a safety-critical domain and deserves more studies regarding spurious correlations, having only medical (chest X-ray) datasets with real (non-synthetic) shortcuts in the experiments is unfortunately non-ideal. Please consider including other commonly used datasets e.g. CelebA (gender vs hair color), Waterbirds (bird type vs background) [1] and/or NICO (object vs context) [2, 3] to diversify and strengthen the empirical evaluation.
2. Please consider strengthening the claim that only “easier” spurious features can be shortcuts with larger-scale experiments, such as a) randomly combining (as Sec 4.1) more datasets with different PD difficulties or b) creating & using synthetic datasets with controllable shortcut difficulties like e.g. [4], in order to quantify the correlation between feature difficulties and utilization (via e.g. GradCAM analysis).
3. Similarly, it would be a plus to showcase the correlation between PD and PVI in Sec 4.4 with larger-scale experiments to empirically support Proposition 1 more solidly.

[1] Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization, ICLR, 2020.\
[2] Towards non-iid image classification: A dataset and baselines, Pattern Recognition, 2021.\
[3] NICO++: Towards Better Benchmarking for Domain Generalization, https://arxiv.org/abs/2204.08040 \
[4] A Fine-Grained Analysis on Distribution Shift, ICLR, 2022.
clarity,_quality,_novelty_and_reproducibility: Please see above for my evaluation of the clarity, quality and novelty of the paper. While the clarity and novelty are undoubtedly very good, the quality (empirical validation) on the other hand can be improved.

Regarding reproducibility, the submission didn’t include anonymized source code. However, given the overall simplicity of the paper, I believe it won’t be too hard to reproduce its results even without the source code.

Below are some additional questions for the authors.
* The authors conjecture that “easy features learned by the initial layers (…) are potential shortcuts”. If so, will simply freezing the early layers at some known good pretrained weights solve the problem entirely? Or will the shortcuts manifest again in a different way (e.g. dynamics) at later layers?
* Prior work suggests that post hoc explanation methods can be ineffective when the spurious features are unknown to the user at test time (e.g. non-visible artifacts, undertraining, etc.) [5]. Is the proposed workflow also vulnerable to this issue, or does it provide additional guarantees to mitigate or eliminate this issue?

[5] Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation, ICLR, 2022.
summary_of_the_review: The paper in its current form has multiple great strengths (clarity, novelty, significance) but also some unignorable weaknesses (empirical validation). In my opinion, since the paper’s strengths outweigh its weaknesses, I’m learning towards recommending its acceptance to the conference.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper is concerned with clarifying the distinction between shortcut features and spurious correlations.  The authors note that while spurious correlations are rightly acknowledged as a source of shortcuts, and thus for shortcut learning to be identified with distribution shifts, there is a distinct source of shortcuts that is based on the difficulty of shortcut features.  They investigate the learning of shortcut features by interrogating training dynamics and example difficulty, positing that easy features learned by the initial layers of a DNN  early in training are potential shortcuts.  They provide empirical evidence for this hypothesis with synthetic and real medical imaging data, and they show that  prediction depth and $\nu$-usable information are correlated.

In summary, the authors establish that:
1. Shortcut features are spurious correlations, but not all spurious correlations are shortcuts
2. The presence of shortcut features may sometimes be detected by looking at the distribution of prediction depths of a validation set (or test set)
3. The readily computable prediction depth is positively correlated with $\nu$-usable information, a refinement of mutual information that takes computability constraints into account.

strength_and_weaknesses: ### Strengths
- The experiments are well-designed.  Reading sections 4.1 through 4.4, I could not help but think of follow-on problems to examine for each, probing the fidelity of prediction depth as a function of number of parameter updates, as well as of the relative abundance or scarcity of the data.  
- The figures (2-7 certainly) are very clearly designed, and informative
- The direct manner in which the authors lay out their premies in section (1) makes the claims of the paper simple to evaluate in light of the evidence they present. 


### Weaknesses
- The authors do not provide any precise definition for easy tasks versus hard tasks, leaving the reader to intuit why certain tasks are ordinally related in difficulty (e.g the claim in section 4 that MNIST is simpler than Fashion MNIST, which in turn is simpler than KMNIST).  Spending a bit more time to agree on a way to measure degrees of difficulty would help not only this paper, but would help to broaden the applicability of their method to other domains.  Perhaps some measure of generalization?

(section 1) Panel (B) of Figure 1 has no caption text; is this intentional or is it an oversight?  
(section 4.1) A few suggestions for strengthening this experiment:
1. The distribution (or first two moments) over several re-trainings should be reported in table 1.  It's always better to report distributions rather than point estimates, especially when the point estimate is not precisely defined (i.e was only one experiment performed?  Or does table 1 contain the max / min disparity?)
2. The results in Table 1 are conditioned on the difficulty of the binary classification tasks, so the authors should either report the difficulty of each of the pairwise core-feature tasks, or try multiple core-feature versus spurious feature combinations to better characterize the shortcut effect.
clarity,_quality,_novelty_and_reproducibility: The writing is clear and concise, which is appreciated.  While the authors do not provide any precise definition of difficulty, they do establish convincingly that shortcuts are more aligned with 'easily detectable spurious correlations' rather than difficult ones. 

The authors demonstrate their claim quite effectively with a well-constructed hybrid (or "dominios" in the parlance of Kirichenko et al.) datasets that couple an unaltered dataset (Fashion MNSIT) with low-PD shortcuts (MNIST) and high-PD shortcuts (CIFAR10).  I find it particularly compelling not only based on the drastic Table 1 figures, but also that CIFAR10 is not considered a particularly difficult task for a standard ResNet18.

The primary novel contribution here is the realization that PD can be used to interrogate models to detect the presence of undesirable shortcut learning, rather than merely to stratify a dataset by degree of difficulty.   The authors use publicly available data, and standard models for their experiments, rendering the work more reproducible (modulo any code to be released upon decision).

summary_of_the_review: The authors succeed in convincing the reader that spurious correlations are not always shortcuts, though I would argue that they omit connecting their work to the growing literature on spurious correlations.  Perhaps this is intentional, to limit the scope of their claims, but I feel it could be a missed opportunity.  For example, recent work by [Veitch, D'Amour, Yadlowsky and Eisenstein](https://arxiv.org/abs/2106.00545) considers spurious correlation detection using tools from causality, where they distinguish different model features as being independent to changes in any latent factor that may introduce causal (or anti-causal) links to the labels. The authors herein may want to follow up on that line of work to characterize the 'easy' features they detect directly via prediction depth.

While the claims of novely may be modest, it is clearly presented and may open up further work to understand shortcut learning through spurious correlations.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper "SHORTCUT LEARNING THROUGH THE LENS OF EARLY TRAINING DYNAMICS" introduces a distinction between spurious correlations and shortcuts, where shortcuts are defined as "easy to learn" by some model. Through a few toy experiments as well as real-world datasets (e.g. chest X-ray), the authors then show that models preferably learn easy features (as identified via prediction depth, an existing technique that trains linear classifiers at each layer to identify the earliest layer after which an example is consistently correctly classified by all subsequent layers). The authors connect their findings to the existing concept of V-usable information and argue that looking at accuracy alone is insufficient to identify shortcuts. The main finding is summarized in this sentence from the paper: "experiments demonstrate how a peak located in the initial layers of the PD plot [prediction depth] should raise suspicion, especially when the classification task is challenging. Visualization techniques like Grad-CAM can further aid our intuition and help identify the shortcuts being learned by the model".

strength_and_weaknesses: **Strenghts:**
- paper connects a few different ideas - prediction depth / example difficulty, early training dynamics, V-information
- the question of which features models learn, and whether those are intended or shortcut features, is an important one

**Weaknesses:**
- inconsistent and problematic shortcut definition
- circular argumentation
- poor reproducibility
- limited novelty: paper never provides evidence of finding shortcuts that were not known in advance; theoretical proof - if my understanding is correct - is not surprising/unexpected at all (if one introduces more information, then a model can use more information)

clarity,_quality,_novelty_and_reproducibility: **Clarity:** Mosty clear. While the concept of undefined prediction depth is explained in appendix A.4, it might help to make it more clear in the main paper as well.

**Quality:** From a technical perspective, the experiments look mostly sensible, however there are two major concerns with the approach:
1. Shortcut definition. The authors introduce a distinction between shortcuts and spurious features, where shortcuts are only those features that are easy to learn. This definition contrasts with the existing definition of "Shortcut learning in deep neural networks" in which shortcuts are defined independently of "easyness". The approach of relying on "easy to learn" comes with a number of problems: now, one can no longer ask whether a dataset has a shortcut since the "easy to learn" definition is inherently tied to a specific model - while some features are easy to learn for some models, the same features can be hard / impossible to learn for others. (This is a consequence of the no free lunch theorem: averaged across all datasets the performance of any two models is equal.) Furthermore, the authors fall short of providing a convincing definition of "easy to learn", instead relying on a proxy, the prediction difficulty method. If shortcuts are defined as easy to learn, then the observation that shortcuts are learned first is trivial. It doesn't help that the authors aren't always using their own definition either: "First, we consider two datasets, one where the shortcut is easier than the core feature and another where the shortcut is harder." -> according to their definition, shortcuts are always easy to learn, so the existence of shortcuts which are harder than other features is inconsistent with the definition that the authors themselves advocate for. At points, the authors revert to visual intuition for defining "easy to learn", e.g. when stating that it is "visually intuitive that MNIST is easier than FMNIST, which is easier than CIFAR10". In summary, this goes to show that there are major problems when defining shortcuts relative to "easy to learn", especially without applying the necessary rigor in doing so consistently and with a convincing definition.
2. Circular argumentation: shortcuts are _defined_ as easy to learn. The authors then "empirically show that models suffer from shortcut learning only when the spurious features are easier than the core features." -> this is pretty much a natural consequence of the definition: if shortcuts are defined as easy to learn by a model, then it is no surprise that the easy to learn features are learned easily.

**Reproducibility:** Poor. The main proof is in the appendix. No code has been submitted, which is less than ideal from a reproducibility perspective. The appendix contains insufficient details on model training - e.g. it is simply stated that "We train a DenseNet121 model to detect chest drains relevant to Pneumothorax using these annotations." without providing any further details on the training (e.g. hyperparameters). The same is true for the models used in the main paper. I would encourage the authors to either submit code, or state why this is not possible; and additionally provide enough details that the findings can be reproduced without any trial and error. Typically, this includes details on hard- and software (including version numbers), all hyperparameters, links to existing training scripts that were used, etc.

**Novelty:** Limited. The paper starts with very simplistic settings (binary classification) and goes on to more realistic datasets. However, in all cases the shortcuts are known beforehand or even introduced by the authors themselves, thus it is unclear whether the method helps in finding novel shortcuts that were not known in advance. Finding a needle in a haystack is much easier if one knows the location beforehand. Furthermore, there is an open question regarding the usefulness of the model - if GRAD-CAM can show the shortcut, why do we need an additional method on top? Furthermore, the proposition (connection to V-information) seems close to obvious if I'm not mistaken: if one introduces more information, there's more information for the model to pick up on.

summary_of_the_review: The paper introduces a new definition of shortcuts which is tied to the notion of "easy to learn", without providing convincing arguments / a clear model-independent definition of easy to learn features. The authors show that the existing method of "prediction depth" can be useful in identifying potential shortcuts. While the paper is mostly clearly written, there are major problems with the definition of shortcuts (inconsistency), circular argumentation, poor reproducibility and limited novelty.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper analyzes shortcut learning and spurious features, studying training dynamics and drawing connections to the concepts of prediction depth (PD) and V-usable information, concluding that only those spurious features which are “easier” than core features can be considered shortcuts, and moreover that the learning of these shortcuts happens early in training and can be observed via PD and the previously established visualization method known as Grad-CAM.

Generally, the reviewers found the paper to be mostly clearly written and the experiments to be well-designed. On its over all merits, however, the reviewers offered split opinions, with some reviewers appreciating the idea of using the PD in order to identify potential spurious correlations, and others raising concerns with some of the main definitions.

In the lengthy discussion, numerous questions arose, such as easy versus hard tasks and the shortcut definition, among many others. There was no single common concern, but a host of smaller concerns adding up to an overall lukewarm impression across the board. No reviewers felt that the current version of this paper had strong enough results to have a significant impact on the field. It seemed instead that the community would be better served with a version of this paper that incorporated further revisions, clarifications, and honing of the message.
justification_for_why_not_higher_score: While this paper presents some novel and interesting ideas for analyzing and understanding spurious correlations and shortcuts, the overall message is not compelling enough to merit publication at this time. Improvements to the storyline, definitions, breadth of experimentation, and other suggestions made by the reviewers could make this a more impactful paper.
justification_for_why_not_lower_score: N/A.
Decision: Reject