There Are No Shortcuts to Anywhere Worth Going: Identifying Shortcuts in Deep Learning Models for Medical Image Analysis

Authors: christopher  boland
keith a goatman
sonia  dahdouh
Venue: MIDL_io_2024_Conference
Published: 2024-01-31 19:50:18

Abstract: Many studies have reported human-level accuracy (or better) for AI-powered algorithms performing a specific clinical task, such as detecting pathology. However, these results often fail to generalize to other scanners or populations. Several mechanisms have been identified that confound generalization. One such is shortcut learning, where a network erroneously learns to depend on a fragile spurious feature, such as a text label added to the image, rather than scrutinizing the genuinely useful regions of the image. In this way, systems can exhibit misleadingly high test-set results while the labels are present but fail badly elsewhere where the relationship between the label and the spurious feature breaks down. In this paper, we investigate whether it is possible to detect shortcut learning and locate where the shortcut is happening in a neural network. We propose a novel methodology utilizing the sample difficulty metric Prediction Depth (PD) and KL divergence to identify specific layers of a neural network model where the learned features of a shortcut manifest. We demonstrate that our approach can effectively isolate these layers across several shortcuts, model architectures, and datasets. Using this, we show a correlation between the visual complexity of a shortcut, the depth of its feature manifestation within the model, and the extent to which a model relies on it. Finally, we highlight the nuanced relationship between learning rate and shortcut learning.

Main paper: 
 introduction: 
 Shortcut learning is the phenomenon where deep neural networks rely on superficial or irrelevant data features, termed 'shortcuts' (Ahmed et al., 2022) . Such features are easy to learn but do not generalize beyond the training data and performance degrades postdeployment (Cohen et al., 2020; Pooch et al., 2020; Zech et al., 2018; Beery et al., 2018) . Models can rely on these spurious features even if they are less predictive than those that are clinically relevant (Shah et al., 2020) . In many cases, the shortcut features may not be recorded in the dataset, making it challenging to identify when a model relies on them (Oakden-Rayner et al., 2020) . This poses a significant challenge to clinical deployment of medical image analysis systems.Unfortunately, shortcuts are common in medical image datasets. For example, Nauta et al. (2021) found that models trained using a popular skin lesion dataset exploit the color calibration charts to make predictions, since they are only present in the malignant cases. Similarly, patients suffering a pneumothorax often have chest drains installed that are visible in chest x-rays; classification models can use these to make predictions (Jiménez-Sánchez et al., 2023) . Even subtle features, such as small differences in the image acquisition process or patient age, can introduce shortcuts (Ahmed et al., 2022; Brown et al., 2023) . Our understanding of shortcut learning remains limited. Recent works propose targeting features learned in the earliest layers to mitigate shortcuts (Murali et al., 2023; Dagaev et al., 2023) . This overlooks shortcut features that may manifest deeper within the model.This paper presents three main contributions. Firstly, we propose and demonstrate the efficacy of a novel methodology to localize learned shortcuts to specific model layers. We utilize a metric called Prediction Depth (PD) (Baldock et al., 2021) and KL Divergence to identify changes in the model's behavior after adding a shortcut to the training data. We show that our approach can consistently and effectively localize learned shortcuts to specific model layers across various architectures, shortcut types, and natural or medical image tasks. Secondly, we evaluate the relationship between a shortcut's complexity, how deep in the model it manifests, and its potential for harm. Finally, we investigate the impact of training condition hyperparameters, such as learning rate, on shortcut learning. 
prediction depth: 
 Prediction depth (PD) quantifies example difficulty by the number of layers a model needs to finalize its prediction. k-NN classification probes provide a per-layer prediction for each sample. PD is defined as the layer after which all subsequent probes make the same prediction. More challenging inputs require disambiguation of more complex features and would be expected to have deeper PD (Baldock et al., 2021) . Murali et al. (2023) connect PD to shortcut learning and the simplicity bias of deep neural networks (Shah et al., 2020) , demonstrating that shortcuts are harmful when they are simpler than the relevant features.KL divergence is a measure of dissimilarity between two probability distributions. This is expressed in Equation (1) where X is a collection of events; P (x) is the probability of event x occurring in distribution P ; and Q(x) is the probability of x occurring in distribution Q.EQUATION3. Method 
shortcut localization: 
 We localize the network layer specific to learned shortcuts based on KL divergence and PD.To illustrate this, we train two binary classification models. One is trained on a dataset with no known shortcuts and the other on a dataset with a known shortcut which is perfectly correlated with one class. Both models are evaluated on the same test set, where the shortcut is balanced between both classes. We fit k-NN probes, with k=29 to avoid ties, to each convolutional layer. This provides a per-layer classification for each sample, from which we obtain a PD distribution over the entire set.(b) (a) (d) (c)To isolate specific layers where the shortcut manifests, we calculate the KL divergence between the two PD distributions (Equation (1)). Each "event", x, is a possible prediction depth, and P and Q are the distributions of the shortcut and clean models, respectively. Rather than summing the divergence of every PD we consider each individually. The asymmetric nature of KL divergence means events that are more likely in P than Q lend more weight to the divergence than those more likely in Q than P . This emphasizes the layers most influenced by the shortcut. For this reason, it was chosen over other metrics, such as the square difference. We isolate the layers that exhibit the most significant divergence (the 95 th percentile). GradCAM heatmaps (Selvaraju et al., 2017) , provide an intuition of the shortcut's influence at these layers (Figure 1 ). While the link between saliency techniques and model outputs is debated, and they are vulnerable to adversarial attacks (Zhang et al., 2022) , GradCAM has been shown to be valuable for interpreting model decisions due to its sensitivity to model parameters and training data labels (Adebayo et al., 2018) .To probe how different types of shortcut influence models, we generate augmented datasets introducing three types of synthetic shortcuts (Figure 2 ):1. Background: Leveraging image background as a spurious signal (only Waterbirds).2. Simple object: Introducing small, discrete objects to the image. We test two variants: • Constant location: The object appears in a fixed spot.• Random location: The object location varies among images.Inspired by real-world objects that commonly trigger shortcuts (e.g. chest drains in pneumothorax-positive X-ray images) we introduce a vertical, semi-opaque curved line, centered in the image. To simulate real-world stochasticity we introduce random rotations and allow for variation in its location.These shortcuts vary in visual complexity: background shortcuts offer a ubiquitous, lowlevel texture cue; more complex shortcuts introduce higher-level features that require more effort to disambiguate.We test our approach to shortcut localization on three datasets: Waterbirds, a subset of CheXpert, and ISIC 2017 (Sagawa et al., 2020; Jiménez-Sánchez et al., 2023; Codella et al., 2018) . Waterbirds introduces a background shortcut and is commonly used for evaluating spurious correlations and out of distribution (OOD) performance. Both CheXpert and ISIC are popular public medical image datasets that have been studied in the context of bias and shortcut learning (Brown et al., 2023; Nauta et al., 2021) . For each dataset, we consider a binary classification task, apply class balancing by undersampling, and use the train/test/validation splits provided by the authors. For CheXpert, we predict the presence of a pneumothorax and with ISIC we classify lesions as benign or malignant. All images are resized to 224 × 224 and random rotation (0 to 10°around the image center) is applied. Four popular model architectures are used: ResNet-34 (He et al., 2016) , DenseNet-121 (Huang et al., 2017) , InceptionNet (Szegedy et al., 2016) , and VGG16 (Simonyan and Zisserman, 2014) . SGD and AdamW optimizers are tested (Loshchilov and Hutter, 2017) with three learning rates: 0.01, 0.001, and 0.0001 with early stopping on the validation loss. 
experiments: 
 To validate our approach to shortcut localization, and enhance our understanding of shortcut learning, we conduct three experiments. Firstly, we show that adding a shortcut to training data significantly alters the mean PD of the test set. Secondly, we demonstrate that the divergence of the shortcut model's PD distribution from the clean model's can identify the layers where the shortcut most influences model behavior. Thirdly, we explore the relationship between the visual complexity of shortcuts, the depth at which they are learned, and the harm they cause (how much they reduce performance and hurt robustness).We hypothesize that including a shortcut in the training data causes a change to the PD distribution. To validate this, we use the Waterbirds dataset (Sagawa et al., 2020) and focus on the binary classification task Waterbird/Landbird. We generate a new, class-balanced, "clean" dataset without the background shortcut. Following Section 3.2, we augment this dataset with synthetic shortcuts (Figure 2 ), producing three new train, validation, and test sets. Models are trained with an SGD optimizer at a learning rate of 0.01. We compare the test set PD distribution of each shortcut model with the clean benchmark (Figure 3 ). When making this comparison, the clean model is evaluated on the test set featuring the shortcut. Using Welch's t-test (that does not assume equal distribution variances), we demonstrate that there is a statistically significant change in the mean PD of the test set when a shortcut is introduced to the training data (p ≪ 0.05) (see Table 1 ). Knowing that shortcuts influence the PD, we demonstrate that a learned shortcut can be localized to particular layers using KL divergence analysis. We test this on the Waterbirds dataset and two medical image datasets (CheXpert and ISIC). Similar to Section 4.1, we create three shortcut datasets each for CheXpert and ISIC, using the original datasets as "clean" controls. For each model trained on the shortcut datasets, we isolate the layers that exhibit the most significant divergence from the PD of the clean model (Figure 4 ). The GradCAM heatmaps showcased in Figure 1 illustrate that shortcuts impact predictions for samples identified in layers with the greatest divergence. This is consistent across all datasets and shortcut types. Figure 4 and Figure 5 provide a more detailed illustration of our approach applied to ISIC and CheXpert examples. We consistently find that the layers with the highest divergence in the model corresponded to samples whose predictions were influenced by the presence or absence of shortcuts. By utilizing KL divergence, we pinpoint the layers where the shortcuts were most pronounced, even in cases where a simple inspection of the PD distribution might not have been enough. For example, in Figure 4 , the shortcut model's largest peak in the PD distribution is in layer 0. However, by using KL divergence, we were able to identify that the layer most impacted by the shortcut was actually layer 27. Further inspection of samples with a PD of 27 revealed that the model was indeed leveraging the shortcut in order to make its predictions.We consider the relationship between a shortcut's visual complexity, the depth at which it is learned, and the harm it causes. We anticipate that more complex shortcuts will require more effort to disambiguate and will, therefore, skew the test set PD distribution deeper. To evaluate this, we take our shortcut models from Section 4.2 and compare their performance, divergence from the benchmark, and mean PD. Furthermore, we explore the impact of learning rate on the potential harm caused by shortcuts.Prior work suggests that shortcuts manifest in the earliest layers of a model, and can be mitigated by targeting these layers (Murali et al., 2023; Tiwari et al., 2024) . In Figures 4 and 5 we demonstrate that shortcuts are not constrained to the early layers. More visually complex shortcuts tend to manifest deep in the model. We see that the red square (random) and complex object shortcuts have a significant influence on the 27 th and 26 th layers respectively. Inspection of samples classified at these layers shows that the shortcut is used for their prediction. We also see a peak in the earliest layer, where we see more samples that do not contain the shortcut are classified. Mitigation that focuses purely on the earliest layers may fail to address more visually complex shortcuts.Inspection of test-set PD distribution of models trained on our datasets highlights clear differences in how shortcuts manifest within the same model. Our findings, illustrated in Table 2 , indicate that as shortcut complexity increases, performance cost decreases. The simplest square shortcut consistently reduced AUC by 10-15% from baseline across all learning rates, compared to the complex object shortcut (3-7%). This suggests that simpler shortcuts are typically more harmful to model performance, in line with the "simplicity bias" of neural networks (Shah et al., 2020) . We also found that reducing the learning rate improved AUC by approximately 8% when the learning rate decreased from 0.01 to 0.001 across all shortcuts. However, even with reduced learning rates, shortcuts in the training data still result in a drop in AUC of 3-11% in the best case (Table 2 ). 
discussion & conclusion: 
 We have demonstrated a novel methodology to localize shortcuts to specific model layers using PD distributions and per-layer KL divergence analysis. The approach identifies both the samples classified using shortcuts and the network layer responsible for the shortcut classification. The results are consistent across models, datasets, and four synthetic shortcuts. With this, we empirically demonstrate the relationship between visual complexity of shortcuts, the depth of their manifestation in neural networks, and their potential for harm. We find more complex shortcuts manifest deeper in the network than simpler shortcuts, but that simpler shortcuts have greater potential for harm. Careful choice of training conditions can help to reduce the influence of shortcuts, but changes to the optimizer type or learning rate were never enough to completely mitigate the shortcuts in our experiments.Our investigations rely on access to both clean and shortcut-influenced data. Though not always practical, this approach is useful in situations where a benchmark model is available, such as detecting site-specific shortcuts in multi-site data or testing new training data for shortcuts when updating models. We leave removing this constraint of a clean baseline to future work. Looking ahead, we aim to explore how to build on this work to not just identify shortcuts, but to mitigate their influence. To gain further insight into the clinical applicability of this work, we need to investigate real-world shortcuts using highly curated medical image datasets. This is reserved for future work, but our findings here represent progress towards rigorous evaluation of model safety for clinical deployment.  
 
 Appendix: Figures 9-11 show the test set PD distributions for models trained with the AdamW optimizer. Like the models trained with the SGD optimizer, we observe some similarities in the effect of the shortcut across different network architectures. However, with the AdamW optimizer, the impact of the shortcut varies more between architectures and tasks than it did with SGD. Moreover, we notice that some shortcuts have a lesser influence at a higher learning rate (0.01) with the AdamW optimizer, compared to SGD. The differences in the impact of shortcuts between models trained on AdamW and SGD are interesting and motivate further investigation in future work. However, we do not conduct a significant investigation of this here. There Are No Shortcuts to Anywhere Worth GoingHere we will elaborate on the correlation between test set PD and model performance, and the correlation between divergence from clean model and performance. Figure 14 displays the correlation between all these values for a DenseNet-121 trained on the CheXpert shortcut datasets and Figure 15 shows the same analysis for a ResNet34 model. We have separated the analysis by the learning rate used during training. We observe a positive correlation between mean test set PD and AUC, and a negative correlation between AUC and divergence from the clean baseline across all learning rates for each model. This suggests that models with deeper PD generally outperform those with shallower PD. This supports our results in Table 2 , where we see that the perceptually simpler shortcuts, which cause the most significant decline in mean test set PD, also cause the largest drop in performance from the clean baseline.  


Decision: This paper received mixed reviews with two very positive reviewers. Ultimately all reviewers recommended acceptance, as the more critical reviewer improved their initial rating to borderline accept. I had a closer look and also believe that the authors addressed the raised concerns adequately. 

I recommend acceptance of the paper and think it will raise interest in the MIDL community.

However, I would like to point out some template violations (e.g. figure captions, side-by-side figure and table), and urge the authors to adhere to the template rules in the future.

Reviews: review: paper_summary: paper_summary: The authors present an analysis of shortcut learning in image classification. Their study includes experiments on bird, chest x-ray pneumothorax, and skin lesion classification. The experiments were conducted with three different neural network architectures, three learning rates, and two optimizers. Shortcuts were either naturally present (e.g., background in water bird classification or drains on chest x-rays) or added in a controlled setup.
The study shows that through PD and KL-divergence analysis, one can locate the layers most prone to a given shortcut, having important implications for mitigation strategies. The authors demonstrate that more complex shortcuts can be located deeper inside the network than simple confounders (i.e., red squares in fixed locations).
main_review: main_review: I am fully satisfied with the replies and edits by the authors and recommend accepting this contribution. The revision has further improved the text and readability and how has added relevant references.
strength_weakness: strength_weakness: - No significant weaknesses in my opinion.
- Since you tested many different setups, I would appreciate it if you could elaborate when describing the findings if you observed common patterns for all tested networks and optimizers (e.g., on the PD vs. shortcut complexity). I am aware of the length constraint; consider, e.g., adding a table showing the results for all experiments or plots for the network architectures you do not showcase in the main text as a supplementary.
questions: questions: - Were the findings the same for the two tested optimizers and for the three tested neural networks?
paper_summary: paper_summary: This paper aims to detect whether shortcut learning is happening in medical image tasks, identify in which specific layers the shortcuts are happening, and propose strategies to mitigate them. Their method relies on prediction depth (PD) and KL divergence to identify changes in the model behaviour after adding a shortcut to the training data. They present results across different type of shortcuts, deep model architectures and datasets. They demonstrate a relationship between the visual complexity of a shortcut, the depth at which its features manifest within the model, and the degree to which the model depends on it.
main_review: main_review: Although I believe the introduction and motivation are still rather short, I understand the limitation of pages. I agree that the structure of the manuscript and the readability of the text have been improved.
strength_weakness: strength_weakness: - The introduction and related work provide an insufficient framework for the work presented.
- The choice of metrics (PD and KL) seems arbitrary. The authors provide references and equations but there is no background about why those metrics make sense for this task.
- The use of heatmaps as methods for explainability is not fully reliable. There is a lack of discussion regarding their limitations. Also, the criteria for selecting the heatmaps displayed in figures is not mentioned.
- No code available for transparency and reproducibility of results.
questions: questions: 1. The structure and writing of the text could be improved:
    - Introduction is very short, the paper could benefit from further motivation and clearer connections with related works.
    - Some relevant references are missing e.g. Oakden-Rayner, Luke, et al. "Hidden stratification causes clinically meaningful failures in machine learning for medical imaging." *Proceedings of the ACM conference on health, inference, and learning*. 2020.
    - Some important information about the methodology is missing in the abstract: Prediction Depth (PD) and KL divergence metrics.
    - Details about experimental setting appear in “Section 3 Method” and “Section 4: Experiments”. This can be confusing for the readers. I would suggest improving the structure of the sections. A suggestion could be “Section 3: Methods”, “Section 4: Experimental Setting”, “Section 5: Results”.

2. Choice of prediction depth and KL divergence. Further elaborate why these metrics are chosen for your methodology. Any related work worth mentioning?

3. Explainability methods are briefly mentioned in Section 3.1. Justify your choice of heatmaps (GradCAM), and discuss the limitations of saliency maps, see for example Zhang, Jiajin, et al. "Overlooked trustworthiness of saliency maps." International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022.
paper_summary: paper_summary: The paper proposes to investigate the feasibility of localizing shortcuts in image classification models. Several datasets are artificially created by adding different types of shortcuts to them. Models are then trained with clean datasets and with datasets containing shortcuts. Prediction depth (PD) is evaluated for each sample of a test set in which the shortcuts are added in balanced way between classes. KL divergence is further used to assess the different PD distributions (with and without shortcuts), to identify the model layers the most impacted by the shortcuts. Finally, the authors show Grad-Cam saliency maps for these layers demonstrating the use of the shortcut in the prediction.
main_review: main_review: The authors carefully answered my comments and added some analyses to strengthen the results, which allowed me to confirm my preliminary rating. The paper is of good quality and tackles a very interesting topic.
strength_weakness: strength_weakness: The authors focused on identifying the location of the shortcut within the model but did not comment much about which samples from the test set are classified correctly or not/using the shortcut or not. This would be a very interesting complementary analysis to add or at least to mention in the discussion (see “Questions to address in the rebuttal”).
questions: questions: I am wondering what is the difference between the PD distribution of test samples with shortcuts and without shortcuts for each class when classified by the shortcut model. For example in the sentence “Further inspection of samples with a PD of 27 revealed that the model was indeed leveraging the shortcut in order to make its predictions.” Do the authors mean that the samples with a PD of 27 were the ones with the shortcut in the test set?

Is there any correlation between the prediction depth of a sample and if it is classified correctly or not?

Comments: title: Thank you to the reviewers
Expanding on PD of shortcut vs no shortcut samples, and correlation between PD and performance
Updates to the structure and writing, justifying choice of PD and KL divergence, and discussing limitations of explainability methods
Commonality of findings across networks and optimizers, and addressing reference to mitigation in the introduction
Gentle reminder of rebuttal deadline
comment: Thanks to the authors for incorporating the feedback. Although I believe the introduction and motivation are still rather short, I understand the limitation of pages. I agree that the structure of the manuscript and the readability of the text have been improved. I have no further questions.
Thank you for your responses, I am satisfied with the replies and have no further comments.
We are grateful to all of the reviewers for their very helpful feedback. We are highly encouraged that they find our proposal to be a novel approach to gaining insight into an important problem in medical image analysis tasks and that each of the reviewers found our experiment design to be thorough and well-evaluated. Answers to every reviewer will be given separately, with references to the modifications made in the paper when appropriate. To note, we made a slight modification to the original ordering of the authors.
We thank the reviewer for the time taken to review the paper and provide feedback. We are pleased that they found our approach to be novel and our experiment design and analysis to be robust and well-explained. We appreciate the feedback the reviewer provides to improve and expand upon our analysis to improve the robustness of the paper. All changes to the text are highlighted in red.

> **Figure 1 is only referenced after Figure 4 and should appear later in the paper**

References to figures have been adjusted so that the order of the figures makes logical sense.

> **I am wondering what is the difference between the PD distribution of test samples with shortcuts and without shortcuts for each class when classified by the shortcut model. [...]**

We thank the reviewer for highlighting this analysis, which is currently missing from our paper. We have now included in the appendix Figures 12 and 13, which shows the PD distribution for each of the shortcut models trained on ISIC and CheXpert. We distinguish between the samples that feature a shortcut and the samples that do not. This figure helps to visualise the peaks specific to the shortcut features. 

> **Is there any correlation between the prediction depth of a sample and if it is classified correctly or not?**

We find that PD positively correlates with performance metrics (AUC and accuracy). Our experiments find that models with deeper average test set PDs perform better than those with shallow test set PDs. We have added Figures 14 and 15 to the appendix that show the correlation matrices for two different models at different learning rates on the CheXpert dataset. We show that we tend to see a positive correlation between PD and AUC and a negative correlation between PD and divergence from a clean baseline.
We are very grateful that the reviewer has taken the time to read and provide feedback on our paper. We are pleased that the reviewer finds our approach novel and appreciates the rigour of the experiment design. We thank the reviewer for the suggestions that they have provided to improve the paper’s structure, readability, and robustness. All changes to the text are highlighted in red.

>  1. **The structure and writing of the text could be improved:**
>      - **Introduction is very short, the paper could benefit from further motivation and clearer connections with related works.**

We have added detail to our introduction (Section 1) to connect our paper to other work investigating shortcut learning. In particular, we highlight two papers that discuss the concept that shortcuts that are harmful to the model are “easy” features with a correlation in the training data, and can be learned in the earliest layers of a network. This connects to our investigation of where, within a neural network, shortcuts of different visual complexities manifest. 

> 1. **[...]**
>    - **Some relevant references are missing e.g. Oakden-Rayner, Luke, et al. "Hidden stratification causes clinically meaningful failures 
in machine learning for medical imaging." Proceedings of the ACM conference on health, inference, and learning. 2020.**

We thank the reviewer for highlighting references relevant to our paper. Oakden-Rayner, Luke, et al. "Hidden stratification causes clinically meaningful failures in machine learning for medical imaging." Proceedings of the ACM conference on health, inference, and learning. 2020, highlights the important issue of hidden stratification. This is particularly relevant to shortcut learning, where the dataset does not label shortcut features. Models can still learn to rely on them, but their identification and mitigation become more challenging. This is an important piece of work to discuss, and we have included it in Section 1.

> 1. **[...]**
>    - **Some important information about the methodology is missing in the abstract: Prediction Depth (PD) and KL divergence metrics.**

We agree that this information should be included in the abstract and have added it.

> 1. **[...]**
>    - **Details about experimental setting appear in “Section 3 Method” and “Section 4: Experiments”. This can be confusing for the readers. I would suggest improving the structure of the sections. A suggestion could be “Section 3: Methods”, “Section 4: Experimental Setting”, “Section 5: Results”.**

We thank the reviewers for highlighting weaknesses in the structure of the paper’s writing. We have identified where there is overlap or repeated explanations regarding the experimental setup in Sections 3 and 4. Details that are consistent across all experiments have been moved to Section 3, and the additional experimental detail in each subsection of Section 4 contains information specific to that experiment. This information is discussed immediately before the results to provide a clear flow for readers looking at specific sections. 

> 2. **Choice of prediction depth and KL divergence. Further elaborate why these metrics are chosen for your methodology. Any related work worth mentioning?**

In Section 3.1, we motivate our choice for KL divergence by explaining that its asymmetric nature emphasises the layers of the model most influenced by the shortcut’s introduction to the training data. This is beneficial over other metrics such as squared difference. 
The choice of Prediction Depth is motivated by its use as a measure of sample difficulty, connecting to the simplicity bias of deep learning models and shortcuts simplicity over task-specific/clinically relevant features. We added text to Section 2.1 to clarify this connection and connect to literature that connects shortcut learning and PD. 

> 3. **Explainability methods are briefly mentioned in Section 3.1. Justify your choice of heatmaps (GradCAM), and discuss the limitations of saliency maps, see for example Zhang, Jiajin, et al. "Overlooked trustworthiness of saliency maps." International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022.**

We agree that saliency maps have limitations. In Section 3.1, we have included a short discussion of their limitations and literature supporting the use of GradCAM to understand the visual features considered important to a model’s prediction.
We would like to thank the reviewer for their positive feedback regarding our paper and their helpful suggestions for improvement. We are delighted that the reviewer believes we make a valuable contribution to a pressing issue in medical AI, and that they are pleased with the experimental design and analysis. We appreciate their suggestion to expand upon the discussion of the results to improve the paper. All changes to the text are highlighted in red.


> **Were the findings the same for the two tested optimizers and for the three tested neural networks?**

Our results are generally consistent across neural network architectures, and our approach's efficacy to localize learned shortcuts is consistent in each case. We have added to the Appendix (Figures 6-11) visualisation of the PD distributions for each dataset across different architectures, learning rates, and optimiser types. We find that each shortcut appears to have a similar influence on each model. We do notice differences in how the shortcuts manifest across the two optimizer types. Some shortcuts appear to have a lesser influence on PD at high learning rates when training with AdamW, while the opposite is true for SGD. While we don’t perform an extensive investigation of this in this paper, it is an interesting insight to motivate future work.

>  **I the abstract, you mention that you explore shortcut mitigation strategies. I agree that your work has important implications for mitigation strategies, but do not see the exploration part in the text.**

We agree that this sentence misrepresents the insight gained from our work. Here, we were referring to our experiments with different optimizer types and learning rates and how their selection can affect the harm caused by shortcuts. However, we understand that referring to this as "mitigation" is misleading, and we apologize for any confusion this may have caused. We have removed this sentence from the end of the abstract.
Dear authors,

This is just a gentle reminder that the rebuttal deadline is coming up. I encourage you to take this opportunity so you can engage with the reviewers in the upcoming discussion phase.

Best wishes,
AC